Sorry for not being around here recently. In the ququart case it would take more time per j but there's also less j required. Either way, it's still scalable. If you have a more general way to implement controlled unitary gates then it is possible to make the circuit for multiplying that number with just knowledge of the N that is the mod and the value without significant classical overhead.