I use -j a lot. But higher is not always better. It controls how many modules are built in parallel. This is infra-package module level parallelism. (Where the dependencies permit. -j4 is my default (it’s that in nix as well btw). Cabals -j flag controls package level parallelism (again if permitted by the dependency graph).
This means passing -j4 —ghc-option=-j4 to cabal can lead to 16 modules being compiled at the same time. Matthew Pickering recently wrote about the addition of -jsem which tries to control the quadratic explosion from the -j combination of the compiler and build tool. And I believe there were some other in-depth details on the -j flag. Maybe you can find that post on discourse here.