Hi!
While re-learning async (great stuff btw!) I just noticed that the -N options, controlling the number of physical threads to be used by the RTS, is not listed in the manual:
https://downloads.haskell.org/ghc/latest/docs/users_guide/runtime_control.html
It’s only mentioned by other flags, but not documented per se, we can only read about it in various stackoverflow comments.
Isn’t it here?
https://downloads.haskell.org/ghc/latest/docs/users_guide/using-concurrent.html#parallel-options
Hmmm I never had to debug a problem like that. Can you tell us more about the computation?
Is it embarassingly parallel? How long do threads live? Is FFI involved?
Thanks for following up!
after a little printf debugging I saw I was simply feeding it too few tasks at a time. Completely parallel, no data dependencies not even MVar/transactional, no FFi, just rejection sampling a lot of random structures under different conditions and streaming to disk.
Now it’s burning rubber on all 64 cores, happy times.