The new subword logic in ghc, was not very well covered by the test-suite. This is being rectified with a new quickcheck based test tool: Ben Gamari / test-primops · GitLab
We do see a pattern here right? The release candidates aren’t getting the necessary testing they need. The first release that’s cut gets some testing because people aren’t hesitant to test a “release candidate” anymore, and the .1 ends up being effectively the -rc1.
I do agree on the PSA though. But then again I don’t even read the Cafe O_O.
I suspect that a lot of people don’t know how head.hackage can be used to work around dependency issues with early stage GHC releases. Maybe, if this was publicized a bit better, more people would give it a try.
Hi, i’ve had heard about hackage.head but i had no take a look, it is great and agree it should be more publicized.
We do see a pattern here right? The release candidates aren’t getting the necessary testing they need.
Today i had to ping for the third time to a maintainer and email for the second time another one to try to make 2 hls upstream packages have support for — ghc-9.0.1!!
So not sure if test ghc release candidates would be easy, at least for project living in the leaves of the dependency graph.
We do see a pattern here right? The release candidates aren’t getting the necessary testing they need.
Indeed. Running tests already present in GHC source tree could be a good first step, I think. E. g., test suite of text submodule clearly demonstrates the issue.
text sadly doesn’t have the same arcane test-infra that ghc mandates for testing
IMO, ghc shouldn’t impose the how of test-suites onto packages; we have cabal files to describe test-suites. We should just use those. This is an uphill battle though.
That’s quite far fetched. If it was just for the calling convention we wouldn’t see these issues. It’s due to the proliferation of subword primops. Those are arguably enabled by the machinery we had to add to properly support foreign calls on AArch64-darwin.
A more accurate spin would be that apple once again forced our hand to be less sloppy (not everything is a WORD!—in a typed language after all) and that this nudge allowed us to finally have enough machinery in place to make subword primops feasible.
We simply did not have good tests for subword primops. This would affect AArch64-Linux as well I believe. That this shows up on AArch64-darwin first should tell us something about platform popularity.
IMO, ghc shouldn’t impose the how of test-suites onto packages; we have cabal files to describe test-suites. We should just use those. This is an uphill battle though.
It’s probably non-trivial to integrate test suites of boot packages into native GHC tests, but it should be reasonably simple just run them as a separate build step, once ghc-stage2 and cabal are available, isn’t it? I mean, what’s the point to maintain head.hackage if not to run tests?
Now I’m a bit afraid to answer as an official voice, I’m not. So please let me prefix with: this is my view on things.
head.hackage is primarily a way to fix packages ahead of upstream to test compilation (and maybe testsuites) against them. It’s mostly an aid in ghc development. We do run head.hackage tests occasionally, but they are not part of regular CI runs, and I’m not sure how much attention anyone pays to them really
I have started working on full blast tests for compilers. If this was further along, we might have been able to test earlier compilers faster. It’s however just a side project Hamish and I work on occasionally. I wish we’d had something like this in better shape. Quite frankly the resource use is intense, and we’ll need to find a better solution on reporting for regressions, …
We’ll likely run this against 9.2.1 soon’ish.
To go back to your earlier point, yes it would possible to set up some smaller tests for shipped packages. It just needs someone to actually write that CI job. Especially once we have building and testing separated into two distinct CI stages. GHC dev is seriously short on manpower. I’d be happy to provide any form guidance, mentoring and assistance!