Remove the caching from the CI workflow
I don't trust it. Compare these two runs:
The second is on the parent commit of the first, the commit itself is a
no-op, and yet the derivations
they build are not the same! Doing the same locally, I get a) a
different derivation, but b) the same derivation for both.
I conclude that what the CI is doing is questionable, the cache seems
like the most likely source of pollution.
I suspect whatever is causing this was also responsible for the no-op PR
spending a lot of time building tons of stuff.