haskell.nix - 13 commits this week

Home / Input Output / haskell.nix
13 commits this week Jun 06, 2026 - Jun 13, 2026
hamishmack · Sun, 07 Jun 26 11:03:25 +0000 · haskell.nix
v2 builder: compose the slicing repo & cabal.project at build time (#2517)
* v2 build-time composition (step 1): emit per-package fragment in $out

First step of moving the v2 slicing-repo / cabal.project assembly out of
the Nix evaluator and into each slice's build phase, so a consumer never
has to walk its transitive dependency graph in Nix.

Each slice now writes its OWN contribution into $out (additive; not yet
consumed):

  * repo-frag/          — this package's source tarball + index .cabal
  * nix-support/v2-frag — the six per-package cabal.project blocks
                          (flags / ghc-options / configure-options /
                          program-options / documentation / extra-lib-dirs),
                          its constraint pin line, pkg-name, and sublib seeds
  * nix-support/lib-dep-slices — flattened lib-dep closure pointer
                          (built the same way as transitive-deps)

All values are computed once for THIS package (O(1) per slice), reusing
the existing per-package block functions.  Verified additive: the Stream
leaf slice's unit-id is unchanged (Strm-0.4.7.2-a9e4dded).

Steps 2 (compose at build time) and 3 (drop the Nix-side closure walks)
follow.

* v2 build-time composition (step 2a): assemble slicing repo at build time

When a slice carries a `v2Fragment`, assemble its hackage slicing repo
in the build phase from the per-package `repo-frag/`s — this package's
own source (passed directly, since its own fragment isn't built yet)
plus every dep's fragment reached through the same `transitive-deps`
closure walk used to compose the store.  Replaces referencing the
Nix-built `localRepo` for these slices.

cabal hashes package source + .cabal descriptions (not the index bytes
or the repo path), so the content-equivalent repo keeps unit-ids
identical: the Stream leaf slice still builds as Strm-0.4.7.2-a9e4dded.

(The Nix `slicingRepo` is still computed but unused for these slices;
it's removed in step 3 along with the other Nix-side closure walks.)

* v2 build-time composition (step 2b): assemble cabal.project closure sections at build time

Compose the dependency-closure-derived parts of each slice's
cabal.project in the build phase, from per-package fragments, instead of
in the Nix evaluator:

  * extra-packages:     self + lib-dep-closure pins
  * the six per-package block groups (flags / ghc-options /
    configure-options / program-options / documentation / extra-lib-dirs)
  * allow-boot-library-installs (gated on closure boot-libs)
  * constraints:        self `any.<pkg> source` + lib-dep-closure pins

The Nix side now emits only the LOCAL/global skeleton
(`localCabalProject`) and each package's OWN fragment; the build script
appends the composed sections.  cabal hashes content, not field order,
so we emit local-then-closure and verify content-equivalence with an
order-insensitive diff against the (still-computed) Nix cabal.project —
empty for every slice tested.

Direct-deps-only in Nix (per design): the lib-dep / constraint closure
is seeded from this component's DIRECT lib-deps and DIRECT build tools
(`buildToolSlices`, not the Nix-walked `transitiveBuildToolSlices`).
Transitivity is accumulated at build time — each slice's emitted
`lib-dep-slices` folds in its own direct build tools' lib closures, so
following a dep's pointer reaches every transitive build tool's lib
closure without a Nix-side allDepClosure walk.

Two multi-sublib fixes (e.g. happy-lib's grammar/frontend/tabular, all
sharing pkg-name happy-lib):
  * repo index copy is no-clobber (`-n`) — duplicate identical .cabal
    across sublib fragments no longer hits the read-only first copy;
  * closure dedup is by pkg-name (`sort -k1,1 -u`), matching the Nix
    side's `sliceCanonicalNames` / `libConstraintPins` dedup, so a
    package's block / constraint / extra-packages entry appears once.

Verified content-identical (empty diff) + unchanged unit-ids on Stream
(leaf), code-page + network (hsc2hs), and happy (multi-sublib +
build-tool with reinstalled happy-lib).  No source-repo packages exist
in this plan, so source-repository-package block composition is deferred.

* v2 build-time composition (step 3): drop the Nix-side closure walks

The real slice now assembles its cabal.project + slicing repo entirely
at build time from per-package fragments, so the per-slice dependency-
closure walk no longer runs in the Nix evaluator:

  * baseSlice's preBuild (slicePreBuildV2) only stages local test/bench
    sources — it no longer interpolates the Nix-assembled `cabalProject`,
    so the six `sliceCanonicalNames` block-assemblies, `depConstraints`
    /`libConstraintPins`/`libConstraintClosure`, `extraPackages`,
    `sourceRepoBlocks` and `allowBootLibBlock` are no longer forced;
  * localRepo is null for these slices, so `slicingRepo` (and the
    `transitiveTarballs` recursion behind it) is no longer forced;
  * the build-time content-equivalence diff harness is removed (it was
    only needed while developing the composer), and the local skeleton
    is written via redirect rather than `cp` (which would otherwise
    create cabal.project read-only and break the closure `>>` append).

`checkAgainstPlan` / `docSlice` keep the full Nix `cabalProject` path —
they don't carry a `v2Fragment` and are lazy (not forced on the main
slice / devShell path).

Effect on `nix path-info --derivation .#devShells…default`:
comp-v2-builder eval calls 13.2M -> 6.5M, total ~62M -> ~49M, CPU
~24s -> ~20s.  The remaining cost is dominated by the module system
(~18M lib/modules.nix calls, unchanged) — the inherent evaluation of
`config.packages.<id>` when each slice references its dep slices — which
is structural and not addressable by build-time composition.

Unit-ids unchanged throughout (Stream Strm-0.4.7.2-a9e4dded,
happy hppy-2.2-70e3af7a).

* v2 build-time composition: remove the now-dead Nix-side closure code

With the slicing repo and cabal.project both composed at build time,
migrate the two remaining Nix-cabal.project consumers and delete the
now-unused closure-assembly machinery (net ~-395 lines in
comp-v2-builder.nix):

  * checkAgainstPlan and docSlice now use the build-time path too
    (v2Fragment + slicePreBuildV2 + localRepo = null), so nothing
    forces the Nix-assembled cabal.project / slicingRepo anymore;
  * sublib reachability seeds (HASKELLNIX_EXTRA_SUBLIB_SEEDS) are
    composed at build time from the per-package fragments' sublib-seeds
    across the all-dep closure plus this slice's own target sublib —
    replacing the last Nix-side allDepClosure walk (extraSublibSeeds);
  * the 'is this a v2 slice' marker moves from passthru.transitiveTarballs
    to the existing passthru.isSlice (comp-v2-builder + shell-for-v2);
  * deleted: cabalProject, slicePreBuild, slicingRepo (+ repoCopyCmds /
    indexEntryCmds / depTarballsDeduped), depTransitives /
    depTransitiveTarballsOf / transitiveTarballs, sliceCanonicalNames +
    the six all*Blocks, depConstraints / libConstraintPins /
    libConstraintClosure / exeUnitsInAllDeps, extraPackages*,
    sourceRepo* / minimalSourceRepo / wrapAsMinimalRepo, allowBootLibBlock,
    and the pre-existing dead libDepClosure / libDepsOf / pkgPlanUnits /
    pkgLibDepClosure.

Kept (still used by the per-package fragment / build-time path): the
per-package block functions (flagBlockFor, ...), bootLibPkgNames,
allDepClosure (build-tool store composition), localCabalProject.

Two build-time fixes for multi-sublib packages: the repo index copy is
no-clobber, and the selfFrag carries its own sublib-seeds (the seeds
loop is robust under set -e/pipefail).

Verified: eval clean, ~15.7s user (v1 parity); Stream / code-page /
network / happy (multi-sublib + build-tool) build with unchanged
unit-ids.  (proto-lens-protobuf-types fails to build, but that's a
pre-existing v2 bug independent of this change — it fails identically
at a77f5524c, before any build-time-composition work.)

* comp-v2-builder: pin Custom-build packages' setup-depends (notably Cabal)

A `build-type: Custom` package's unit-id hashes its custom-setup
configuration, including which `Cabal` the setup is built against.  The
v2 slice's cabal.project pinned library dep versions but left the
per-package custom-setup solve unconstrained, so when a slice's closure
contained a reinstalled `Cabal` (e.g. pulled in by another package's
`proto-lens-setup` custom-setup) cabal preferred it over the GHC-bundled
`Cabal` plan-nix had used — forking the Custom-build unit-id.

Concretely: `ghc-paths` (build-type: Custom, setup-depends Cabal) built
standalone as plan-nix's `ghc-pths-0.1.0.12-142b137f` (setup against
bundled Cabal-3.10.3.0), but inside `proto-lens-protobuf-types`' slice
its setup resolved against the reinstalled Cabal-3.16.1.0 →
`…-e732a98d`.  The pre-built `proto-lens-protoc` exe (linked against
142b137f) was then rejected, cabal rebuilt it + ghc-paths, and the
"exactly one expected unit" check failed.

Fix: emit per-package `constraints: <pkg>:setup.<dep> ==<ver>` for each
package's `components.setup.depends` (which plan-nix records), composed
at build time across the all-dep closure alongside the library
constraints.  cabal accepts the per-package `pkg:setup.dep` qualifier
(verified), so each Custom-build package's setup now resolves to the
same deps plan-nix used, reproducing its unit-id.

Verified: proto-lens-protobuf-types now builds (with the project's
`build-tools = [protobuf]` providing `protoc` for codegen); Stream
(Strm-0.4.7.2-a9e4dded) and happy (hppy-2.2-70e3af7a) unchanged.

* default builderVersion to 2 (cabal v2-build slicing builder)

Make the cabal v2-build slicing builder (comp-v2-builder) the default
for all projects, so CI exercises it.  v2 eval is now at parity with v1
(per-slice work composed at build time, Nix touches only direct deps),
and the v2 builder handles the cases the test suite and cardano-node
exercise (multi-version deps, multi-sublib packages, build-tool
closures, Custom-build setup unit-ids).  Projects can still opt back to
the Setup.hs builder with `builderVersion = 1`.

* comp-v2-builder: compose source-repository-package deps at build time; always allow boot-lib installs

Two fixes for source-repo / reinstallable-lib:ghc projects under v2,
surfaced by the ghc-lib-reinstallable-cabal test:

1. Build-time source-repository-package composition.  The cleanup
   commit dropped source-repo handling (cardano-node has none), which
   broke any project with source-repo deps — including lib:ghc via
   useLocalGhcLib.  Each source-repo package now emits its own
   `source-repository-package` block (wrapping its source in a minimal
   git repo) into its fragment, and the build-time composer collects
   them across the closure and omits those packages from
   `extra-packages:`.  The block references a derivation, so it's
   written with `writeText` (not `toFile`).

2. `allow-boot-library-installs: True` is now emitted unconditionally,
   combined with `allow-newer`/`allow-older` in the project skeleton
   (`solverRelaxations`).  It only *permits* reinstalling boot libs;
   cabal can act on it solely when a boot lib's source is in this
   slice's repo (one plan-nix reinstalled), so it's a no-op otherwise.
   This drops the `bootLibPkgNames` gating machinery and lets cabal
   pick the source-repo `ghc` for useLocalGhcLib slices.  Also added
   `allow-older: *:*` alongside `allow-newer: *:*` (harmless given the
   per-slice repo has one candidate version per package).

Verified: tests.ghc9141.native.tests.ghc-lib-reinstallable-cabal builds
(reinstalls lib:ghc end to end); Stream / happy / proto-lens-protobuf-types
unaffected.

* Bump head.hackage

* v2: version-pin the full installed closure, not just the lib-dep closure

A test/bench slice (or any component that reaches a dep only through
its own package's library or a sibling component, rather than through
component.depends) seeds the blocks closure (blkFrags / transitive-deps)
but not the lib-dep closure (libFrags / lib-dep-slices).  The composed
cabal.project only emitted version pins from libFrags, so those deps
were installed-but-unpinned.  With allow-older + allow-boot-library-
installs the solver was then free to re-solve a reinstalled boot library
(e.g. text 2.1.4 -> GHC-bundled text 2.1.3), drifting every downstream
UnitId and failing checkAgainstPlan.

Emit a bare '<pkg> ==<ver>' pin for every unit in the installed
(blkFrags) closure; keep extra-packages and the 'source' qualifier
scoped to libFrags so build-tool exes aren't forced to rebuild from
source.  Surfaced by ghcjs tests.js-template-haskell.check (reinstalls
text).

* v2: don't leak pkgsBuildBuild build-tool sources into cross slices

On cross, `buildToolSlices` are the build-build (pkgsBuildBuild) tool
slices.  They were added to `depSlices` -> `propagatedBuildInputs`, so
the build-time slicing-repo composer walked their `repo-frag` (source
tarball + index .cabal) into the cross slice's hackage repo, alongside
the cross tool's repo-frag.  cabal then saw two candidates for the tool
(e.g. a build-build `hsc2hs-0.68.10` and the cross one); a dependent's
build-tool exe-dep could resolve to the build-build unit, whose UnitId
feeds the consumer's cross UnitId but never appears in its Setup
configure args.  That forked dependent unit-ids (e.g. network) away from
what cabal computes elsewhere, so `checkAgainstPlan` failed in any slice
depending on a build-tool-using package -- breaking the whole
aarch64/armv7a-android-prebuilt suite (via iserv-proxy) and other cross
targets.

On cross the tool's cross store unit already comes from
`transitiveBuildToolSlices` (the cross `targetSlice`) and its runnable
binary from `buildToolBinOverlays` / `withProgFlags` (both reference
`buildSlice` directly), so the build-build slices must stay out of
`propagatedBuildInputs` here.

* v2: scope constraints to the depends closure, not the whole store

The build-time cabal.project pulled version pins from too many places:
a slice's whole `transitive-deps` closure (an over-broad bare `==<ver>`
pin added to cover test/bench components) plus build-tool `exe-depends`
lib closures.  When a boot library is reinstalled at the *same* version
(e.g. `text-2.1.4` boot `-inplace` vs a reinstalled `txt-2.1.4-<hash>`),
a bare `text ==2.1.4` pin can't disambiguate the two units, so cabal
splits lib deps between them and every dependent UnitId drifts
(`checkAgainstPlan` fails for js-template-haskell on ghcjs / wasm / the
profiled native variants).

Treat each dependency kind distinctly at build time:

  * depends      -> source (repo) + `<dep> ==<ver>, <dep> source`
  * setup-depends-> source (repo) + `<pkg>:setup.<dep> ==<ver>`
  * exe-depends  -> source only (no constraints)

Constraints now come only from the *complete* library-`depends` closure
(`dependsSlices` = directDepSlices ++ home library deps ++ the package's
own library for non-lib components) and per-package setup-constraints.
The `source` qualifier on every reinstalled lib forces cabal off the
GHC-bundled `-inplace` unit even when versions match.  Source + the six
per-package block groups keep riding the existing `transitive-deps`
closure (already free of pkgsBuildBuild tool slices after the previous
commit).

Removed: the blkFrags whole-closure version pin and the exe-depends
contribution to the constraints scope.  Skip the self pkg-name when
emitting extra-packages/constraints (its own library can now enter the
depends walk via `ownLibSlice`; also listing the local target as a repo
package triggers cabal's target-matcher ambiguity, Cabal-7130).

Verified green: native + ghcjs js-template-haskell.check and the
aarch64-android iserv-proxy-interpreter.

* cabal-sublib-shell test: source the real v2 shellHook

The test replayed individual shellHook steps (cabal-store-sync,
project-local-sync) but skipped the rest of the hook — notably the
EM_CACHE setup the ghcjs shell provides.  So `cabal build` of the
ghcjs consumer failed in the test with
`emcc: cache directory "/tmp/_cache" is not writable` even though a
real shell works.

Source `env.shellHook` from the project dir instead.  The hook is
written to be sourced (it uses `return`); sourcing it seeds the
cabal-store, writes `cabal.project.<prefix>local`, and sets a writable
EM_CACHE — exactly what a user gets on shell entry — so the test now
faithfully exercises the v2 shell.  No shell-for-v2 change needed (the
shell already sets EM_CACHE).

Verified: ghcjs ghc984 cabal-sublib-shell.run builds the consumer
.jsexe and reuses the shell's provider lib + slib sublib (no rebuild).

* call-cabal-project-to-nix test: link android exe statically

This test builds cabal-simple via the low-level callCabalProjectToNix /
mkCabalProjectPkgSet path with a minimal modules list, so it doesn't
pull in modules/cabal-project.nix's config — in particular the android
default that adds `package * ghc-options: -optl-static -optl-ldl` so the
exe links statically.  A dynamically-linked Android binary references
/system/bin/linker64 at runtime, which qemu-user can't open on the build
host, so the v2 run-check (which executes the built exe) failed with
`qemu-aarch64: Could not open '/system/bin/linker64'`.  (v1 only passed
because lib/check.nix re-overrides the *check* exe with
setupBuildFlags = -optl-static; v2 runs the pre-built slice exe, so it
needs the flag at the project level.)

Replicate modules/cabal-project.nix's android default in the test, for
both the plan (callCabalProjectToNix) and the build (modules), so the
exe is statically linked and runnable under qemu-user.

Verified: aarch64-android-prebuilt callCabalProjectToNix.run now runs
the exe ('Hello, Haskell!') under v2.

* Revert "default builderVersion to 2 (cabal v2-build slicing builder)"

This reverts commit dd18126a54671b4afde83f161519adfe6cb83457.
08b27295 · master · 6/506 ++ 471 --
Showing 13 of 13 recent commits. Check out Input Output on GitHub to see more.