Cardano Updates

also adapt networking tests to stricter default reconnect policy

Signed-off-by: Roland Kuhn <[email protected]>

Surface the static-linking / library-for-ghci / shared-libs flags
that comp-builder and `lib/check.nix` already apply at the
artefact level into the plan-to-nix step, so plan-nix's recorded
configure-args and UnitIds match what cabal v2-build would compute
against the real compiler.

Four mkIfs added to `modules/cabal-project.nix`:

* **musl host** — `package * \n executable-static: True`.
  comp-builder adds `--ghc-option=-optl=-static` at build time;
  surfacing the toggle here makes plan-to-nix record
  `--enable-executable-static`.  Build artefacts are unchanged.
* **x86_64-darwin host** — `package * \n library-for-ghci: True`.
  Mirrors what comp-builder passes for `!ghcjs && !wasm && !android`
  (always true on darwin).
* **android host** —
  `package * \n ghc-options: -optl-static -optl-ldl` (plus
  `-optl-no-pie` on aarch32).  Mirrors `lib/check.nix`'s test-exe
  setupBuildFlags re-wrap.
* **wasm GHC ≥ 9.12** — `package * \n shared: True`.  Wasm's RTS
  linker only loads `.so` files; `--disable-shared` (cabal's
  default given the real compiler's reported capabilities) would
  force a `.a`-only install that TH-eval can't load.

All four directives sit at `mkBefore` priority so a project's own
`cabalProjectLocal` overrides them.

Cache impact: plan-nix hashes change for affected platforms.

Changelog entry added in `changelog.md` describing the cache-bust
and the opt-out path.

Pulled out of #2504 (`hkm/builder-v2`).

* Add `useLocalGhcLib` project option

Surface what `modules/configuration-nix.nix` used to do
unconditionally as an opt-in `useLocalGhcLib` flag, so the
`packages.ghc.src` override only fires when a project actually
constrains the `ghc` package (e.g. `ghc-lib-reinstallable`).

Four pieces:

* `modules/project-common.nix`: add the `useLocalGhcLib` option
  (default `false`).
* `modules/configuration-nix.nix`: drop the unconditional
  `packages.ghc.src` / `packages.ghc.package-description-override`
  overrides — they're moved into the per-project wiring below.
* `modules/stack-project.nix`: under `useLocalGhcLib`, re-apply
  the `packages.ghc.src` post-plan override.  Stack-to-nix can't
  use the cabal-project route, so this keeps the existing
  behaviour for stack users who flip the flag.
* `modules/cabal-project.nix`: under `useLocalGhcLib`, inject a
  `source-repository-package` block into `cabalProjectLocal`
  pointing at the configured-src + generated GHC tree, and add an
  `inputMap` entry so haskell.nix doesn't try to fetch the URL.
  Cabal then hashes the wrapped repo's content into
  `pkg-src-sha256` and installs `lib:ghc` like any other
  reinstallable dep.

Projects that need the previous always-on behaviour now set
`useLocalGhcLib = true` on the project module; everyone else gets
a smaller plan-nix and avoids the unconditional `configured-src`
materialisation.

Pulled out of #2504 (`hkm/builder-v2`).

* ghc-lib-reinstallable test + changelog: set useLocalGhcLib = true

Flips the new opt-in flag in both test variants and adds a
changelog entry telling users to do the same when they constrain
`lib:ghc`.

Updates cabal.project to point at the three rebased Leios forks that
ship with this cardano-node 11.0.1 build:

  - ouroboros-consensus  @  IntersectMBO/ouroboros-consensus
    branch leios-prototype-remake-3.0.1.0
    Single squashed Leios commit on top of release-3.0.1.0
    (mempool + diffusion + chain inclusion + voting + demo).
  - ouroboros-network    @  IntersectMBO/ouroboros-network
    branch leios-prototype-remake-1.1.0.0
    Three Leios commits on top of ouroboros-network-1.1.0.0
    (BearerBytes, Reception arrival-time, drop unused imports)
    + a trace-dispatcher version bump for compat with cardano-node 11.0.1.
  - cardano-ledger       @  IntersectMBO/cardano-ledger
    branch leios-prototype-remake
    Adds 'Maybe LeiosCert' on the Dijkstra block body
    (Dijkstra-only, mirrors 'Maybe PerasCert').

Also drops the 'dmq-node' extra-package (its latest CHaP release
requires ouroboros-network:framework-tracing, a sublib name that
the remade network fork doesn't carry).

Vendors a one-line 'ekg-forward' source patch (left untouched here)
to match the network's pre-bump 'ConnectToArgs' shape.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Both had been on the leios-prototype and and need to be re-exposed after
resetting to a recent cardano-node version.

Only the rest-in-mempool execution units metric was not easily
applicable. That part of the code will change anyways though.

The consensus Leios chunks added four fields to the consensus tracer
records that cardano-node assembles in 'Cardano.Tracing.Tracers' (old
style) and 'Cardano.Node.Tracing.Tracers' (new style):

  - Consensus.Tracers'      gained 'leiosKernelTracer' and 'leiosPeerTracer'
  - NodeToNode.Tracers'     gained 'tLeiosNotifyTracer' and 'tLeiosFetchTracer'

This commit populates those fields in both 'mkTracers' code paths so
the record-construction sites are total. All four are wired to
'nullTracer' for now; the corresponding 'Transformable' / 'MetaTrace'
instances and EKG metrics are left as TODO. The new-style tracing
system will accordingly emit a 'TracerConsistencyWarnings' for the
four Leios namespaces declared in the config.yaml — that surfaces the
gap explicitly rather than silently dropping the configuration.

Adds the corresponding selectors to 'TraceSelection' / 'PartialTraceSelection'
and their JSON parsing:

  - traceLeiosKernel          / Consensus.LeiosKernel
  - traceLeiosPeer            / Consensus.LeiosPeer
  - traceLeiosNotifyProtocol  / LeiosNotify.Remote
  - traceLeiosFetchProtocol   / LeiosFetch.Remote

Also drops the duplicate 'LogFormatting (Simple/Stateful.TraceSendRecv)'
and 'MetaTrace (Simple/Stateful.TraceSendRecv)' instances that the
upstream leios-prototype branch carried in
'Cardano/Node/Tracing/Tracers/NodeToClient.hs' — they are now provided
by 'ouroboros-network:framework-tracing' and would collide if redefined
here.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Introduces 'Cardano.Node.Configuration.Leios.LeiosDbConfig' with two
constructors:

  - LeiosDbInMemory          (matches 'LeiosDbConnection's in-memory backend)
  - LeiosDbSQLite !FilePath  (SQLite backend, file relative to node CWD)

Wired through 'PartialNodeConfiguration' / 'NodeConfiguration' as
'ncLeiosDbConfig' with JSON parsing ('"LeiosDbConfig":{"Backend":...,
"Filepath":...}') and a default of 'LeiosDbSQLite "leios.db"'.

At node startup ('Cardano.Node.Run.handleSimpleNode'), the configured
backend is materialised into a 'LeiosDbHandle' (via 'newLeiosDBInMemory'
/ 'newLeiosDBSQLite') and passed into 'RunNodeArgs' through a new
'rnLeiosDb :: Maybe (LeiosDbHandle m)' field, which consensus's
'runWith' threads into the Leios kernel's shared connection.
'Nothing' falls back to the in-memory default.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

The consensus Leios chunks extended 'NTN.Codecs' with two extra type
parameters ('bLN' / 'bLF', for LeiosNotify and LeiosFetch), and the
network-side BearerBytes/Reception work changed 'runPeerWithLimits' to
return 'Maybe (Reception bytes)' (instead of 'Maybe bytes') and made
'byteLimitsKeepAlive' parameter-less.

Update 'tx-generator' to compile against the rebased stack:

  - Pull in 'ouroboros-network:framework-tracing' as a sublib dependency.
  - Extend the local 'myCodecs' type to 9 'ByteString' parameters (was 7).
  - Update 'kaClient' return type to 'IO ((), Maybe (Mux.Reception ByteString))'.
  - Drop the size-function argument from 'byteLimitsKeepAlive'.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Mempend a small OuroborosBundle with leiosNotify/leiosFetch on the Hot
temperature onto the existing five-protocol bundle in initiator and
initiatorAndResponder.  Define the matching ingress-queue limits from the
demo's LeiosFetchStaticEnv.  This matches the upstream leios-prototype
pattern - the change is contained to consensus and leaves the network
NodeToNodeProtocols record untouched.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Bumps [qs](https://github.com/ljharb/qs) from 6.14.0 to 6.15.2.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.14.0...v6.15.2)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.15.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

Upstream cleanup, not Leios-specific: eight unused-import warnings that
upstream's own ghc-options promote to errors via 'nix/ouroboros-network.nix'
('-Werror' on forAllProjectPackages). They've been present at the
'main-peras-5202-merged' SHA that cardano-node and ouroboros-consensus
master both consume; surface when downstream haskell.nix builds also
inherit `-Werror`.

Affected files (one unused qualified import each):
- framework/lib/Ouroboros/Network/Protocol/Handshake/Codec.hs (BL)
- framework/tests-lib/.../ConnectionManager/Experiments.hs (LBS)
- framework/io-tests/.../Driver.hs (List)
- framework/io-tests/.../Socket.hs (IntMap)
- tests/lib/.../TxSubmission/AppV1.hs (BSL)
- tests/lib/.../TxSubmission/AppV2.hs (BSL)
- protocols/tests-lib/.../KeepAlive/Test.hs (BL)
- protocols/tests-lib/.../PeerSharing/Test.hs (BL)

Verified with 'cabal clean && cabal build all --ghc-options=-Werror' →
exit 0.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Wrap `Channel m a` recv values in a new `Reception a = MkReception
!(IntMap Time) !a`, recording per-chunk arrival times.  In the mux
demuxer the chunk's arrival time is paired with its starting byte
offset; codec/driver layers either strip the time map (the codec sees
only bytes) or propagate it through `wrapMiniProtocolTrailing` at the
mini-protocol callback boundary.  Tests/demos/benches use `IntMap.empty`
when synthesising trailing.

This is the network half of the November Leios demo's arrival-time
plumbing; consumers (e.g. Praos block-arrival tracing on the consensus
side) will be wired up separately.

Co-Authored-By: Claude Opus 4.7 <[email protected]>

The validateField method in PresentationVerify passes the filter.pattern
from the presentation definition directly to new RegExp() without any
safety checks. An attacker can craft a presentation request with a regex
pattern that causes catastrophic backtracking (e.g. (a|aa)+b), blocking
the JavaScript event loop indefinitely.

This is an unauthenticated attack vector since the presentation definition
originates from an external verifier.

Add a safeRegex utility that:
- Validates the pattern is a non-empty string
- Enforces a maximum pattern length of 256 characters
- Verifies the pattern compiles as a valid regex
- Detects and rejects patterns with nested quantifiers (ReDoS vectors)
- Detects and rejects patterns with alternation inside quantified groups

Fixes #646

Signed-off-by: A-Chronicle <[email protected]>

The validateField method in PresentationVerify passes the filter.pattern
from the presentation definition directly to new RegExp() without any
safety checks. An attacker can craft a presentation request with a regex
pattern that causes catastrophic backtracking (e.g. (a|aa)+b), blocking
the JavaScript event loop indefinitely.

This is an unauthenticated attack vector since the presentation definition
originates from an external verifier.

Add a safeRegex utility that:
- Validates the pattern is a non-empty string
- Enforces a maximum pattern length of 256 characters
- Verifies the pattern compiles as a valid regex
- Detects and rejects patterns with nested quantifiers (ReDoS vectors)
- Detects and rejects patterns with alternation inside quantified groups

Fixes #646

Signed-off-by: A-Chronicle <[email protected]>

Flips the new opt-in flag in both test variants and adds a
changelog entry telling users to do the same when they constrain
`lib:ghc`.

Surface what `modules/configuration-nix.nix` used to do
unconditionally as an opt-in `useLocalGhcLib` flag, so the
`packages.ghc.src` override only fires when a project actually
constrains the `ghc` package (e.g. `ghc-lib-reinstallable`).

Four pieces:

* `modules/project-common.nix`: add the `useLocalGhcLib` option
  (default `false`).
* `modules/configuration-nix.nix`: drop the unconditional
  `packages.ghc.src` / `packages.ghc.package-description-override`
  overrides — they're moved into the per-project wiring below.
* `modules/stack-project.nix`: under `useLocalGhcLib`, re-apply
  the `packages.ghc.src` post-plan override.  Stack-to-nix can't
  use the cabal-project route, so this keeps the existing
  behaviour for stack users who flip the flag.
* `modules/cabal-project.nix`: under `useLocalGhcLib`, inject a
  `source-repository-package` block into `cabalProjectLocal`
  pointing at the configured-src + generated GHC tree, and add an
  `inputMap` entry so haskell.nix doesn't try to fetch the URL.
  Cabal then hashes the wrapped repo's content into
  `pkg-src-sha256` and installs `lib:ghc` like any other
  reinstallable dep.

Projects that need the previous always-on behaviour now set
`useLocalGhcLib = true` on the project module; everyone else gets
a smaller plan-nix and avoids the unconditional `configured-src`
materialisation.

Pulled out of #2504 (`hkm/builder-v2`).

Cabal-install reads `ghc --info` during plan elaboration to decide
per-unit settings (`--enable-shared` vs `--disable-shared`, RTS-way
inclusion, etc.) that feed into the recorded UnitId hash.  The
inline `dummy-ghc` script in `lib/call-cabal-project-to-nix.nix`
was a stripped-down stub whose `--info` output didn't match the
real cross GHCs', so plan-nix recorded UnitIds that diverged from
what a downstream cabal v2-build (or any consumer running cabal
against the real compiler) would compute.

Three changes:

1. **`lib/dummy-ghc.nix`** (new): extract the dummy-ghc script and
   emit cross-aware `--info` capabilities matching real cross GHCs
   for windows, ghcjs, wasm, android, static, native-musl, and
   native — `Support dynamic-too`, `Support shared libraries`,
   `RTS ways`, `Stage`, `GHC Dynamic`, the iserv-related fields,
   etc.

2. **`lib/call-cabal-project-to-nix.nix`**:
   * gate the dummy-ghc-pkg `-inplace` suffix on GHC ≥ 9.8 — older
     GHCs register pre-existing packages without it, so the dummy
     was synthesising the wrong package ids for plan-to-nix
     against ghc 8.10 / 9.0 / 9.2 / 9.4 / 9.6.
   * pin `CABAL_INSTALLED_PACKAGE_ID_OS` to the build platform's
     OS when running plan-to-nix.  Without this the patched
     cabal-install's installed-package-id format tracks the eval
     system; a darwin host evaluating a linux derivation gets the
     `VeryShort` form and forks UnitIds from what cabal v2-build
     on a linux builder would compute.

3. **`lib/default.nix`**: tighten `uniqueWithNameKey` so
   derivations (`.name = "<pkg>-<ver>"`) and module values
   (`.identifier.{name,version}`) can't collide on shared name
   fragments — partition into `unit-id:` / `id:` / `name:` buckets.
   Correctness is unchanged when buckets are correct; this just
   avoids the slow `lib.unique` fallback in more cases.

Pulled out of #2504 (`hkm/builder-v2`).