fix(database): harden genesis bootstrap consistency across metdata plugins
Signed-off-by: cryptodj413 <[email protected]>
Signed-off-by: cryptodj413 <[email protected]>
Bumps [github.com/blinklabs-io/gouroboros](https://github.com/blinklabs-io/gouroboros) from 0.166.1 to 0.168.0. - [Release notes](https://github.com/blinklabs-io/gouroboros/releases) - [Changelog](https://github.com/blinklabs-io/gouroboros/blob/main/RELEASE_NOTES.md) - [Commits](https://github.com/blinklabs-io/gouroboros/compare/v0.166.1...v0.168.0) --- updated-dependencies: - dependency-name: github.com/blinklabs-io/gouroboros dependency-version: 0.168.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/utxorpc/go-sdk](https://github.com/utxorpc/go-sdk) from 0.0.2 to 0.0.4. - [Release notes](https://github.com/utxorpc/go-sdk/releases) - [Commits](https://github.com/utxorpc/go-sdk/compare/v0.0.2...v0.0.4) --- updated-dependencies: - dependency-name: github.com/utxorpc/go-sdk dependency-version: 0.0.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/blinklabs-io/gouroboros](https://github.com/blinklabs-io/gouroboros) from 0.153.1 to 0.170.1. - [Release notes](https://github.com/blinklabs-io/gouroboros/releases) - [Changelog](https://github.com/blinklabs-io/gouroboros/blob/main/RELEASE_NOTES.md) - [Commits](https://github.com/blinklabs-io/gouroboros/compare/v0.153.1...v0.170.1) --- updated-dependencies: - dependency-name: github.com/blinklabs-io/gouroboros dependency-version: 0.168.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [github.com/fxamacker/cbor/v2](https://github.com/fxamacker/cbor) from 2.9.1 to 2.9.2. - [Release notes](https://github.com/fxamacker/cbor/releases) - [Commits](https://github.com/fxamacker/cbor/compare/v2.9.1...v2.9.2) --- updated-dependencies: - dependency-name: github.com/fxamacker/cbor/v2 dependency-version: 2.9.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.50.0 to 0.54.0. - [Commits](https://github.com/golang/net/compare/v0.50.0...v0.54.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.54.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>
Bumps [github.com/utxorpc/go-codegen](https://github.com/utxorpc/go-codegen) from 0.18.1 to 0.19.0. - [Release notes](https://github.com/utxorpc/go-codegen/releases) - [Commits](https://github.com/utxorpc/go-codegen/compare/v0.18.1...v0.19.0) --- updated-dependencies: - dependency-name: github.com/utxorpc/go-codegen dependency-version: 0.19.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: cryptodj413 <[email protected]>
Introduce 'Maybe LeiosCert' (mirrored after the existing 'Maybe PerasCert')
as an optional field on the Dijkstra-era 'BlockBody'. Other eras are
unaffected — this is the entire Leios-related ledger change per the
rebase plan's decision #3 (no 'Body'/'BodyInline'/'BodyCertificate' sum,
no early-era touch).
The change spans four pieces, all kept together so the type, codec,
hashing, and lens story land coherent:
- 'LeiosCert' / 'PerasCert' placeholders encode as @encodeListLen 0@
rather than @encCBOR ()@ — the latter is 'encodeNull' (0xf6), which
collides with the null-tag of 'encodeNullStrictMaybe' and made
'SJust LeiosCert' indistinguishable from 'SNothing' on the wire.
Adds a regression test in 'BaseTypesSpec' covering both certs.
- 'DijkstraBlockBody' carries pre-encoded bytes for body / wits /
aux-data / is-valid / Leios cert / Peras cert, six slots total
(numSegComponents = 6). The Leios and Peras cert slots always emit
a CBOR token (null when absent) so the on-wire layout is fixed.
- Lens setters ('txSeqBlockBodyL', 'leiosCertBlockBodyL',
'perasCertBlockBodyL') rebuild via the bidirectional
'DijkstraBlockBody' pattern. The previous setters only updated the
structural field and silently kept the stale pre-encoded bytes, so
e.g. setting txs left 'dbbTxsBodyBytes' as the empty list and the
block on the wire had no transactions even though the in-memory
rep was correct. Mirrors the Alonzo idiom
@lens abbTxs (\_ s -> AlonzoBlockBody s)@.
- 'Arbitrary' and 'TreeDiff' instances for the body extended to
generate / show the new cert slot.
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Chris Gianelloni <[email protected]>
Signed-off-by: Chris Gianelloni <[email protected]>
Flip the ouroboros-network SRP pin from the upstream
'main-peras-5202-merged' tag to the local 'leios-prototype-remake' branch
(file://path so the project-specific repo can later pin the same SHA).
Adapt the consensus side to two API changes from chunk N:
1. BearerBytes class replaces dataSize on ProtocolSizeLimits.
- byteLimitsLeiosFetch/byteLimitsLeiosNotify drop their (bytes -> Word)
parameter.
- Every byteLimits* call site loses its (const 0) / size argument
(consensus diffusion's noByteLimits/byteLimits records and the
PeerSimulator chainSyncNoSizeLimits/blockFetchNoSizeLimits).
- mkApps on N2N and N2C grows BearerBytes constraints for each bytes
parameter; Ouroboros.Network.Protocol.Limits is imported for it.
2. Reception arrival-time wrapping on channel recv.
- Every (a, Maybe bytes) trailing in N2N/N2C aChainSync*/aBlockFetch*/
etc. becomes (a, Maybe (Reception bytes)).
- createConnectedChannelsWithDelay in ThreadNet/Network wraps recv'd
values with `MkReception IntMap.empty` (mirroring how the network
repo's tests/demos synthesise trailings — no real arrival times when
the channel is an in-process TMVar).
- PeerSimulator ChainSync/BlockFetch runners strip the Reception when
packaging results into PeerSimulatorResult (`fmap received trailing`).
- immdb-server's MiniProtocols re-add the arrival timestamps in the
JSON tracers — restoring the `mux_at` field the C8 commit had to
placeholder while the old SRP didn't carry them.
Unrelated upstream-drift fixes folded in:
- TraceSendMsg/TraceRecvMsg now carry a Time / Maybe Time argument; the
PeerSimulator.Trace pattern is updated.
- FetchModeGenesis constructor renamed to GenesisFetchMode in
Ouroboros.Network.BlockFetch.ConsensusInterface; updated in the
BlockFetch test + PeerSimulator.BlockFetch.
- Node.hs: Cardano.LedgerPeersConsensusInterface's `getBlockHash` field
renamed to `getImmutableBlockPoint` returning Either
GetImmutableBlockPointError, with separate error cases for
Genesis/NotYetImmutable.
- ChurnMode is now newtype { getFetchMode :: FetchMode }; replace
`ChurnModeNormal` with `ChurnMode (PraosFetchMode FetchModeDeadline)`.
- Qualify NodeKernel.getFetchMode to disambiguate from
Cardano.Network.PeerSelection.Churn's identically-named field.
End-to-end gate after migration:
- cabal build all (including --enable-tests): clean.
- cardano-test -p "Leios" --quickcheck-tests=200: PASS, 1/1, 78.6s.
- consensus-test -p "Peras": PASS, 7/7, 2.3s.
- leios-schedule-gen smoke + immdb-server --help: green.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
Per the rebase plan's C5. Adds an era-abstract running total of transaction bytes applied so far on the Shelley-based ledger, and lands the real 'ResolveLeiosBlock' splice for Dijkstra-era CardanoBlocks (C3 only added the structural class; the actual EB-closure read sits in Shelley/Ledger/Ledger.hs and lands here together with the cumulative tx-bytes change since they share the same file). - 'Ouroboros.Consensus.Ledger.SupportsMempool' gains a per-block 'blockTxBytes' projection that the cumulative counter updates from. - 'shelleyCumulativeTxBytes :: Word64' on 'ShelleyLedgerState' and 'tickedShelleyCumulativeTxBytes' on the ticked variant; both CBOR-versioned through (en|de)codeShelleyLedgerState. - 'HardFork.Combinator.Mempool' threads cumulative tx bytes through per-era projections. - 'ResolveLeiosBlock (ShelleyBlock (Praos c) DijkstraEra)': reads the parent's 'praosStateLeiosAnnouncement' + 'praosStateLastSlot', queries 'leiosDbQueryCompletedEbByPoint', and splices the EB's tx closure into the body via 'Core.txSeqBlockBodyL'. Errors with a clear message if the announced EB isn't available locally (matches the prototype's FIXME — race that the threadnet harness avoids by snapshotting LeiosDb per node, see C7). Refs: #1988 Co-Authored-By: Claude <[email protected]>
Cleanup that haskell.nix's -Werror surfaces but local cabal -Wall does not. The touched files were last modified by C7 (threadnet harness), the N migration (BearerBytes + Reception) and C8 (immdb-server + schedule generator), so this is left as a follow-up commit rather than squashed into C7 (the autosquash conflicts with later commits that touch the same files). - Test/ThreadNet/Network.hs: drop unused LeiosDemo* / Control.Exception / Ouroboros.Network.Mux imports; drop dead 'cfg'' binding and unused NodeLeiosDb data type + the record field that referenced it. - TxGen/Cardano.hs: drop unused LedgerState / mkShelleyTx / Praos imports + dead ejectDijkstraTickedLedgerState. - ouroboros-consensus.cabal: drop unused io-sim + unstable-byron-testlib + unstable-cardano-testlib + unstable-mock-block from unstable-cardano-tools deps; drop unused cardano-slotting from leios-schedule-gen deps. - Cardano/Tools/DBAnalyser/Block/Cardano.hs: drop unused ShelleyLeaderCredentials import. - Cardano/Tools/ThreadNet/Run.hs: fill missing txLogicVersion field (upstream's TestConfig grew this in 0dac43e80 Aug 2024); use maxBound to match other call sites. - Test/ThreadNet/Cardano.hs: fill missing ctgeExtraTxGen field (upstream's CardanoTxGenExtra grew this). - Ouroboros.Consensus.Node: drop unused nullTracer import. Co-Authored-By: Claude Opus 4.7 <[email protected]>
Brings back the immdb-server's Leios bits — the schedule-driven LeiosNotify and LeiosFetch servers — plus the prototype's schedule generator as a standalone binary. Source modules (under unstable-cardano-tools): - Cardano/Tools/ImmDBServer/Json.hs (+ Json/Say.hs + Json/SendRecv.hs) Atomic JSON-line tracer (`mkUltimateTracer`) used by the schedule-driven immdb-server. - Cardano/Tools/ImmDBServer/MiniProtocols.hs Grafted onto main's structure: keep the modern `Cardano.Network.NodeToNode` imports + `PerasUnsupported` in `stdVersionDataNTN`, add the schedule-driven `leiosNotifyProt` / `leiosFetchProt`, plus the per-message send/recv JSON tracers (`maybeShowSendRecvCS/BF/LF`, `mapBFEvent`). `chainSyncServer` now consults `getSlotDelay` so blocks are released at their wallclock slot. Leios mini-protocol limits are inlined locally (5 MiB ingress queue) since `Consensus.N2N` no longer exposes leios-specific limit helpers. - Cardano/Tools/ImmDBServer/Diffusion.hs Exports `LeiosSchedule` + extended `run` taking `getSlotDelay`, `leiosDbFile`, `LeiosSchedule`. The scheduler thread feeds `MP.leiosMailbox` from the time-ordered schedule entries. Server plumbing uses main's `Server.with`-shaped API. Executables: - app/immdb-server.hs gains `--leios-db`, `--leios-schedule`, `--initial-slot`, `--initial-time` (+ `--address`). Slot-to-wallclock uses a 1-second slot assumption. - app/leios-schedule-gen.hs is the prototype's `leiosdemo202510.hs` carrying only the `generate` subcommand: read a JSON manifest, write a SQLite Leios DB + a matching schedule JSON. Per-message helper subcommands (MsgLeiosBlock*, ebId-to-point, fetch-logic-iteration, hash-txs, cache-copy) are intentionally dropped; those were used by the November demo shell scripts, which are out of scope. Drift note: the JSON tracers now set `mux_at = Nothing` instead of an arrival timestamp. The prototype's "arrival-time on TraceRecvMsg" patch lives on the leios-prototype `ouroboros-network` fork; until that pin is flipped, `TraceSendMsg` / `TraceRecvMsg` here only carry the message. Library deps added to `unstable-cardano-tools`: `io-classes:si-timers`, `time`, `typed-protocols`. `immdb-server` exe adds aeson, cardano-slotting, time. New `leios-schedule-gen` exe pulls direct-sqlite, base16-bytestring, cardano-binary, cardano-crypto-class, cborg, random, vector, etc. Smoke-tested end-to-end: a minimal manifest produces a valid SQLite DB and a schedule JSON with the expected MsgLeiosBlockOffer/MsgLeiosBlockTxsOffer entries. `cabal build all` clean. Refs: #1718, #1724, #1883 Co-Authored-By: Claude Opus 4.7 <[email protected]>
Per the rebase plan's C7. Brings up the Leios threadnet QuickCheck test
on the new structural foundation (chunks C1-C5). The November-demo
scripts ('scripts/leios-demo/', 'nix/leios-mvd/') are intentionally
dropped; the 'run-threadnet' executable is also deferred (its prototype
has upstream API drift — see follow-up).
- 'test/cardano-test/Test/ThreadNet/Leios.hs': the property test
itself — verifies cert blocks happen, no vacuous runs, gap >
minCertificationGap, all nodes converge, and an independent replay
of each node's chain through 'resolveLeiosBlock' produces the same
final ledger as the live node.
- 'unstable-diffusion-testlib/Test/ThreadNet/Network.hs' is the
central rebase: 'NodeInfo' parametrises over a Leios state functor
so the live version holds a 'StrictTVar m' and the snapshot reader
produces 'Identity'; 'mkArgs' threads the TVar to
'fromMinimalChainDbArgs' via the new 'mcdbLeiosDbVar' field on
'MinimalChainDbArgs'; 'mkTestOutput' returns the snapshotted LeiosDb
so the post-hoc 'replayNodeChain' / 'sumChainTxBytes' can resolve
EBs (without this, every test would crash at the first cert block).
Per-node 'leiosVotingKey' wired so voting fires.
- All other test-side call sites updated for new ChainDB / forge /
mempool signatures (ChainDB.Unit, StateMachine, FollowerPromptness,
LedgerSnapshots, BlockFetch.Client, LocalStateQuery.Server,
PeerSimulator, Genesis tests, HardFork.Combinator tests).
- 'unstable-cardano-testlib': 'Test.Consensus.Cardano.ProtocolInfo'
gains a 'hardForkInto Dijkstra' entry; 'Test.ThreadNet.TxGen.Cardano'
projects DijkstraEra from the cardano TxGen.
- 'unstable-shelley-testlib': generators / examples updated for the
new Praos header field and ShelleyLedgerState cumulative tx bytes.
Refs: #1883, #1906
Co-Authored-By: Claude <[email protected]>
Per the rebase plan's C2, extends the Praos header and chain-dep state to carry the optional Leios EB announcement (the certify flag is gone — the cert lives on 'DijkstraBody' instead, per resolved decision #3). - 'Header': add 'hbLeiosEbAnnouncement :: !(StrictMaybe EbAnnouncement)'; emit / decode it alongside the existing header fields. - 'PraosState': add 'praosStateLeiosAnnouncement :: !(StrictMaybe ...)' and update it from the applied block's signed header view. The announcement is intentionally *non-sticky* — every block overwrites the slot with that block's own announcement (or 'SNothing'), so a certifying block can only be the *direct* successor of the announcer. Simpler than the prototype's 'LeiosState' record; we drop the unused 'leiosStateCumulativeEbAnnouncementSize'. - 'ProtocolHeaderSupportsKES.protocolStateLeiosInfo': new class method returning '(EbAnnouncement, WithOrigin SlotNo)' for the forger's certify decision. TPraos default returns 'Nothing'; Praos reads from 'PraosState'. - 'Block.SupportsProtocol' / Shelley protocol shims threaded accordingly. Test generators updated to populate the new field. Refs: #1978, #1921 Co-Authored-By: Claude <[email protected]>
Per the rebase plan's C1, brings the prototype's Leios scaffolding into
'ouroboros-consensus' / 'ouroboros-consensus-diffusion'. Self-contained
infrastructure that later chunks (C2-C7) build on top of:
- LeiosDemo{Types, Db, Logic, Exception, OnlyTest{Fetch, Notify}} and
'LeiosVoteState' under 'ouroboros-consensus/src/ouroboros-consensus/'.
Module names kept flat-and-prefixed, matching the prototype layout.
- 'LeiosNotify' / 'LeiosFetch' mini-protocols (types, codecs, server
+ client, handlers) and 'leiosFetchLogic' background thread, plumbed
through 'NodeToNode' and 'NodeKernel'. The voting thread and voting
message paths land in this chunk too — splitting them out would
fracture single files for no review benefit; C6 only contributes the
threadnet-side voting key wire.
- 'LeiosDbConnection' threaded into 'ChainDbArgs' (cdbsLeiosDb), the
ChainDB.Impl pipeline, and the LedgerDB args. Real-node start path
('Ouroboros.Consensus.Node') opens the DB.
- 'cabal.project' SRP pins flipped to the rebased ledger fork
('cardano-ledger-leios-prototype-remake-on-chap') and the network
fork that carries the demo-tuning add-ons.
- 'cardano-blueprint' submodule added at the new prototype branch tip,
so consensus / network can pull the Leios CDDL from one place.
- New modules registered in 'ouroboros-consensus.cabal'.
Refs: #1832, #1837, #1979
Co-Authored-By: Claude <[email protected]>
Per the rebase plan's C3, threads a 'ResolveLeiosBlock' typeclass through
the block-application pipeline so that Dijkstra-era blocks carrying a
'SJust LeiosCert' get their (empty) tx list filled in from the EB
closure in 'LeiosDb' before validation. Plumbing only — no chain-
selection rule change (per resolved decision #1, decision #3).
- 'Storage.LedgerDB.Forker' grows a 'ResolveLeiosBlock' class and the
'applyBlock' hook calls 'resolveLeiosBlock leiosDb hdrSt b' before
'tickThen{Apply,Reapply}'. The forker's 'ExtLedgerState' is read to
extract 'headerState' for the splice (used to look up the previous
EB announcement on the chain-dep state).
- 'Storage.LedgerDB.V2.validate' threads 'LeiosDbConnection' alongside
the existing args, so the new applyBlock signature reaches the V2
pipeline.
- Default 'ResolveLeiosBlock' instances ('return blk') for every block
type that doesn't carry Leios: Byron, mock SimpleBlock', DualBlock,
TestBlock, HardForkBlock, ShelleyBlock (TPraos). The Dijkstra-era
'ShelleyBlock (Praos c)' splice lives next to the cumulative tx
bytes change in C5 since both live in
'Ouroboros.Consensus.Shelley.Ledger.Ledger'.
- CardanoBlock dispatch in 'Ouroboros.Consensus.Cardano.Block' routes
per-era to the appropriate instance.
Refs: #1873, #1889
Co-Authored-By: Claude <[email protected]>
Per the rebase plan's C4. The forging pipeline learns to (1) emit EB
announcements in Dijkstra blocks and (2) decide between announcing a
new EB and certifying a previously-announced one, gated by
'minCertificationGap'. Other eras are unaffected.
- 'Block.Forging.ForgeBlockArgs' gains 'fbEbTxs', 'fbLeiosDb',
'fbCurrentTickedLedgerState', 'fbChainDepState' (for the certify
decision), and 'fbLeiosTracer'. The HFC's 'Combinator.Forging'
dispatches the new fields per-era; only Dijkstra reads them.
- 'Shelley.Ledger.Forge.forgeShelleyBlock' grows 'decideLeios':
certify if a previously-announced EB has gap > minCertificationGap
and is available locally; otherwise forge a new EB from 'fbEbTxs'
(honest forger only emits an EB when the mempool has txs) and
embed its announcement on the header. The cert lives in the body
via 'leiosCertBlockBodyL' (no header certify flag).
- 'Cardano.CanHardFork.ebCapacityTxMeasure' ported verbatim from the
prototype so the HFC mempool knows the Dijkstra EB capacity.
- DBSynthesizer / DBAnalyser call sites updated for the new
'ForgeBlockArgs' shape ('fbEbTxs = mempty' for non-Leios tools).
Refs: #1888, #2018
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Chris Gianelloni <[email protected]>
Co-authored-by: sg-doc-holiday[bot] <219201796+sg-doc-holiday[bot]@users.noreply.github.com>
Upstream cleanup, not Leios-specific: eight unused-import warnings that
upstream's own ghc-options promote to errors via 'nix/ouroboros-network.nix'
('-Werror' on forAllProjectPackages). They've been present at the
'main-peras-5202-merged' SHA that cardano-node and ouroboros-consensus
master both consume; surface when downstream haskell.nix builds also
inherit `-Werror`.
Affected files (one unused qualified import each):
- framework/lib/Ouroboros/Network/Protocol/Handshake/Codec.hs (BL)
- framework/tests-lib/.../ConnectionManager/Experiments.hs (LBS)
- framework/io-tests/.../Driver.hs (List)
- framework/io-tests/.../Socket.hs (IntMap)
- tests/lib/.../TxSubmission/AppV1.hs (BSL)
- tests/lib/.../TxSubmission/AppV2.hs (BSL)
- protocols/tests-lib/.../KeepAlive/Test.hs (BL)
- protocols/tests-lib/.../PeerSharing/Test.hs (BL)
Verified with 'cabal clean && cabal build all --ghc-options=-Werror' →
exit 0.
Co-Authored-By: Claude Opus 4.7 <[email protected]>
Signed-off-by: Doc Holiday <[email protected]> Signed-off-by: Doc Holiday <[email protected]>
Signed-off-by: Doc Holiday <[email protected]> Signed-off-by: Doc Holiday <[email protected]>