Cardano Updates

Signed-off-by: Sasha Bogicevic <[email protected]>

# Description

Forkers have been a source of headaches for a very long time already,
and all due to the limitation of the Resource registry that enforces
registries to only be used from known threads. This has lead to a
situation where we had hierarchies of registries just to satisfy that
rule and even we had to extend the resource-registry API to allow
transferring between registries which is unsafe.

The general pattern followed now is:
1. At the ledgerdb level we only keep track of the resources, running
WithTempRegistry while opening the DB and then closing them when closing
the LedgerDB.
2. Forkers that open new handles are open in a `with*` combinator style,
so the handles are deallocated when exiting the scope (or committed).
3. Forkers that only duplicate existing handles (read-only forkers) are
only used in Mempool and LocalStateQuery and there we put them in a
registry, in particular in the top level registry.
4. The handles are not tracked directly---each handle is tracked
indirectly because some forker or the LedgerDB owns it and is tracked.
Forkers are tracked so that they can be released promptly which avoids
space-leaks and permits the backends (LSM/LMDB) to do their gradual
bookkeeping/rearranging work.
5. In particular, when the node is shutting down, there is no need to
close each handle/forker:
   1. in InMemory it doesn't matter as it is pure
2. in LSM, closing the session will close any open handles, and closing
the session happen when shutting down the node,
3. in LMDB, each forker has one single read-transaction open and it will
be closed as described above. There are no further resources allocated
on each step. Also closing the backing store is optional as described in
the LMDB docs (also mentioned in LedgerDB API).

Sadly, the change is so fundamental that there is no meaningful way of
splitting the changes into reviewable commits that compile on their own.

Proposed review order (recommended to disable whitespace-diff in Github
diff view and hide already viewed files):

1. Ouroboros.Consensus.Storage.ChainDB.API
5. Ouroboros.Consensus.MiniProtocol.LocalStateQuery.Server +
Ouroboros.Consensus.Network.NodeToClient
7. Ouroboros.Consensus.Mempool.{Impl.Common, Update, Init}
8. Ouroboros.Consensus.Storage.ChainDB.ChainSel
9. Ouroboros.Consensus.Storage.ChainDB.Impl.Types +
Ouroboros.Consensus.Storage.ChainDB.Impl.Query
10. Ouroboros.Consensus.Storage.ChainDB.Impl
11. Ouroboros.Consensus.Storage.ImmutableDB.{Impl, Impl.Validation}
12. Ouroboros.Consensus.Storage.VolatileDB.Impl
13. Ouroboros.Consensus.Storage.LedgerDB.API (both the API changes and
the initialization)
14. Ouroboros.Consensus.Storage.LedgerDB
15. Ouroboros.Consensus.Storage.LedgerDB.Forker
16. Ouroboros.Consensus.Node (just comments changes)
17. Ouroboros.Consensus.NodeKernel
18. Ouroboros.Consensus.Storage.LedgerDB.V2.LedgerSeq
19. Ouroboros.Consensus.Storage.LedgerDB.V2.Forker
20. Ouroboros.Consensus.Storage.LedgerDB.V2.Backend
21. Ouroboros.Consensus.Storage.LedgerDB.V2.InMemory
22. Ouroboros.Consensus.Storage.LedgerDB.V2.LSM
23. Ouroboros.Consensus.Storage.LedgerDB.V2    
24. Ouroboros.Consensus.Storage.LedgerDB.V1.Forker
25. Ouroboros.Consensus.Storage.LedgerDB.V1.Snapshots
26. Ouroboros.Consensus.Storage.LedgerDB.V1

The rest of the changes are just adapting tests or adapting the
cardano-tools to use the new access pattern. I did not do any meaningful
change in the tests by my will, I only followed GHC errors where they
took me.

This is just for maven and iOSX64 was giving an expected error, so deactivated.

* Add ./script/nix-format.sh for non-Nix users.
* Add nixfmt in our Nix pre-commit hook.
* Add Github CI check

State machine (StIdle/StBusy/StDone), CBOR codec, client helpers
(request_next, done), and security audit. Protocol ID 18. All Leios
types are opaque blobs. 19 new tests including allocation bounds,
codec round-trips, and MemBearer integration. 191 total tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Switch mux from RoundRobin to StrictPriority scheduler so Praos
protocols always outprioritize Leios, preventing head-of-line blocking.
Add named priority constants, fix KeepAlive/PeerSharing/Leios priority
assignments, consolidate deferred items into Future Work section.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Convert WrappedHeader from opaque tuple struct to named-field struct
with parsed HeaderInfo. Parser navigates era-tagged #6.24-wrapped CBOR
to extract block_number, slot, prev_hash, issuer_vkey, body_size, and
block_body_hash, plus CIP-0164 optional Leios fields (announced_eb,
certified_eb). Array length alone disambiguates which optional fields
are present (10=none, 11=certified_eb, 12=announced_eb, 13=both).

Convert BlockBody similarly with parsed LeiosBlockInfo. Block parser
extracts the optional eb_certificate from the 5th element of the
Shelley+ block array. Byron headers/blocks return None gracefully.

Box InjectBlock header to fix large_enum_variant. Update all call
sites (~20 files). 238 total tests, 17 new parser tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Thread-per-peer coordinator with peer-agnostic application interface.
Each peer runs independent tokio task tree (ChainSync, BlockFetch,
KeepAlive, PeerSharing sub-tasks). Coordinator aggregates tips with
deduplication, routes fetch requests to best peer by RTT, and
reconnects failed peers with exponential backoff (1s-30s).

New modules: net-core/src/peer/ (types, peer_task, coordinator,
connect). Connection helpers moved from net-cli to net-core.
New CLI: multi-follow --host <addr> [--host <addr>...].
ConnectionMode enum supports future ResponderOnly/Duplex modes.

153 tests (147 existing + 6 new). Live-tested against mainnet.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Hierarchical documentation: top-level README links to net-core/ and
net-cli/, net-core links to bearer/mux/types/protocols/peer modules,
protocols/ links to each of the 8 protocol subdirectories. Each
protocol README includes Mermaid state machine diagram, agency table,
limits, and API entry points. All links point to directories (not
README.md) so GitHub shows files alongside docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The server-side Request/Response enums were 1:1 mappings of Message
variants, and receive_request()/send_response() were pure conversion
boilerplate. The Runner already enforces agency and state transitions,
so these types added no safety.

Server code now uses runner.recv()/runner.send() with Message directly,
matching the simplicity of the protocol framework design.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Document all Phase 2 deliverables: shared types, ChainSync/BlockFetch/
KeepAlive protocols (client + server), persistent chain follower, fake
server with Poisson blocks/rollbacks, serve.rs in workspace structure,
server-uses-Message-directly design decision. Add Phase 3 detail for
TxSubmission and PeerSharing protocols.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Refactor the 1086-line types.rs into a directory module:
- types/mod.rs: Point, Tip, encode/decode_points, constants, re-exports
- types/header.rs: WrappedHeader, HeaderInfo, Shelley+ header parser
- types/block.rs: BlockBody, LeiosBlockInfo, block body parser

All import paths preserved via re-exports. 238 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

codec.rs wraps mux channels with CBOR framing — belongs in mux/.
protocol.rs defines the protocol state machine framework — belongs
in protocols/. Re-export from parent modules so imports become
crate::mux::{CodecSend,CodecRecv} and crate::protocols::{Protocol,
Runner,Agency,Role,ProtocolError}. Update all ~40 import sites.
238 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Wire LeiosNotify (ID 18) and LeiosFetch (ID 19) into the per-peer task
architecture behind a `leios_enabled` config flag (default false).

- LeiosStore: content-addressed blob store for EBs/votes (separate from
  ChainStore since Leios data is keyed by (slot, hash), not a linear chain)
- Client tasks: spawn_leios_notify (continuous request_next loop),
  spawn_leios_fetch (command-driven, like BlockFetch)
- Server handlers: serve_leios_notify (from LeiosStore + subscribe),
  serve_leios_fetch (block/txs/vote lookups)
- Types: 6 PeerEvent, 2 PeerCommand, 5 NetworkEvent, 3 NetworkCommand variants
- Coordinator: stub-forwards Leios events, populates LeiosStore on fetch
- Wiring: peer_task, responder_task, duplex_task all conditionally register
  and spawn Leios protocol sub-tasks
- CLI: --leios flag on serve (synthetic EB/vote generation) and multi-follow
  (logs Leios notifications) for local end-to-end testing

247 total tests (9 new). Locally tested: serve --leios -> multi-follow --leios.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Returns fake peers (192.0.2.x documentation addresses) when
a client requests peer sharing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Mux now supports both directions on one connection via composite
(ProtocolId, u16) channel keys. register_with_mode() lets callers
specify direction explicitly; register() remains backward compatible.
Demuxer routes by (protocol_id, mode) — no single-mode validation.
Scheduler stays keyed by ProtocolId (both directions share priority).

New: DuplexConnection, connect_duplex(), accept_duplex() for
bidirectional connection setup. DuplexTask combines client + server
protocol sub-tasks on one mux. Coordinator spawns duplex tasks when
config.duplex is set. multi-follow --duplex flag.

All three ConnectionMode variants now implemented:
- InitiatorOnly: outbound, client protocols
- ResponderOnly: inbound, server protocols
- Duplex: both on one connection

172 tests. Live-tested duplex against mainnet (version 15).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Configurable maximum rollback depth (default 3), always capped at
chain length - 1 to avoid rolling back past genesis.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Replace stub-forwarded Leios events with smart coordinator logic:
- Slot-bounded dedup for EB, TX, and vote offers across peers
- Per-offer peer tracking for RTT-based smart fetch routing
- Pending fetch dedup and cleanup on peer failure
- Separate LeiosBlockTxsOffered/LeiosBlockTxsReceived events
- FetchLeiosBlockTxs (with bitmap) and FetchLeiosVotes commands
- leios_dedup_window config (default 1000 slots)
- 8 new coordinator tests (255 total)
- Live-tested: serve --leios → multi-follow --leios (two connections)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Implement PeerSharing mini-protocol (protocol ID 10) — simple
request/reply for peer discovery with IPv4/IPv6 address support.
CLI 'peer-share' command with peer_sharing=1 handshake negotiation
and graceful rejection when node doesn't support it.
18 new tests (147 total), live-tested against mainnet relays.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

All 6 N2N mini-protocols now implemented (147 tests). Multi-peer
coordination layer remains TBD for Phase 3.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>