Cardano Updates

11-experiment everyone sweep (5 NA throughputs + 6 Plutus levels) re-run with
the unbounded-mempool default. Completes the 750n trio alongside the wfa-ls
and top-stake-fraction commits. Includes case.csv, config.yaml, summary.txt,
time.txt for each.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Adds two modules to con-rs:

- `types::Point` (with Display + minicbor Encode/Decode codec).  Both
  consumers must agree on the on-wire encoding of a chain point, so the
  codec lives with the type.  con-rs picks up `minicbor` as a dep.
- `chain_tree` (`ChainTree`, `ChainTreeEntry`, `is_better_tip`,
  `ChainNode`): the fork-aware tree of block headers used by Praos
  longest-chain selection.  Lifted verbatim; HashMap iterations are
  sort-stabilized at the public API boundary, so internal collection
  choice doesn't leak determinism.  Adds `Default` impl and `is_empty`
  to satisfy clippy.

Net-core now depends on con-rs and re-exports `Point` via `pub use
con_rs::Point`, so all callers continue to import from
`net_core::types`.  Net-core's `Tip`, `MAX_POINTS`, `decode_points` /
`encode_points`, and the existing tests stay where they are — they're
wire-protocol concerns.

Net-node's `chain_tree.rs` is deleted; imports in telemetry, consensus
mod, and the praos sub-layer are rewritten to `con_rs::chain_tree::*`.

Verified: 57 con-rs tests pass (was 42 — +4 Point tests, +15 chain_tree
tests now run in con-rs), 529 net-rs workspace tests pass (was 544 —
the 15 chain_tree tests moved out), cargo clippy --all-targets shows 11
warnings in net-rs (baseline) and 0 in con-rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Aligns PeerChain with Haskell's AnchoredFragment pattern: the
ChainSync intersection point is stored as an explicit anchor on
each peer's candidate chain. When select_chain_once's walk through
PeerChain entries fails to find a common ancestor (narrow window
after reconnection), the anchor provides a guaranteed fallback.

Changes:
- Forward NetworkEvent::IntersectionFound from coordinator to
  consensus (previously swallowed at the coordinator level)
- Add PeerChainAnchor struct and anchor field to PeerChain, set
  from IntersectionFound, persists through rollbacks
- Add anchor fallback in select_chain_once after the walk and
  prev_hash fallback both fail
- Detect gap between anchor and oldest PeerChain entry; thread
  anchor_point through WaitingForBlocks to issue_fetch so the
  fetch range starts from the anchor, filling the gap via
  BlockFetch

Three new tests: anchor_set_on_intersection_found,
anchor_used_as_fallback_ancestor, anchor_ignored_when_walk_succeeds.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Trim roadmap to what was actually implemented, document module
structure and test coverage, add next steps: mempool-driven EB
production, stake-weighted quorum, telemetry events.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Receiver decodes the EB manifest on LeiosBlockReceived, caches the
ordered tx_hash list per EB, and on LeiosBlockTxsOffered builds a
sparse bitmap of indices NOT present in its mempool. Consensus
cuts wire bytes proportional to mempool overlap — local production
plus prior tx dissemination already cover most txs in steady state.

Fallback when the offer arrives before our EB fetch: select_all up
to MAX_BITMAP_ENTRIES * 64 (the protocol's maximum). The server
returns whatever subset it actually has.

New helpers: production::decode_overflow_eb (pure decoder, paired
with make_overflow_eb) and Mempool::current_tx_ids (HashSet for
O(1) membership). LeiosConsensus now owns SharedMempool and an
eb_tx_hashes cache, plumbed through Consensus::new from main.rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Add try_stored_switch(): walk back from chain_tree's best tip to
the adopted tip and switch to the longest contiguous prefix of
cached blocks. Finds switchable chains from ANY source regardless
of which peer announced them — blocks from multiple peers that
form a contiguous path all get applied.

Called at the top of select_chain before peer-chain evaluation.
Replaces the per-peer partial switch that could only see one
peer's replay at a time.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

When an EB election reaches quorum and enters CertEligible phase,
the next RB produced by this node includes certified_eb=true in its
header (CIP-0164 11-field extended header). Receiving nodes parse the
flag via the existing HeaderInfo Leios extension fields.

End of Leios consensus MVP: EB → votes → quorum → cert in RB header.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

LeiosStore's votes / eb_tx_hashes / blocks / block_txs maps had no
slot-based eviction — receivers accumulated every vote and every
EB manifest forever. With actual EB and voting traffic, that's
~600 vote entries per EB committee × every EB seen = ~70 MB/s of
unbounded growth on a 25-node cluster. Pre-codec-fix, broken tx
flow meant few EBs / votes flowed and the leak was hidden.

Add max_slot + retention_slots to LeiosStoreInner. Each inject_*
updates max_slot; bump_version evicts entries with slot < max_slot
- retention_slots. Default retention is 100 slots, sized for the
13-slot Linear Leios pipeline plus headroom — far smaller than
the LeiosTracker dedup window (1000), since the tracker stores
tiny offer IDs while this store holds full bodies.

new_with_retention exposes the knob for explicit configuration.
Notifications (small fixed-size enum entries used by LeiosNotify
with absolute read indices) are intentionally left growing for
now; pruning them requires reworking the read_index protocol.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

When nodes diverge early under p=0.2, they end up on separate forks
sharing no common blocks — the only common point is Origin (genesis).
Previously, select_chain_once refused to roll back an adopted node to
Origin, causing permanent fork sticking (observed: 3/25 nodes stuck,
628K orphan log lines per node).

Three changes in praos.rs:

1. select_chain_once: remove the adopted_tip_hash.is_none() guard on
   the genesis fallback. Genesis is a universal common ancestor; the
   peer was already filtered to be strictly better by is_better_tip.
   The entire peer chain becomes the replay set.

2. execute_switch: when ancestor is [0u8;32] (Origin), submit
   Rollback { target: Point::Origin } and reset both
   queued_validator_tip and adopted_tip_hash to None. This puts the
   node back in fresh-node state so the first replay block
   (prev_hash=None) aligns correctly in submit_for_validation.

3. handle_rolled_back: handle Point::Origin by setting
   last_validated_tip to None and sending InjectRollback. Previously
   Origin fell through the _ => return arm, silently skipping the
   chain store rollback.

Cluster testing confirms: zero orphan log lines for genesis-diverged
forks. Nodes that previously got permanently stuck now find Origin as
the common ancestor and switch to the longer chain. Remaining lag
under p=0.2 is fetch throughput under churn, not structural.

Two new tests: select_chain_accepts_genesis_ancestor_for_adopted_node,
handle_rolled_back_origin_sends_inject_rollback.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The wire format gives no per-body index in MsgLeiosBlockTxs, so a
partial or out-of-order server response can't be interpreted by
position alone. Each response body is now hashed with blake2b-256
and looked up in the cached manifest, restricted to the indices we
actually requested.

LeiosConsensus tracks the pending request bitmap per EB and on
match_eb_tx_response returns the verified bodies in manifest order
along with a requested count. main.rs forwards only verified bodies
to the validator and warns on partial responses, setting up the next
stage where the missing indices can be re-fetched from another peer.

Three new tests: keep-manifest-hashes-only (with bogus mixed in),
unknown-manifest passes through, pending bitmap consumed after match.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Three fixes for stuck forks under p=0.2:

- tip_hash()/next_block_number() now return adopted_tip_hash instead
  of chain_tree.best_tip_hash(), preventing block production on top of
  unvalidated peer headers with incomplete chain_tree ancestry

- PeerChain::rollback_to clears entries when rolling back to the
  anchor (ChainSync intersection), preventing non-contiguous entries
  from multiple forks accumulating after a failed rollback

- select_chain_once verifies chain_tree contiguity before returning
  Switched: walks ancestors from the last replay block and checks all
  replay hashes lie on that chain and it reaches the ancestor/genesis.
  Mixed-fork replays become OrphanCandidate; genuine gaps become
  WaitingForBlocks with a fetch

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

LeiosConsensus.eb_tx_hashes (EB manifest cache) and
pending_eb_tx_fetches (per-EB requested-bitmap cache) were inserted on
every received EB but never pruned. With ~5 EBs/sec across the cluster
this map grew indefinitely. Tag each entry with the EB's announced slot
and drop entries whose age has passed pipeline expiry in on_slot. Doing
the cleanup independently of `elections.retain` covers the case where
an EB is received but the validator never produces a corresponding
election entry — without that the manifest would still leak.

Before this commit, a fork switch synchronously sent InjectRollback +
InjectBlock commands to the coordinator, and a real ledger would never
see the rollback. Peer-fetched blocks were also handed to the validator
without checking that the ledger's current tip matched their parent, so
two competing chains would silently corrupt any real ledger state.

This commit makes consensus drive the ledger (via the Ledger actor) as
the source of truth for chain_store updates:

- `submit_for_validation(point, body, prev_hash)`: new helper that
  issues a `LedgerCommand::Rollback` before `Apply` if the validator
  queue isn't currently aimed at the block's parent, then updates
  `queued_validator_tip` and `adopted_tip_hash` eagerly so subsequent
  decisions see the post-submission view.

- `on_validation_outcome` (replaces `on_validation_complete`) switches
  on all four `LedgerOutcome` variants:
  - `Applied { point }`: mark validated, emit `InjectBlock` to the
    coordinator (the only path for non-self-produced publishes), run
    select_chain, and prune.
  - `RolledBack { target }`: emit `InjectRollback` (the only path for
    chain_store rollbacks).
  - `ApplyFailed`: log, rewind `queued_validator_tip`/`adopted_tip_hash`
    to `last_validated_tip` so the next submission realigns.
  - `RollbackFailed`: log loudly.

- `execute_switch` rewritten: submits `LedgerCommand::Rollback` (if
  needed) then a chain of `Apply` commands. The validator processes them
  in order; outcomes drive the chain_store. Deletes the old `inject_block`
  and `drain_validated_blocks` helpers (redundant now that applies are
  strictly serialized).

- `register_self_produced(point, header, body)` now takes the body and
  routes it through the validator, matching Haskell's
  `ChainDB.addBlockAsync` behaviour — no fast-path for self-produced.
  The `InjectBlock` for self-produced blocks moves from `main.rs:144`
  into the `Applied` outcome handler.

- `main.rs` consumes `LedgerOutcome` directly from the validator's
  receiver and routes each outcome through `on_validation_outcome`.
  The old synchronous `InjectBlock` send after `register_self_produced`
  is gone.

- `validation.rs` drops the `ValidationComplete` shim (and the
  `run_outcome_shim` task). `Validator::new` now returns
  `mpsc::Receiver<LedgerOutcome>` directly and exposes a `submit`
  method for `LedgerCommand`. Tests updated accordingly.

- Tests: existing tests that manually simulated `on_validation_complete`
  are rewritten to use a `drain_validator` pump helper. A new
  `stale_in_flight_eviction_allows_refetch` replaces the old
  `stale_in_flight_evicted_so_fetch_retries` (which exercised a
  degenerate out-of-order scenario that doesn't apply under strict
  ordered validation). New tests added:
  - `applied_outcome_emits_inject_block`
  - `rolled_back_outcome_emits_inject_rollback`
  - `self_produced_runs_through_validator`
  - `queued_validator_tip_tracks_submission_order`

Cluster verification: 25-node cluster runs cleanly to block 27, all
nodes within lag 0. After pausing production (jam test protocol), all
25 nodes converge at block 31 with zero lag — fork switches route
through the validator and emit InjectRollback/InjectBlock in the right
order. 67 net-node tests pass, 298 net-core tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Add a shared Mempool accumulator that collects transactions from both
local generation and peer receipt. When producing an RB, the mempool
determines the path: if total pending bytes fit within rb_body_max_bytes,
txs go in the RB body directly; otherwise ALL txs drain into an EB
manifest (list of tx hashes) and the RB body is empty with the EB
announced in the header's announced_eb field.

- mempool.rs: Mempool struct with push/drain_all/drain_up_to, shared via
  Arc<Mutex>; tx_from_received_bytes for peer tx accumulation
- config.rs: rb_body_max_bytes (default 64KB), mempool_capacity (10K)
- production.rs: ProducedRb with optional announced_eb, make_fake_block
  encodes txs in CBOR tx_bodies map, make_overflow_eb builds content-
  addressed EB manifests [slot, [tx_hash, ...]]
- main.rs: wire mempool to generator and main loop, replace stage-boundary
  EB production with overflow-triggered path

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Mechanical refactor — no behavioral changes. Split the 2900-line
praos.rs into separate files by concern:

  praos/mod.rs        — PraosConsensus struct, public API, integration tests
  praos/peer_chain.rs — PeerChain tracking (entries, anchors, rollbacks)
  praos/selection.rs  — chain selection (select_chain_once, try_stored_switch)
  praos/fetching.rs   — fetch decisions (issue_fetch, in_flight, retry)
  praos/validation.rs — validator pipeline (apply, rollback, block received)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Two fetch pipeline fixes for lagging nodes:

1. FetchBlockRange carries optional peer_id hint so the coordinator
   routes directly to the peer that announced the chain, bypassing
   fragment-based lookup (fragments lose points after rollbacks).

2. ChainStore::get_range returns available blocks when the requested
   `to` point isn't in the store (peer rolled back past that tip).
   Previously returned empty/NoBlocks, starving lagging nodes.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Add three telemetry events surfacing the Leios certification path:
- LeiosQuorumReached { node, eb_slot, voted_stake, voters } — per-node,
  fired when local vote tally first crosses quorum_stake_fraction × total_stake.
  Useful for quorum-propagation latency analysis (multiple per EB).
- RbCertifiedEb { node, rb_slot, eb_slot } — per-RB, fired only on the
  producing node when the emitted RB header carries certified_eb=true.
- LeiosElectionExpired { node, eb_slot, had_quorum, voted_stake, voters } —
  fired when an election is pruned past leios_dedup_window with final tally.

record_vote now returns Option<QuorumFormed> so the caller knows when
quorum first fires. LeiosConsensus buffers these in pending_telemetry,
drained by main.rs after on_slot/on_validation_outcome.

EventLog UI:
- Default unmapped-type color was #999 (3-char hex), which became invalid
  5-char #99922 after alpha concatenation, causing inconsistent rendering
  for TipAdvanced and other unmapped events. Use #999999 + complete the
  color map for all common types.
- RbCertifiedEb gets a distinctive gold-glow highlight (bigger, bold,
  thicker border, stronger bg, box shadow) since it's the moneyshot event.

Cluster-verified: 275 LeiosQuorumReached and 11 RbCertifiedEb (= 12 RBs
minus 1 first block before any cert was eligible) over 12 EBs in a 5-min
run with EveryoneVotes + tx_rate=2.0. LeiosElectionExpired fires past
the 1013-slot pipeline+dedup window (covered by unit test).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

When an EB election enters the Voting pipeline phase (3×Δhdr slots
after announcement), committee selection determines whether this node
votes. Structured 130-180B vote bodies are injected into the network
via InjectLeiosVotes. Stage-boundary vote production removed from main.

Verified in 25-node cluster: EBs propagate, elections track pipeline
phases, votes flood the network (~1200 VotesReceived in ~10s run).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Each validated EB gets its own election with phases driven by pipeline
timing (3×Δhdr + L_vote + L_diff). Phase transitions computed from
elapsed slots since announcement. Elections pruned after dedup_window.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

glibc's allocator holds freed pages on its freelist and returns them
to the OS lazily, inflating RSS under heavy small-allocation churn.
At 50 TPS the cluster grew to ~10 GB with glibc but ~7 GB with
jemalloc — a ~30% reduction with no other change.

tikv-jemallocator is the standard Rust binding and is already used
by other Cardano-adjacent tools.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Folds cluster-status.sh, tx generation count, Leios pipeline tally
(eb elections / votes / quorum), recent block production, telemetry
event-type summary, and node-0 validation lag into one report.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Capture the next-steps plan for the Leios consensus layer: validation
delay wiring, per-stage election tracking, vote aggregation and
certificate formation, and RB headers advertising certified EBs.

Document the invariants (maintained and violated) of ChainTree,
PeerChain, ChainStore, ChainFragment, and select_chain_once as doc
comments. This makes the correctness requirements explicit for
fork convergence under high block production (p=0.2).

Three bug fixes:
- ChainTree::prune_below() now recomputes best_tip if the current
  best was pruned, preventing a dangling reference.
- on_block_received() now inserts into chain_tree even when the
  header is opaque, using fallback block_no/prev_hash to avoid
  gaps that break ancestors() walks.
- FetchBlockRange routing now checks fragment.contains() for both
  endpoints (from and to), not just to, preventing misrouted
  fetches to peers that lack the range start.

Four new invariant-encoding tests for ChainTree: ancestors_stops_at_gap,
ancestors_reaches_genesis, prune_preserves/recomputes/clears_best_tip.

Known remaining issue: Origin-as-ancestor is still broken for adopted
nodes (select_chain_once rejects genesis rollback when adopted_tip_hash
is Some). Observed in cluster testing: 3/25 nodes stuck on solo forks
with adopted_ancestors_len=2 despite adopted_block_no=67.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>