Cardano Updates

Survey pass across CLAUDE.md and every crate/module README to clear
accumulated drift:

- Drop stale `responder_task.rs` references everywhere (the file no
  longer exists; inbound peers run as duplex via the accept loop).
  Updates: top-level README mermaid + workspace tree, net-rs/CLAUDE.md
  peer subtree, net-core/src/peer/README.md, and a comment in
  duplex_task.rs.
- Fix Leios feature blurb in README.md / net-node/README.md: Leios
  produces RBs/EBs/votes, not IBs.
- Bump test count 376 → 406 across README.md and CLAUDE.md to match
  `cargo test`.
- Handshake README: state names Propose/Confirm/Done (no `St` prefix),
  matching the actual `State` enum.
- ChainSync README: `can-await` timeout is 3,673s, not 10s.
- net-cli README: document `scheduler_args.rs` module and the shared
  `--scheduler` / `--protocol-priority` flags.
- net-ui README: list `IconSidebar.tsx` and `ControlPanel.tsx`.
- multi_peer README: refresh the NetworkEvent/NetworkCommand sketch to
  reflect current variants (FetchBlockRange, SubmitTransaction,
  LeiosBlockAnnounced, LeiosBlockTxsOffered, etc.).
- peer README: add `command_dispatch.rs` to the file table.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Tiny curl+python wrapper around the aggregator /api/stats endpoint
that prints a per-node tip/lag/slot/prod/rcvd/val table and flags any
node lagging more than LAG_THRESHOLD behind the max tip. Useful when
chasing consensus jams in live cluster runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Clear out accumulated lints across net-core and net-node: needless
borrows, redundant slicing/closures, manual range checks, a while-let
loop, vec\![] init, and an unused test import.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Before this commit, a block whose parent was unknown to the validator
(not yet applied, not in the in-flight queue) was submitted to the
validator anyway. The fake ledger accepted it silently, but a real
ledger (Acropolis) would reject it with ApplyFailed because its state
is at some other tip, not at the missing parent. The child's cascade
would follow, and the network's useful work would stall.

This commit adds a pending-validation queue so out-of-order arrivals
wait locally until their parent is known, then drain in chain order.

Changes:

- New `in_flight_validation: HashSet<[u8; 32]>` tracking hashes
  submitted to the validator but whose outcome hasn't arrived. Added
  to by `submit_for_validation`, removed by `handle_applied` and
  `handle_apply_failed`.

- New `pending_validation: HashMap<[u8; 32], Vec<(Point, BlockBody)>>`
  keyed by parent hash, holding bodies waiting for their parent to
  become known to the validator.

- New `parent_known_to_validator(prev_hash)` helper: true when the
  parent is None (genesis), already validated, in-flight at the
  validator, or equal to `queued_validator_tip`. This is the gate
  deciding "submit now" vs "queue in pending".

- `on_block_received` tightened dedup (checks `validated` and
  `in_flight_validation` in addition to `block_cache`), branches on
  the gate: either submits directly (and then drains any children
  that were waiting on *this* hash) or stashes in pending_validation.

- `drain_pending(parent_hash)` iterates pending children, submits
  each via `submit_for_validation`, and uses a VecDeque worklist to
  cascade recursively (flattened to avoid async recursion).

- `handle_applied` also drains pending children after inserting the
  newly-validated hash into `validated`, and extends `prune_below_k`
  to drop pending entries whose parent has fallen out of the k-window.

- `handle_apply_failed` drops `pending_validation[failed_hash]` —
  children waiting on a block whose apply failed will never become
  valid via that parent.

- `register_self_produced` drains pending children of the new
  self-produced hash after submitting it to the validator. Covers
  the case where a peer already delivered the child of our
  in-progress block.

Tests added (all in consensus.rs mod tests):
- `child_arriving_before_parent_queued_in_pending`
- `parent_arrival_drains_pending_in_order`
- `self_produced_block_drains_pending_children`
- `deep_chain_out_of_order_drains_correctly` (3-deep reverse delivery)
- `pending_evicted_when_parent_pruned`
- `apply_failed_drops_pending_children`

Cluster verification: 25-node cluster runs cleanly to block 14, all
nodes within lag 1. After pausing production, all 25 nodes converge
at block 17 with zero lag — same as Phase 4b. 73 net-node tests pass,
298 net-core tests pass.

This completes the Phase 4 sequence (4a+4b+4c). The consensus layer
is now fully Haskell-aligned and architected to drop in an Acropolis
ledger without further consensus changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Before this commit, a fork switch synchronously sent InjectRollback +
InjectBlock commands to the coordinator, and a real ledger would never
see the rollback. Peer-fetched blocks were also handed to the validator
without checking that the ledger's current tip matched their parent, so
two competing chains would silently corrupt any real ledger state.

This commit makes consensus drive the ledger (via the Ledger actor) as
the source of truth for chain_store updates:

- `submit_for_validation(point, body, prev_hash)`: new helper that
  issues a `LedgerCommand::Rollback` before `Apply` if the validator
  queue isn't currently aimed at the block's parent, then updates
  `queued_validator_tip` and `adopted_tip_hash` eagerly so subsequent
  decisions see the post-submission view.

- `on_validation_outcome` (replaces `on_validation_complete`) switches
  on all four `LedgerOutcome` variants:
  - `Applied { point }`: mark validated, emit `InjectBlock` to the
    coordinator (the only path for non-self-produced publishes), run
    select_chain, and prune.
  - `RolledBack { target }`: emit `InjectRollback` (the only path for
    chain_store rollbacks).
  - `ApplyFailed`: log, rewind `queued_validator_tip`/`adopted_tip_hash`
    to `last_validated_tip` so the next submission realigns.
  - `RollbackFailed`: log loudly.

- `execute_switch` rewritten: submits `LedgerCommand::Rollback` (if
  needed) then a chain of `Apply` commands. The validator processes them
  in order; outcomes drive the chain_store. Deletes the old `inject_block`
  and `drain_validated_blocks` helpers (redundant now that applies are
  strictly serialized).

- `register_self_produced(point, header, body)` now takes the body and
  routes it through the validator, matching Haskell's
  `ChainDB.addBlockAsync` behaviour — no fast-path for self-produced.
  The `InjectBlock` for self-produced blocks moves from `main.rs:144`
  into the `Applied` outcome handler.

- `main.rs` consumes `LedgerOutcome` directly from the validator's
  receiver and routes each outcome through `on_validation_outcome`.
  The old synchronous `InjectBlock` send after `register_self_produced`
  is gone.

- `validation.rs` drops the `ValidationComplete` shim (and the
  `run_outcome_shim` task). `Validator::new` now returns
  `mpsc::Receiver<LedgerOutcome>` directly and exposes a `submit`
  method for `LedgerCommand`. Tests updated accordingly.

- Tests: existing tests that manually simulated `on_validation_complete`
  are rewritten to use a `drain_validator` pump helper. A new
  `stale_in_flight_eviction_allows_refetch` replaces the old
  `stale_in_flight_evicted_so_fetch_retries` (which exercised a
  degenerate out-of-order scenario that doesn't apply under strict
  ordered validation). New tests added:
  - `applied_outcome_emits_inject_block`
  - `rolled_back_outcome_emits_inject_rollback`
  - `self_produced_runs_through_validator`
  - `queued_validator_tip_tracks_submission_order`

Cluster verification: 25-node cluster runs cleanly to block 27, all
nodes within lag 0. After pausing production (jam test protocol), all
25 nodes converge at block 31 with zero lag — fork switches route
through the validator and emit InjectRollback/InjectBlock in the right
order. 67 net-node tests pass, 298 net-core tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Replace the inline collapser bars on Chain tree, Charts, and Event
log with dedicated toggle icons in the left sidebar. Each panel now
appears/disappears in place while the toggle lives in a fixed
location.

New sidebar icons:
- Chain tree: 3 main-chain blocks with a fork diagonally off the
  middle block, rendered in ChainTreeView's outlined-rectangle style
  (main=#90caf9, fork=#ffb74d).
- Charts: jagged spiky line chart.
- Event log: document with text lines.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The previous validator was a stateless stub: each validate_block call
spawned a detached task that slept and reported success. That can't
host a real validating ledger, where every apply mutates internal
state and depends on the previous apply having completed.

Replaces it with:

- A `Ledger` trait (`apply` / `rollback` / `tip`), `#[async_trait]` so
  `Box<dyn Ledger>` works for runtime selection. The target real impl
  is Acropolis (message-based, async Go/NoGo voting), so the trait is
  async-first.
- `LedgerCommand` (Apply / Rollback) and `LedgerOutcome` (Applied /
  RolledBack / ApplyFailed / RollbackFailed).
- A single long-lived actor task that owns a `Box<dyn Ledger>` and
  processes one `LedgerCommand` at a time. Strict sequential order is
  enforced by construction — the consensus layer no longer needs to
  serialize calls.
- `FakeLedger`: stateless stand-in matching today's behaviour. `apply`
  sleeps for the configured per-block delay (preserving the cluster
  test's wall-clock validation cost) and updates `head`. `rollback`
  updates `head` with no I/O.
- A temporary `ValidationComplete` shim wraps the actor's outcome
  receiver so the consensus layer's existing `on_validation_complete`
  entry point keeps working. This shim disappears once consensus
  consumes `LedgerOutcome` directly (rollback + failure outcomes).

Adds `async-trait` to net-node deps.

Tests:
- delay_computation (regression on the original formula)
- validate_block_completes
- apply_then_apply_processes_in_order (the critical sequential invariant)
- fake_ledger_tracks_head_through_apply_and_rollback
- apply_failure_reported_and_actor_continues (ApplyFailed surfaces and
  the actor keeps draining its queue)

No consensus or main.rs changes — call sites still see the same
`Validator::new` / `validate_block` / `ValidationComplete` shape. 25-node
cluster smoke test reaches block 12 with all nodes within lag 1, no
behavioural regression. 62 net-node tests + 298 net-core tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The eight per-direction variables (RATE_UP_TO_N0, DELAY_N0_TO_UP, etc.)
all carried identical defaults and were never configured differently in
practice. Asymmetric link parameters are not a meaningful assumption for
a synthetic benchmark — propagation delay is a property of the path, not
the direction.

Replace with a single RATE and DELAY, matching the convention already
used in demo/proto-devnet. The TC script is simplified accordingly:
per-peer topology is now expressed as plain PEERS1/2/3 lists rather than
associative arrays, and the bridge-side delay uses a plain netem root
qdisc (as fixed in the previous commit) without any per-source filtering.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

The previous approach attached netem qdiscs to the bridge-side veth
interfaces (veth${i}-br) using filters matching `ip dst PEER_IP`. This
never worked: egress traffic on veth${i}-br goes *toward* node i, so it
carries dst=IP_NODE_i, not dst=PEER_IP. The filters never matched, and
delay was only applied accidentally via the prio qdisc's default band
fallback — and only for nodes whose peer indices happened to land on that
default band. In a 3-node topology, node 2 (the middle node, with peers
1 and 3) consistently received no delay at all because neither peer index
mapped to the default priomap band (1:2).

Fix for demo/proto-devnet (uniform delay across all links): replace the
broken prio+filter+netem on each veth${i}-br with a plain netem root
qdisc. Since all edges share the same $DELAY, no per-source
classification is needed — every packet arriving at node i is delayed
identically regardless of origin.

Fix for demo/burst (per-direction, potentially asymmetric delays):
separate the TC setup into two passes. The egress pass (HTB on each
node's veth) is unchanged. A new ingress pass sets up prio+netem on each
veth${dst}-br, this time filtering by `ip src SRC_IP` so the correct
per-direction delay is applied based on which node sent the packet.

Also removes the unused JITTER variable in demo/burst.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

Bump width 55→72px and add whiteSpace: nowrap so "node-NNNN" no
longer wraps in the event log.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

The shadow select_chain from phase 2.2 is now the primary chain selection
driver. Deleted the old try_fork_switch + fetch_unvalidated_ancestors
walks and the unfetchable scaffolding around them. The two-walk-disagreement
class of jams is gone by construction: there's one walk now.

Changes:

- Renamed ShadowDecision -> SelectionDecision; carries replay/missing data
  needed for execution.
- select_chain_once walks the peer chain newest→oldest finding the first
  hash in the adopted chain's ancestry (not the validated set), classifies
  as Switched / WaitingForBlocks / OrphanCandidate / NoBetterChain.
- select_chain async driver executes Switched (rollback+replay) and
  WaitingForBlocks (range fetch). On OrphanCandidate, the peer is added
  to a per-pass `tried` set and we try the next-best candidate; we do NOT
  remove the peer chain — that would lose all historical announcements,
  which is the only place they're kept. ChainSync won't re-stream blocks
  the peer already acknowledged, so dropping the chain creates an
  irrecoverable orphan loop on lagging nodes.
- on_tip_advanced/on_rolled_back/on_peer_disconnected/on_validation_complete
  /register_self_produced all call select_chain.
- on_tip_advanced no longer inserts headers into chain_tree. chain_tree
  now contains only validated bodies + self-produced blocks (set by
  on_block_received and register_self_produced). Header-only inserts are
  gone.
- BlockFetchFailed clears in_flight and re-runs select_chain. The old
  mark_unfetchable_and_prune scaffolding is gone.
- register_self_produced is now async so it can call select_chain at the
  end (catches the case where a peer fork is still better than the
  newly-produced tip).
- Deleted: try_fork_switch, fetch_unvalidated_ancestors, on_tip_advanced,
  on_rolled_back, mark_unfetchable_and_prune, unfetchable HashSet,
  ChainTree::remove_subtree, ChainTree::contains, ChainTree::find_common_ancestor.
- Deleted obsolete tests: unfetchable_*, fetch_skips_roots_dead_ending_in_unfetchable,
  fork_switch_kicks_fetch_for_missing_intermediate, node_14_jam_regression,
  catchup_fetch_uses_range_not_single_block, fork_switch_works_after_fetch_fills_chain_tree,
  fetch_range_when_parent_unknown, no_cross_fork_range_in_fetch_unvalidated_ancestors,
  remove_subtree_*, common_ancestor_*. ~1300 lines of scaffolding removed.
- Added select_chain-focused tests: orphan candidate, fresh node accepts
  any chain, picks best of multiple candidates, etc.

Cluster verification: 25-node cluster runs cleanly to block 130+, all nodes
within lag 2 throughout. After pausing block production (jam test), all 25
nodes converge at the same tip with zero stuck nodes — better than the
24/25 best the old design achieved with all the unfetchable scaffolding.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

This uses the same approach using a bridge instead of IFB devices.

This reverts commit d41e297a9359d1122bc3532aa4c2375b37137e49.

Initial cluster run showed every shadow decision was "would switch":
the walk checked chain_tree.contains, which under the current design
returns true for unvalidated peer-announced headers too. That's wrong
for the shadow semantics — the "common ancestor" has to be a block we
have validated, not just heard about.

Fix: walk against self.validated. Also handle the genesis case: when
the peer's oldest entry has prev_hash=None, treat the synthetic all-
zero hash as a shared anchor (both our chain and theirs root at
genesis). Without this, any time two nodes self-produced different
first blocks the shadow would declare the peer orphaned.

After the fix, a fresh cluster run shows realistic decision mix:
  8311 would fetch   (normal steady-state, missing_len=1)
    68 would switch  (all blocks validated, tip replacement only)
    12 no common ancestor (genuine orphans)

Added test: shadow_genesis_root_chain_treated_as_waiting.

Tests: 298 net-core + 75 net-node passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

Signed-off-by: Chris Gianelloni <[email protected]>