refactor(cardano): shard mem-hungry work units (#978)
* refactor(cardano): shard EWRAP into prepare/shard/finalize work units
Partitions the epoch-boundary EWRAP work unit — previously one monolithic
work unit that materialised O(active_accounts) in memory (rewards map,
deltas, logs, applied_rewards) — into three phase-specific work units that
each commit independently:
* EwrapPrepare: global classification (pools/dreps/proposals), MIRs,
enactment + refund visitors for non-account entities, emits EpochEndInit
seeding EpochState.end with the prepare-time globals and zeroed reward
accumulators.
* EwrapShard(i): range-scoped (first-byte prefix bucket) load of pending
rewards + accounts, runs rewards + drops visitors per account, emits
EpochEndAccumulate with the shard's reward contribution.
* EwrapFinalize: reads the accumulated EpochState.end, emits EpochWrapUp
(which transitions rolling/pparams snapshots and clears ewrap_progress).
Cross-shard handoff piggy-backs on EpochState rather than a new entity:
ewrap_progress: Option<u32> is the durable cursor and EpochState.end
accumulates across shards via the new deltas.
WorkBuffer gains EwrapShardingBoundary{shard_index, total_shards} and
EwrapFinaliseBoundary states; pop_work now takes ewrap_total_shards from
CardanoConfig (default 16). EpochEndAccumulate has an idempotency guard
keyed on ewrap_progress so shard re-execution after a crash is safe.
Detection-only crash recovery at initialize time logs a warning when
ewrap_progress is set; full block-rehydration resume is flagged as TODO.
Memory tests in tests/memory.rs verify both fjall and redb3 honour
range-scoped iter_entities with O(1) heap — the load-bearing property for
the shard design.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): rename EwrapPrepare → Ewrap, seed EpochState.end in ESTART
Decouple two responsibilities that were tangled in EwrapPrepareWorkUnit:
the global epoch-boundary entity processing (now plain `Ewrap`) and the
structural opening of the `EpochState.end` slot (now done by ESTART's
`EpochTransition`). Ewrap's `EpochEndInit` delta keeps its overwrite
semantics; it now writes into a default-seeded slot rather than from
None.
Also adds `prev_end` / `prev_ewrap_progress` undo fields to
`EpochTransition` (serialized, like the other prev_* fields) so a
rollback after restart correctly restores them.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): rename EwrapShard → AccountShard
The per-account leg of the epoch close was named after its position in the
EWRAP pipeline; AccountShard names what it actually does — apply rewards
and pool/drep delegator drops over a key-range slice of the account
namespace.
Also renames the related symbols (BoundaryWork::load_shard /
commit_shard → load_account_shard / commit_account_shard,
WorkBuffer::EwrapShardingBoundary → AccountShardingBoundary,
InternalWorkUnit::EwrapShard → AccountShard). The user-facing
`ewrap_total_shards` config field is intentionally preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): run AccountShard before Ewrap on the boundary pipeline
The epoch-boundary sequence is now AccountShard ×N → Ewrap → EwrapFinalize
(was Ewrap → AccountShard ×N → EwrapFinalize). Per-account work settles
first; the global Ewrap phase then patches the prepare-time fields onto an
EpochState.end that already has its reward accumulators populated.
State machine: WorkBuffer::pop_work transitions reordered, and
on_ewrap_boundary now takes ewrap_total_shards so the restart-at-boundary
entry can construct AccountShardingBoundary directly. The
total_shards == 0 defensive branch now skips to EwrapBoundary (global
phase) instead of EwrapFinaliseBoundary.
Delta semantics:
- EpochEndInit::apply is now a PATCH — writes only the prepare-time
fields (pool counts, epoch_incentives, MIR amounts, proposal refunds)
and leaves the accumulator fields alone. ewrap_progress is no longer
touched by this delta. Dropped the unused prev_ewrap_progress field.
- EpochEndAccumulate::apply treats ewrap_progress = None as the natural
starting state for shard 0 (unwrap_or(0) as the expected cursor).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): merge EwrapFinalize into Ewrap; drop EpochEndInit delta
The boundary close is now a single Ewrap work unit: it runs the global
visitors AND emits EpochWrapUp carrying the assembled final EndStats
(prepare-time fields combined with the AccountShard-populated accumulator
fields). The wrap-up visitor now constructs the final stats locally
instead of routing them through a separate EpochEndInit delta.
Side-benefits: one fewer state-machine state, one fewer delta type, one
fewer commit cycle. Atomicity also improves — the boundary close is now
a single state-writer commit, so a crash between Ewrap and EwrapFinalize
is no longer possible.
Test fixture in tests/epoch_pots/main.rs restructured to match the
post-reorder pipeline: accumulator reset gates on AccountShard
shard_index == 0; rewards CSV is dumped on the Ewrap arm.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): seed EpochState.end in Genesis
After the boundary-pipeline reorder (AccountShard runs before Ewrap),
the first epoch's AccountShard hits `EpochEndAccumulate::apply` with
`entity.end == None` because Genesis bootstrapped the EpochState before
ESTART's `EpochTransition` had a chance to seed the slot. Seed
`end = Some(EndStats::default())` directly in Genesis to match the
invariant ESTART maintains for every subsequent epoch.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): split AccountShard into its own `ashard` module
Pulls the AccountShard work unit out of `ewrap/` into a peer module
`ashard/` (matching the layout of `estart/`, `rupd/`, `roll/`,
`genesis/`). The shared `BoundaryWork` / `BoundaryVisitor` infrastructure
and the drops visitor (used by both phases) stay in `ewrap/`; `ashard/`
imports them.
Moves: `rewards.rs`, `shard.rs`, `AccountShardWorkUnit` (from
`work_unit.rs`), and the `load_*` / `commit_*` impl blocks. Visibility
on shared `BoundaryWork` helpers (`new_empty`, `load_pool_data`,
`load_drep_data`, `stream_and_apply_namespace`) widened from private to
`pub(crate)`. The `ending_state` field also widened to `pub(crate)` so
peer modules can mutate it (e.g. `wrapup.flush` already does this).
Method/identifier renames to match the new module path:
- `BoundaryWork::load_account_shard` → `load_ashard`
- `BoundaryWork::commit_account_shard` → `commit_ashard`
- `WorkUnit::name()` returns `"ashard"`
Type name `AccountShardWorkUnit` is preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* docs(cardano): fix stale references in EWRAP/AccountShard refactor comments
Sweeps the docstrings/comments touched in this PR for references to
phases, work units, and deltas that no longer exist after the rename /
reorder / merge / split sequence:
- Restore the in-place explanation for the "rewards before drops"
HACK in `ashard/loading.rs` (the dangling "see comment on the
pre-shard path" pointed to a comment that was deleted when the
prepare phase was removed).
- Drop "prepare phase" / "finalize phase" wording from `BoundaryWork`
field docstrings, `commit_ewrap` comments, and `loading.rs` section
dividers — neither phase exists; there's only Ewrap (global + close)
and AccountShard (per-account).
- Update the ESTART `EpochTransition` description in `work_units.md`
so it reflects the post-merge data flow: AccountShards populate the
accumulators directly, then Ewrap reads them back and emits
`EpochWrapUp` with the final `EndStats` (no `EpochEndInit` patch step
anymore).
- Rename `compute_prepare_deltas` → `compute_ewrap_deltas`. The
"prepare" name was a leftover from the `EwrapPrepare` work unit; the
method is now the only Ewrap-phase compute helper.
- Tighten `load_pending_rewards_range` docstring; flag that the `None`
branch is currently unused.
No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): decouple shard naming from `ewrap`
The shard-related identifiers and comments were named after the legacy
EWRAP pipeline that bundled the global epoch-boundary work and the
per-account shards together. With AccountShard now a distinct work unit
in its own module, those names are misleading. Rename to use the
`ashard` prefix consistently with the module path:
- `CardanoConfig::ewrap_total_shards` → `ashard_total`
- `CardanoConfig::DEFAULT_EWRAP_TOTAL_SHARDS` → `DEFAULT_ASHARD_TOTAL`
- `EpochState::ewrap_progress` → `ashard_progress`
- `prev_ewrap_progress` → `prev_ashard_progress` on `EpochEndAccumulate`,
`EpochWrapUp`, and `EpochTransition`
- `WorkBuffer::receive_block` / `on_ewrap_boundary` / `pop_work`
parameter `ewrap_total_shards` → `ashard_total`
- Error messages in `ashard/shard.rs` updated to match.
Also fixes comment / doc misattributions where "EWRAP" was used for
work that's now in `AccountShard`:
- `PendingRewardState` / `DequeueReward` are consumed by `AccountShard`,
not Ewrap.
- `PendingMirState` / `DequeueMir` are consumed by Ewrap (clarified).
- `AppliedReward` and the `applied_rewards` field are populated during
AccountShard, not Ewrap.
- RUPD's docstring now says rewards are consumed by `AccountShard`.
- Crash-recovery wording in `lib.rs` says "mid-boundary" instead of
"mid-EWRAP" since the cursor specifically tracks AccountShard
progress.
BREAKING CONFIG CHANGE: existing `dolos.toml` files that explicitly set
`ewrap_total_shards` need to rename the key to `ashard_total`. Users
relying on the default (omitted) are unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): rename AccountShard → AShard for structs and variants
Aligns the type and variant names with the module path convention:
- struct `AccountShardWorkUnit` → `AShardWorkUnit`
- enum variant `CardanoWorkUnit::AccountShard` → `AShard`
- enum variant `InternalWorkUnit::AccountShard` → `AShard`
- WorkBuffer state `AccountShardingBoundary` → `AShardingBoundary`
- module re-export and all callers updated to match
- prose / docstrings / log messages also use `AShard` consistently
The module path is `crate::ashard`, so the type now reads as
`ashard::AShardWorkUnit`. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): rename `ashard_total` → `account_shards`
Per review feedback: the user-facing config name should be self-explanatory
in `dolos.toml`. Renames everywhere for consistency:
- `CardanoConfig::ashard_total` field → `account_shards`
- `CardanoConfig::ashard_total()` accessor → `account_shards()`
- `CardanoConfig::DEFAULT_ASHARD_TOTAL` → `DEFAULT_ACCOUNT_SHARDS`
- WorkBuffer parameters and error messages updated to match.
BREAKING CONFIG CHANGE: existing `dolos.toml` files that explicitly set
this option (under any prior name from this PR) need to use
`account_shards`. Users relying on the default are unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(cardano): persist shard count alongside ashard_progress
Guards against a config change to `account_shards` corrupting an
in-flight boundary. Previously, if dolos crashed mid-boundary and the
operator changed `account_shards` between crash and restart, the resume
would re-partition the account key space with the new count, mismatching
the cursor's already-committed shards.
Fix: snapshot the boundary's shard count into state at the first
`EpochEndAccumulate` apply. The persisted total is authoritative for the
duration of the in-flight boundary; the new config value only takes
effect on the next boundary.
Changes:
- New `AShardProgress { committed, total }` struct stored at
`EpochState.ashard_progress: Option<AShardProgress>` (was
`Option<u32>`).
- `EpochEndAccumulate` carries `total_shards`. Its apply validates the
delta's `total_shards` matches any previously persisted total and
surfaces an error if they diverge (would only happen if a work unit
was constructed with a stale config view).
- `EpochWrapUp` and `EpochTransition` undo fields adapted to the new
type.
- `AShardWorkUnit::load` / `commit_state` read the persisted total when
present and fall back to `config.account_shards()` for fresh
boundaries.
- `CardanoLogic` caches `effective_account_shards` (= persisted total
if a boundary is in flight, else config). Refreshed at every
`pop_work` call so `receive_block` (which has no state access) can
use the up-to-date value when constructing
`WorkBuffer::AShardingBoundary`.
- Crash-recovery wording updated to surface a clear warning when the
persisted total disagrees with current config.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): make EpochEndAccumulate::undo no-op on skipped apply
apply has three early-return guards (idempotent repeat, out-of-order,
total_shards mismatch) that leave state untouched. undo unconditionally
subtracted the deltas and overwrote ashard_progress, so a rollback
following a skipped apply would underflow the u64 end.* fields and
clobber the cursor.
Capture prev_ashard_progress and set an applied flag during apply only
when state is actually mutated; undo early-returns when !applied and
restores from the snapshot. Same pattern as EpochWrapUp/EpochStatsUpdate.
Also broaden any_epoch_state to vary ashard_progress so the existing
roundtrip proptests for EpochWrapUp and EpochTransition exercise the
Some(_) → None → Some(_) path their apply/undo introduced earlier in
this branch (previously only None → None was covered). Add a dedicated
epoch_end_accumulate_roundtrip proptest covering all four progress
shapes and all three skip branches.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(cardano): validate account_shards at startup
The divides-256 invariant on account_shards is enforced only via
debug_assert! in shard_key_range(), which is stripped in release
builds. An invalid TOML value (0, 3, 7, 100, ...) would deserialize
cleanly and silently corrupt key-range coverage.
Call validate_total_shards() at the top of CardanoLogic::initialize
and surface failures as ChainError::InvalidConfig so misconfiguration
fails the startup with a clear message.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* docs(skills): split AShard from Ewrap in debug-epoch-mismatch guide
After this branch's restructure, per-account reward application,
unspendable routing, and EWRAP-time registration filtering live in
AShard; only MIRs, refunds, and boundary close remain in Ewrap.
Update the Classify, Work Units, Source Files, and Instrumentation
tables so the bisection workflow points at the right work unit
(and module path) for each failure shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(core): emit RSS deltas around each work-unit phase
Wraps every phase (load/compute/commit_*) of `execute_work_unit` in
sync and import with an `RssProbe` that emits an `info!` event with
`phase`, `rss_before_mb`, `rss_after_mb`, `rss_delta_mb`. Attaches via
the surrounding `#[instrument(name = "work_unit")]` span so the work
unit's name is included automatically. Helps localize boundary memory
spikes (RUPD/AShard/Ewrap/ESTART) without per-crate boilerplate.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): shard accounts on entropic byte of credential key
Account/PendingReward keys are 32-byte CBOR encodings of `StakeCredential`
whose first 4 bytes (`0x82 <variant> 0x58 0x1c`) carry no entropy across
credentials. Sharding by `key[0]` therefore funnelled every credential
into a single bucket regardless of `account_shards`, so only one shard
ever did real work.
`shard_key_range` becomes `shard_key_ranges` and returns one range per
`StakeCredential` variant, sliced on `key[4]` (first byte of the actual
hash). Each AShard now scans two contiguous ranges that together cover
~1/N of the credential keyspace, giving even per-shard work without any
data migration. The CBOR layout invariant is asserted in a unit test.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(cardano): shard ESTART account-snapshot transitions
Hoist the credential-keyed shard helpers from `ashard/shard.rs` to a
top-level `crate::shard` module and apply the same per-shard pattern
to ESTART so per-account snapshot rotations stream through bounded-
memory shards instead of accumulating millions of `AccountTransition`
deltas in one Vec.
New work unit `EStartShardWorkUnit` lives in a sibling `estart_shard/`
module that adds shard-aware load/commit methods to `WorkContext`,
mirroring the `ashard/` ↔ `ewrap/` relationship. The existing
`EstartWorkUnit` is repurposed as the finalize half: pool / drep /
proposal transitions, the closing `EpochTransition` (epoch advance +
new pots + era migration), archive logs, and the cursor advance
(which only ever moves here — never per shard).
Boundary pipeline is now:
Blocks → AShard×N → Ewrap → EStartShard×N → Estart(finalize) → Blocks
Same `CardanoConfig::account_shards` drives both halves. Progress is
tracked separately on `EpochState.estart_shard_progress` (parallel to
`ashard_progress`); only one of the two is ever populated at once.
`EStartShardAccumulate` mirrors `EpochEndAccumulate`'s idempotency,
ordering, and total-mismatch guards. `EpochTransition` snapshots and
clears both progress fields. The crash-recovery warning at startup
covers both halves; full mid-EStart-shard resume remains the same TODO
posture as the existing AShard pipeline (`AccountTransition` is not
natively idempotent on re-apply).
`AShardProgress` is renamed to `ShardProgress` since it now serves
two phases.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* docs(cardano): document EStartShard split in work_units.md
Update sequence diagram to show the open-half pipeline
(EStartShard ×N → Estart) alongside the close-half (AShard ×N → Ewrap),
add a new section for `EStartShardWorkUnit`, and trim Estart's section
to its finalize-only delta set (drops `AccountTransition`, calls out
that `EpochTransition` now clears both `ashard_progress` and
`estart_shard_progress`).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): unify epoch boundary into Ewrap and Estart work units
Reshape the epoch-boundary pipeline so each half is one self-contained
work unit instead of four:
- `WorkUnit` trait gains `total_shards()`, `initialize()`, `finalize()`,
and a `shard_index: u32` parameter on the per-phase methods. The core
executor (`crates/core/src/sync.rs::run_lifecycle`) loops the
load/compute/commit cycle once per shard between `initialize` and
`finalize`. Default `total_shards = 1` keeps non-sharded work units
untouched.
- `EwrapWorkUnit` (close half, formerly `AShardWorkUnit`) and
`EstartWorkUnit` (open half, formerly `EStartShardWorkUnit`) absorb
their respective global passes via `finalize()`. The pre-existing
`Ewrap`/`Estart` work units are removed; `CardanoWorkUnit` shrinks
from 7 variants to 5 and `WorkBuffer` from 12 states to 10. Stop-epoch
logic moves into `EstartBoundary`'s transition (cursor still advances
only there).
- All code that runs as part of a single work unit lives under one
module: `ashard/` + `ewrap/` collapsed into `crates/cardano/src/ewrap/`,
`estart_shard/` + `estart/` collapsed into `crates/cardano/src/estart/`.
AVVM reclamation hoisted out of the per-shard `load` into `initialize`.
- Persisted state field names (`ashard_progress`, `estart_shard_progress`)
and the Serde-tagged `EStartShardAccumulate` delta are preserved to
avoid an on-disk migration.
The `WorkBuffer` no longer enumerates shards; `CardanoLogic` drops the
`effective_account_shards` cache. Crash recovery still relies on the
existing `committed` guards in `EpochEndAccumulate` /
`EStartShardAccumulate` — same correctness posture as before.
Test harness gains a post-finalize callback (fires once with
`shard_index == total_shards`) so per-boundary introspection (e.g.
`epoch_pots`) sees the global teardown state.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): align progress delta and field names with new boundary nomenclature
Rename the per-shard progress deltas and the persisted EpochState
progress fields to match the post-merge work-unit identities (Ewrap /
Estart):
- `EpochEndAccumulate` → `EWrapProgress` (carries shard reward
accumulators + cursor)
- `EStartShardAccumulate` → `EStartProgress` (cursor only — per-account
Estart work lands directly on AccountState)
- `EpochState.ashard_progress` → `EpochState.ewrap_progress`
- `EpochState.estart_shard_progress` → `EpochState.estart_progress`
- `EWrapProgress.prev_ashard_progress` → `prev_ewrap_progress`
- `EStartProgress.prev_estart_shard_progress` → `prev_estart_progress`
CBOR compatibility preserved: the minicbor `#[n(15)]` / `#[n(16)]`
positional indices on the EpochState fields are unchanged, so existing
on-disk state deserializes unmodified. Field-name changes only affect
serde-tagged paths (ad-hoc JSON dumps, debug prints), not the durable
state.
The delta type rename does change the `CardanoDelta` enum's serde
encoding (variant tag is the type name), so any node mid-sync with a
populated WAL will need to wipe / re-bootstrap. Fresh nodes are
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* feat(cardano): shard RUPD reward computation by credential key
RUPD was the boundary path's worst memory spike: a single load built
the full per-account `accounts_by_pool` + `registered_accounts` plus
an O(N) `RewardMap`. Mirror the Estart/Ewrap pattern — hoist
pool-bounded globals into `initialize()` (pots, incentives, pparams,
pool snapshots, pool_stake totals) and shard the per-credential leg
across the same `account_shards` partitions: each shard streams
`AccountState` over its two key ranges only, builds a shard-scoped
delegator + registered set, runs `define_rewards` over every pool
but emits only in-range credentials, and writes the in-range
`PendingRewardState` entities.
Leader-reward emission gates on a new default-true
`RewardsContext::should_include`, so the shard whose range contains
the operator credential is the sole emitter for that pool's leader
reward. Delegator emissions are filtered naturally via
`pool_delegators` returning only in-range creds, with a defensive
`should_include` check at the merge site.
Per-shard progress is tracked by a new `RupdProgress` delta on
`EpochState.rupd_progress`, with the same idempotency + ordering +
total-mismatch guards as `EWrapProgress` / `EStartProgress`.
`EpochTransition` rollback now also captures and clears
`rupd_progress`. `EpochState.incentives` and per-pool `StakeLog`
archive entries move to `finalize()` (one-shot, after every shard
has committed) so concurrent shard commits can't race; the per-pool
reward + delegator-count contributions accumulate on the work unit
as O(pools) state.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(core): drop RSS probe instrumentation around work-unit phases
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): replace account_shards config with ACCOUNT_SHARDS constant
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* refactor(cardano): version EpochWrapUp/EpochTransition for WAL compat
Bincode encodes CardanoDelta variants by positional index and structs by
positional fields, so this branch's three new variants (EWrapProgress,
EStartProgress, RupdProgress) inserted mid-enum and the appended `prev_*_progress`
fields on EpochWrapUp / EpochTransition would have made pre-upgrade WAL
rows undecodable.
Freezes CardanoDelta indices 0..=38 to match pre-PR `main`, restores the
legacy struct shapes verbatim under `#[deprecated]`, and introduces
EpochWrapUpV2 / EpochTransitionV2 carrying the new undo state. The three
sharded-progress variants plus the V2 variants are appended at the end
of the enum. New commit paths in estart/reset.rs and ewrap/wrapup.rs
emit V2; the legacy types remain solely for replay of older WAL rows
and carry TODO(wal-compat) cleanup notes.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): resume sharded work units at committed shard cursor
Previously, `EwrapWorkUnit` / `EstartWorkUnit` / `RupdWorkUnit::initialize`
read only `*_progress.total` from persisted state and the core lifecycle
loop iterated `0..total_shards` unconditionally, so a crash mid-boundary
replayed shards `0..k-1` on restart. That is unsafe: `AccountTransition`
deltas (Estart) are non-idempotent — replaying a committed shard would
double-rotate every account in it.
Adds `WorkUnit::start_shard()` (defaulting to `0`); `run_lifecycle` now
loops `start_shard..total_shards`. The three sharded work units cache
the committed cursor in `initialize` from `*_progress.committed`, and
their `start_shard()` returns it. `CardanoWorkUnit` delegates the new
method to its variants. The bootstrap test harness mirrors the real
runner.
Also propagates `load_epoch` failures via `?` instead of the previous
`Err(_) => ACCOUNT_SHARDS` swallow, so a state-read failure can no
longer silently repartition an in-flight boundary.
Addresses PR #978 review comments 3153861491 (restart cursor) and
3154574387 (propagate load_epoch failures).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): refresh ending_state before archive write in ewrap finalize
`commit_finalize` was archiving `self.ending_state()` at the epoch-start
temporal key, but `stream_and_apply_namespace::<D, EpochState>` only
applied the boundary-closing deltas (PParamsUpdate, TreasuryWithdrawal,
EpochWrapUp) to the writer — `ending_state` itself was never refreshed.
The archived row therefore carried the pre-commit snapshot (stale
rolling/pparams, populated `ewrap_progress`).
Adds an EpochState-specific variant of the streaming helper that
returns the post-apply singleton; `commit_finalize` swaps the result
into `self.ending_state` before the archive write, so the archived
EpochState matches what's about to be committed to the live state
store.
Addresses PR #978 review comment 3153861497.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): assert shard invariants in release builds
`shard_key_ranges` previously guarded `total_shards` and `shard_index`
with `debug_assert!`, which compiles to nothing in release. A `0` would
divide by zero in `variant_range`, and a non-divisor of 256 would
silently produce broken partitions. Since `total_shards` can come from
persisted `ShardProgress.total` (not just the compile-time
`ACCOUNT_SHARDS` constant), the invariants must hold at runtime in all
profiles.
Promotes the validation to unconditional `assert!` / `panic!` with
informative messages.
Addresses PR #978 review comment 3153861503.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* chore(cardano): housekeeping fixes from PR review
- genesis/mod.rs: fix copy/paste typo "Ewrap (which now runs before
Ewrap)" → "per-shard Ewrap pass (which runs before the global Ewrap
finalize)" (PR #978 comment 3154574432).
- work_units.md: add `text` language to fenced sequence diagram block
for markdownlint MD040 (3153861508); fix stale loader name
`BoundaryWork::load_ewrap` → `load_finalize` (3153861515).
- tests/memory.rs: sample heap allocation across full iteration as well
as iterator construction, so a backend that buffers the shard on first
`next()` no longer slips through (3153861530).
- model/epochs.rs: hoist `#[allow(deprecated)]` from per-item attributes
on the prop_compose!/test items (where the macro hides the lint site)
to the whole `prop_tests` module — silences the test-build deprecation
warnings introduced by the V1/V2 split.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* chore(cardano): clear clippy warnings
- shard.rs: const-context divisibility check now uses
`u32::is_multiple_of` (stable-const since Rust 1.87) per
`clippy::manual_is_multiple_of`.
- model/epochs.rs: replace `let mut x = T::default(); x.f = ...; x` with
a struct-literal `..Default::default()` form per
`clippy::field_reassign_with_default`; also demote a doc comment
attached to a `prop_compose!` invocation to a regular `//` comment
(rustdoc can't attach docs to macro invocations).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* docs(skills): drop stale AShard/EpochEndAccumulate references
The `crates/cardano/src/ashard/` module was merged into
`crates/cardano/src/ewrap/` (it became EWRAP's per-shard leg) and the
`EpochEndAccumulate` delta was renamed to `EWrapProgress`. The debug
guide's tables, narrative, file-path references, and instrumentation
hints still pointed at the old names and paths.
Updates the Step 3 classification table, Step 5 work-unit narrative +
source-file map, and Step 6 instrumentation table to:
- talk about EWRAP/ESTART each having a per-shard leg + finalize, not
a separate ASHARD work unit;
- point at `ewrap/rewards.rs`, `crates/cardano/src/shard.rs`, and the
per-unit `work_unit.rs` files;
- reference `EWrapProgress` (not `EpochEndAccumulate`) and the
`EpochWrapUpV2` / `EpochTransitionV2` deltas where boundary close /
open is described.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cardano): precompute per-pool live pledge for sharded rupd
Compute pool live pledge globally in load_globals so each shard sees
the full pledge sum, instead of summing only the owner accounts that
fall in the current shard's key range.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4&s=80)
