Cardano Updates

Code in `cardano-base` depends on the generalized type sig.

Reshape the storage-backend layer:

  * The `ouroboros-consensus:lsm` sublibrary used to expose
    `Storage.LedgerDB.V2.LSM` — an 853-line file that had been
    fully commented out since the V2 rework began. That file is
    deleted; the sublibrary is repurposed to expose just
    `Ouroboros.Consensus.Backends.LSM`, with a real cursor-based
    implementation (`drainTableFiltered` uses `LSM.withCursor` +
    paginated `LSM.take` with a 100000-entry batch, matching the
    older `implReadAll`).
  * The new LSM module lives at `lsm/Ouroboros/Consensus/Backends/LSM.hs`
    (root-level, no longer nested inside `ouroboros-consensus/src/`).
    The sublib gains the `cardano-ledger-core`, `cardano-ledger-shelley`,
    `microlens`, `ouroboros-consensus:cardano` deps it needs to build
    against the Shelley UTxO type; drops `contra-tracer`, `filepath`,
    `nothunks`, `random`, `serialise`, `streaming` (no longer needed).
  * `Ouroboros.Consensus.Backends` (in the cardano sublib) becomes
    a small umbrella exposing `inMemoryBackendArgs` plus the shared
    `loadSnapshot` and `mkSnapshotManager` helpers — both helpers
    now take a `SnapshotBackend` parameter so the LSM and in-memory
    paths can each stamp their own tag and the per-era Byron
    placeholder picks up the supplied tag correctly.
  * `Ouroboros.Consensus.Backends.InMemory` lives in the cardano
    sublib because it depends on the Shelley UTxO. The
    writer/reader path-mismatch (writer wrote to `<snapshotDir>/utxo`
    inline, reader read from `<snapshotDir>/tables`) is unified on
    `<snapshotDir>/utxo` via `snapshotToUTxOFilePath`.
  * Targets that don't want lsm-tree (e.g. wasm cross-compilation)
    can simply not depend on `ouroboros-consensus:lsm`.

Cabal:

  * Main library loses the deleted `Ledger.Tables.*`,
    `Util.IndexedMemPack`, `Storage.LedgerDB.V2.Forker`,
    `Storage.LedgerDB.V2.InMemory` exposed modules and the now-unused
    `FailT` and `mempack` deps.
  * `cardano` sublib gains `Backends`, `Backends.InMemory` modules
    and `fs-api`; loses `Cardano.Ledger`, `Cardano.QueryHF`, and the
    `singletons` dep.
  * `lsm` sublib changes `hs-source-dirs` from
    `ouroboros-consensus/src/ouroboros-consensus-lsm` to `lsm`,
    exposes `Backends.LSM`, and depends on
    `ouroboros-consensus:cardano`.

`Ledger.Dual` carries the dual-ledger glue used by the test suite (it
runs two ledger implementations side-by-side and checks they agree).
Adapt it to the handle-based foundations: declare the
`BlockSupportsLedgerHD` instance with paired `StateHandle` /
`TickedStateHandle` records that hold a handle from each side, route
ticking and block application through both, and drop the `mk`
parameter throughout.

  * `Cardano.CanHardFork` is rewritten on top of the new
    `HFLedgerTablesFactory` plumbing and the era-translation API that
    no longer routes through `WrapTxIn`/`WrapTxOut`.
  * `Cardano.Ledger` and `Cardano.QueryHF` are deleted entirely: their
    role was to wire `mk`-aware `LedgerState (CardanoBlock c) mk` and
    its HFC-projected queries through the era-translation
    machinery. With handles owning the on-disk tables and HFC queries
    living in `HardFork.Combinator.Ledger.Query`, neither file has
    anything to do.
  * `Cardano.Block` and `Cardano.Node` lose `mk` from their signatures
    and threading.

Adapt the Shelley ledger (excluding mempool — that went out in commit
2) to the handle-based foundations:

  * The `BlockSupportsLedgerHD` instance for `ShelleyBlock proto era`
    declares its `StateHandle` / `TickedStateHandle` records. The
    on-disk part of the state (the UTxO) is owned by a
    `LedgerTablesHandle` and read on demand; the in-memory part is
    the ticked Shelley state minus the UTxO.
  * `Ledger`, `Forge`, `Inspect`, `Query`, `SupportsProtocol`,
    `Node.Serialisation`, `Node.TPraos`, `ShelleyHFC` lose the `mk`
    parameter and thread handles instead.
  * Missing `deriving instance ShelleyBasedEra era => NoThunks (Ticked
    LedgerState (ShelleyBlock proto era))` is added next to the
    `TickedShelleyLedgerState` data instance, matching the existing
    `NoThunks` deriving on the un-ticked `LedgerState`. Without it
    the `MempoolAcc (ShelleyBlock proto era)` `NoThunks` deriving
    cannot be discharged.

Tests are intentionally left at the old shape and will be revisited as
a separate piece of work.

Adapt the Byron ledger (excluding mempool — that went out in commit 2)
to the handle-based foundations:

  * The `BlockSupportsLedgerHD` instance for `ByronBlock` declares its
    own `StateHandle` and `TickedStateHandle` records. Byron does not
    maintain on-disk tables, so `LedgerTablesFactory m ByronBlock =
    ()` and the `LedgerTablesHandle` is a trivial no-op record.
  * `Ledger`, `Forge`, `Inspect`, `Node`, `Node.Serialisation` lose
    the `mk` parameter from `LedgerState` signatures.
  * `ByronHFC` adapts to the HFC's new `HardForkStateHandle` /
    `HFLedgerTablesFactory` plumbing.

Tests are intentionally left at the old shape and will be revisited as
a separate piece of work.

Mechanical adaptation to the new handle-based APIs:

  * `EmptyMK` strip on signatures throughout `Node.hs`,
    `NodeKernel.hs`, `GSM.hs`, `NodeToClient.hs`.
  * `ProtocolInfo b` -> `ProtocolInfo m b` (matching the change in
    `Node.ProtocolInfo`); `lgrGenesis` now requires a
    `LedgerTablesFactory`.
  * `BlockSupportsLedgerHD` / `NoThunks` constraint propagation where
    handles need to be opened.
  * `openChainDB` and its callers thread the `LedgerTablesFactory`
    instead of the obsolete `TransCtx`.
  * The mempool-snapshot callsite in `NodeKernel.hs` no longer passes
    `roforkerReadTables` — it matches the new mempool signature that
    reads tables via the duplicate it now owns.
  * `withReadOnlyForkerAtPoint` callers move to
    `withReadOnlyHandleAtPoint` (the underlying handle is bracketed by
    the LedgerDB).
  * `MempoolTxAdded` simplification consequent to the per-tx workflow
    in the mempool commit.

Mechanical EmptyMK/ValuesMK strip plus replacing `ReadOnlyForker' m
blk` with `Handle ExtLedgerState m blk` in the mini-protocol servers
and clients.

`Node.ProtocolInfo`'s `ProtocolInfo b` becomes `ProtocolInfo m b` and
`pInfoInitLedger` becomes a `LedgerTablesFactory m b -> m
(ExtStateHandle m b)` continuation — matching the new shape of
`lgrGenesis`, so that genesis construction uses the same
`LedgerTablesFactory` the backend will be providing.

`Node.Run` drops the now-unused `CanUpgradeLedgerTables` superclass on
`RunNode` and `BlockSupportsLedgerHD` propagates as a constraint where
the mini-protocols need to dereference handles.

Mechanical: `EmptyMK` strip across `Impl.hs`, `ChainSel.hs`,
`Query.hs`, `Background.hs`, `Init.hs`, `Impl/Args.hs`; the
`ReadOnlyForker' m blk` parameter on tip queries becomes
`Handle ExtLedgerState m blk`.

Structural:

  * New `getCurrentLedgerRef` field on the `ChainDB` record returns a
    `Handle` to the current ledger state. Existing
    `getCurrentLedgerState` is preserved for tip-only consumers.
  * `LedgerInterface` is split into two narrower fields, fixing a
    mempool/ChainDB race:
      - `getCurrentLedgerTip :: STM m (Point blk)` — returning only a
        Point makes it impossible for the caller to accidentally
        close or dereference the underlying handle in STM.
      - `withCurrentLedgerStateDup :: (StateHandle m blk -> m a) ->
        m a` — bracketed: opens a fresh duplicate via
        `openReadOnlyHandleAtPoint VolatileTip` under the LedgerDB
        read lock and closes it on exit. Used by `initMempoolEnv` and
        `implSyncWithLedger`.
  * `VolatileDB.Impl.State` loses an `EmptyMK`-shaped parameter that
    is no longer meaningful.

Replace the "everything through Forker" model with a Handle-first API,
folding what used to be `V2.Forker` and `V2.InMemory` back into
`V2.LedgerSeq`, `V2.Backend`, and the top-level `Forker` module:

  * `openHandleAtTarget` is the new core method on the `LedgerDB`
    record: it allocates a fresh `Handle ExtLedgerState m blk` at a
    requested target without going through a `Forker` (no TVars, no
    tracer, no lock ref) and without firing a stray `ForkerOpen`
    trace.
  * `openReadOnlyHandle` is a thin wrapper over `openHandleAtTarget`
    and replaces `openReadOnlyForker`. Read-only consumers
    (db-analyser, tests) acquire handles directly.
  * `getVolatileTip` / `getVolatileTipRef` and `getCurrentLedger` /
    `getCurrentLedgerRef` are differentiated in haddock: the `Ref`
    variants return a handle and bind its lifetime to the LedgerDB
    (caller must not close).
  * The `Forker` abstraction is now used only by chain selection. The
    `validate.rewrap` and `withTipForker` `error "Unreachable"` calls
    are replaced by labelled precondition-violation messages.
  * `forkerPush`'s lost-pruning TODO is resolved (and turned into a
    haddock): pruning of the main `LedgerSeq` is the responsibility of
    `implGarbageCollect`; the forker's local seq is per-forker and
    released by `forkerCommit` or `forkerClose`.

The standalone `V2/Forker.hs` and `V2/InMemory.hs` modules are
deleted; their content lives inline in `V2/Backend.hs` and
`V2/LedgerSeq.hs` (whose APIs are now shaped around handles, not
diffs). Miscellaneous: stale `openStateHandleFromSnapshot` reference
in `Snapshots` haddock now points at `brLoadSnapshot`, stray empty
`where` clause at the bottom of `applyBlock` in `Forker` removed,
`InitDB.currentTip` haddock grammar fixed.

Adapt the Hard Fork Combinator to the handle-based foundations from
the first commit (mempool changes already went out in the previous
commit).

  * `HardForkBlock` has no `BlockSupportsLedgerHD` instance: its
    on-disk tables are managed era-by-era. Instead, the per-era
    `StateHandle`s are carried inside `HardForkStateHandle` (declared
    in `HardFork.Combinator.Basics`), and the bridging equation
    `LedgerTablesFactory m (HardForkBlock xs) = HFLedgerTablesFactory
    m xs` lives in that instance.
  * `HFLedgerTablesFactory` is the per-era recipe used by
    era-translation functions to materialise the destination era's
    tables. It is declared on `CanHardFork`.
  * `tickedHardForkStateHandle…` fields replace `Ticked
    LedgerState … DiffMK`-style records; the in-flight handle is owned
    end-to-end through ticking, era translation, and block
    application.
  * `HeaderStateHistory.fromChain` no longer leaks intermediate
    handles — it closes its allocations and leaves the caller-owned
    `initState` alone.
  * Stale `DiffMK`/`emptyLedgerTables` haddocks in
    `State`, `State.Types`, and the era-translation modules are
    cleaned up to describe the current handle-based reality.

The query / common-protocol-params / peer-selection sub-modules,
`Embed.{Binary,Nary,Unary}`, `Forging`, `Node`, `Node.InitStorage`,
`Serialisation.SerialiseDisk`, and `Degenerate` are all mechanical
adapter updates: `EmptyMK` → no-MK in signatures and the threading of
`Handle`/`ExtStateHandle` through what used to take a bare
`LedgerState`.

Replace the per-state-diff machinery with two block-indexed data
families and a per-tx workflow:

  * `TxLocalData blk` — block-specific data owned by each tx in the
    mempool sequence. For Shelley, this is the UTxO entries the tx's
    inputs reference. For Byron it is `()`. For the HFC it is an
    era-tagged `NS` over per-era `TxLocalData`. Stored alongside the
    validated tx in the sequence; rebuilt only on tip changes.
  * `MempoolAcc blk` — the mempool's aggregated view: the ticked
    ledger state plus whatever cumulative state validation needs (for
    Shelley, the combined Diff of every cached tx so far).

The class methods change shape:

  * `prepareTx :: …TickedStateHandle m blk -> MempoolAcc blk -> GenTx
    blk -> m (TxLocalData blk)` is the sole monadic entry point. It
    reads the disk-resident data the tx will need; subsequent
    operations on this tx are pure.
  * `applyTx` and `reapplyTx` become pure (`Except (ApplyTxErr blk)`)
    and take `MempoolAcc blk` + `TxLocalData blk` rather than a
    `TickedLedgerState blk ValuesMK`.
  * `reapplyTxs` and the `InputTxDiffs` / `WhatToDoWithTxDiffs` /
    `ReapplyTxsResult` infrastructure are removed; callers fold
    `reapplyTx` over the kept txs directly because their local data is
    already on hand.

The Byron, Shelley, and HFC mempool instances declare their
`TxLocalData` / `MempoolAcc` data instances and adapt to the new
methods. The HFC's `applyHelper` no longer needs `undefined` /
`error "Impossible!"` / `unsafeCoerce` because it can match
era-by-era on the new structures (via `matchPolyTx` + `matchTelescope`).
The Shelley `txMeasureConway` now consults the real per-tx UTxO
entries via `TxLocalData`, fixing a latent bug where the cached ticked
state's UTxO appeared empty.

Drop the `mk :: MapKind` parameter from `LedgerState` and introduce the
handle vocabulary that replaces it:

  * `LedgerTablesHandle m blk` — the on-disk side (UTxO) of a ledger
    view, owned and produced by the LedgerDB backend.
  * `StateHandle` / `TickedStateHandle` — opaque per-block records
    declared via the new `BlockSupportsLedgerHD` class, bundling a
    pure `LedgerState blk` (or its ticked variant) with a
    `LedgerTablesHandle`.
  * `ExtStateHandle` / `TickedExtStateHandle` — the `ExtLedgerState`
    counterparts, declared as plain records in `Ledger.Extended` (no
    instance of `BlockSupportsLedgerHD`).
  * `Handle l m blk` — the injective type family mapping a ledger view
    `l` to its canonical handle record (`StateHandle` or
    `ExtStateHandle`).
  * `LedgerTablesFactory m blk` — the per-block context the LedgerDB
    threads into genesis construction and era translation.

Block application and ticking move from
@l blk EmptyMK -> Ticked l blk DiffMK@ to @Handle l m blk -> m (Handle
(Ticked l) m blk)@.

Consequently:

  * The whole `Ouroboros.Consensus.Ledger.Tables.*` hierarchy and
    `Ouroboros.Consensus.Util.IndexedMemPack` are no longer needed and
    are deleted (`Tables.Diff` survives — it's the only piece still
    used).
  * `WrapTxIn` / `WrapTxOut`, `GetTipSTM` / `getTipM` are dropped.
  * `fillJavier` (deprecated placeholder, "Fill this") is added to
    `Util` as a marker for stubs that are still owed before merge.

Mechanical mk-strip in modules where the signature change is the only
change: `Block.Forging`, `Forecast`, `BlockchainTime.WallClock.HardFork`,
`HeaderValidation`, `Genesis.Governor`,
`Ledger.{CommonProtocolParams,Inspect,Query,SupportsPeerSelection,SupportsPeras}`.

Signed-off-by: akrepala <[email protected]>

Signed-off-by: akrepala <[email protected]>