Home / Input Output / cardano-db-sync
Sep 16, 3-4 PM (0)
Sep 16, 4-5 PM (0)
Sep 16, 5-6 PM (0)
Sep 16, 6-7 PM (0)
Sep 16, 7-8 PM (0)
Sep 16, 8-9 PM (0)
Sep 16, 9-10 PM (0)
Sep 16, 10-11 PM (0)
Sep 16, 11-12 AM (0)
Sep 17, 12-1 AM (0)
Sep 17, 1-2 AM (0)
Sep 17, 2-3 AM (0)
Sep 17, 3-4 AM (0)
Sep 17, 4-5 AM (0)
Sep 17, 5-6 AM (0)
Sep 17, 6-7 AM (0)
Sep 17, 7-8 AM (0)
Sep 17, 8-9 AM (0)
Sep 17, 9-10 AM (0)
Sep 17, 10-11 AM (0)
Sep 17, 11-12 PM (0)
Sep 17, 12-1 PM (0)
Sep 17, 1-2 PM (0)
Sep 17, 2-3 PM (0)
Sep 17, 3-4 PM (0)
Sep 17, 4-5 PM (0)
Sep 17, 5-6 PM (0)
Sep 17, 6-7 PM (0)
Sep 17, 7-8 PM (0)
Sep 17, 8-9 PM (0)
Sep 17, 9-10 PM (0)
Sep 17, 10-11 PM (0)
Sep 17, 11-12 AM (0)
Sep 18, 12-1 AM (0)
Sep 18, 1-2 AM (0)
Sep 18, 2-3 AM (0)
Sep 18, 3-4 AM (0)
Sep 18, 4-5 AM (0)
Sep 18, 5-6 AM (0)
Sep 18, 6-7 AM (0)
Sep 18, 7-8 AM (0)
Sep 18, 8-9 AM (0)
Sep 18, 9-10 AM (0)
Sep 18, 10-11 AM (0)
Sep 18, 11-12 PM (0)
Sep 18, 12-1 PM (0)
Sep 18, 1-2 PM (0)
Sep 18, 2-3 PM (0)
Sep 18, 3-4 PM (0)
Sep 18, 4-5 PM (0)
Sep 18, 5-6 PM (0)
Sep 18, 6-7 PM (0)
Sep 18, 7-8 PM (0)
Sep 18, 8-9 PM (0)
Sep 18, 9-10 PM (0)
Sep 18, 10-11 PM (0)
Sep 18, 11-12 AM (0)
Sep 19, 12-1 AM (0)
Sep 19, 1-2 AM (0)
Sep 19, 2-3 AM (0)
Sep 19, 3-4 AM (0)
Sep 19, 4-5 AM (0)
Sep 19, 5-6 AM (0)
Sep 19, 6-7 AM (0)
Sep 19, 7-8 AM (0)
Sep 19, 8-9 AM (0)
Sep 19, 9-10 AM (0)
Sep 19, 10-11 AM (0)
Sep 19, 11-12 PM (0)
Sep 19, 12-1 PM (0)
Sep 19, 1-2 PM (0)
Sep 19, 2-3 PM (0)
Sep 19, 3-4 PM (0)
Sep 19, 4-5 PM (0)
Sep 19, 5-6 PM (0)
Sep 19, 6-7 PM (0)
Sep 19, 7-8 PM (0)
Sep 19, 8-9 PM (0)
Sep 19, 9-10 PM (0)
Sep 19, 10-11 PM (0)
Sep 19, 11-12 AM (0)
Sep 20, 12-1 AM (0)
Sep 20, 1-2 AM (0)
Sep 20, 2-3 AM (0)
Sep 20, 3-4 AM (0)
Sep 20, 4-5 AM (0)
Sep 20, 5-6 AM (0)
Sep 20, 6-7 AM (0)
Sep 20, 7-8 AM (0)
Sep 20, 8-9 AM (0)
Sep 20, 9-10 AM (0)
Sep 20, 10-11 AM (0)
Sep 20, 11-12 PM (0)
Sep 20, 12-1 PM (0)
Sep 20, 1-2 PM (0)
Sep 20, 2-3 PM (0)
Sep 20, 3-4 PM (0)
Sep 20, 4-5 PM (0)
Sep 20, 5-6 PM (0)
Sep 20, 6-7 PM (0)
Sep 20, 7-8 PM (0)
Sep 20, 8-9 PM (0)
Sep 20, 9-10 PM (0)
Sep 20, 10-11 PM (0)
Sep 20, 11-12 AM (0)
Sep 21, 12-1 AM (0)
Sep 21, 1-2 AM (0)
Sep 21, 2-3 AM (0)
Sep 21, 3-4 AM (0)
Sep 21, 4-5 AM (0)
Sep 21, 5-6 AM (0)
Sep 21, 6-7 AM (0)
Sep 21, 7-8 AM (0)
Sep 21, 8-9 AM (0)
Sep 21, 9-10 AM (0)
Sep 21, 10-11 AM (0)
Sep 21, 11-12 PM (0)
Sep 21, 12-1 PM (0)
Sep 21, 1-2 PM (0)
Sep 21, 2-3 PM (0)
Sep 21, 3-4 PM (0)
Sep 21, 4-5 PM (0)
Sep 21, 5-6 PM (0)
Sep 21, 6-7 PM (0)
Sep 21, 7-8 PM (0)
Sep 21, 8-9 PM (0)
Sep 21, 9-10 PM (3)
Sep 21, 10-11 PM (2)
Sep 21, 11-12 AM (0)
Sep 22, 12-1 AM (4)
Sep 22, 1-2 AM (3)
Sep 22, 2-3 AM (0)
Sep 22, 3-4 AM (0)
Sep 22, 4-5 AM (0)
Sep 22, 5-6 AM (0)
Sep 22, 6-7 AM (0)
Sep 22, 7-8 AM (0)
Sep 22, 8-9 AM (0)
Sep 22, 9-10 AM (0)
Sep 22, 10-11 AM (0)
Sep 22, 11-12 PM (0)
Sep 22, 12-1 PM (0)
Sep 22, 1-2 PM (0)
Sep 22, 2-3 PM (0)
Sep 22, 3-4 PM (0)
Sep 22, 4-5 PM (0)
Sep 22, 5-6 PM (0)
Sep 22, 6-7 PM (0)
Sep 22, 7-8 PM (0)
Sep 22, 8-9 PM (0)
Sep 22, 9-10 PM (0)
Sep 22, 10-11 PM (0)
Sep 22, 11-12 AM (0)
Sep 23, 12-1 AM (0)
Sep 23, 1-2 AM (0)
Sep 23, 2-3 AM (1)
Sep 23, 3-4 AM (0)
Sep 23, 4-5 AM (1)
Sep 23, 5-6 AM (0)
Sep 23, 6-7 AM (0)
Sep 23, 7-8 AM (0)
Sep 23, 8-9 AM (0)
Sep 23, 9-10 AM (0)
Sep 23, 10-11 AM (0)
Sep 23, 11-12 PM (0)
Sep 23, 12-1 PM (0)
Sep 23, 1-2 PM (0)
Sep 23, 2-3 PM (0)
Sep 23, 3-4 PM (0)
14 commits this week Sep 16, 2020 - Sep 23, 2020
Update dependencies to be the same as cardano-node
This was incredibly painful due to:

* Metadata to JSON changes in the 'node' repo which both drasitcally changed
  and removed functionality that 'db-sync' was using.
* The added 'era' parameterisation in the 'ledger-specs' code for the protocol
  param data type which resulted a cascade of type errors which did not obviously
  help figuring out the solution.
db-sync: Finalize naive handling of ledger state
Ledger state is not persisted to disk. This means the db-sync process
can be killed and when restarted, restore the ledger state from disk.
However, when syncing the chain, not every ledger state is persisted
so the database may need to be rolled back to a slot number where a
ledger state esists.

While syncing the chain, only every 10000th ledger state is persisted
but when following the chain, every ledger state is saved. In either
case, only the last 8 ledger states are stored to disk. That means
that if fully synced, rollbacks of more than 8 block are no longer
possible.
db-sync: Finalize naive handling of ledger state
Ledger state is not persisted to disk. This means the db-sync process
can be killed and when restarted, restore the ledger state from disk.
However, when syncing the chain, not every ledger state is persisted
so the database may need to be rolled back to a slot number where a
ledger state esists.

While syncing the chain, only every 10000th ledger state is persisted
but when following the chain, every ledger state is saved. In either
case, only the last 8 ledger states are stored to disk. That means
that if fully synced, rollbacks of more than 8 block are no longer
possible.
db-sync: Config refactoring to allow generation of LedgerState config
In order to correctly build and maintain ledger state 'db-sync'
needs to construct a 'LedgerState' configuration. This was incredibly
difficult because the ledger state config is buried within a
'TopLevelConfig' and building that is difficult because it requires
a bunch of config parameters that 'db-sync' does not otherwise care
about.
db-sync: Reject TxMetadata objects containing NUL characters
TxMetadata is stored as JSON and that JSON is stored in a 'jsonb' column
in PostgreSQL. However, there are limitations to that Postgres 'jsonb'
data type. Specifically, it cannot contain Uniciode NUL characters. This
temporary fix simply drops TxMetadata JSON objects that would otherwise
be rejected by Postgres. Hopefully a better solution will be will be
dreamt up and implemented later.

Temporary workaround fix for:
https://github.com/input-output-hk/cardano-db-sync/issues/297
db-sync: Reject TxMetadata objects containing NUL characters
TxMetadata is stored as JSON and that JSON is stored in a 'jsonb' column
in PostgreSQL. However, there are limitations to that Postgres 'jsonb'
data type. Specifically, it cannot contain Uniciode NUL characters. This
temporary fix simply drops TxMetadata JSON objects that would otherwise
be rejected by Postgres. Hopefully a better solution will be will be
dreamt up and implemented later.

Temporary workaround fix for:
https://github.com/input-output-hk/cardano-db-sync/issues/297