Commit by iohk-bors[bot]

Merge #633

633: byron-proxy: fix chain sync server memory leak r=avieth a=avieth

The problem: ImmutableDB gives an imperative iterator interface which does not resemble a pure functional stream. It also requires explicit closing. The byron-proxy iterator attempts to give back some pure functional-ness by giving each iterator result a “next” iterator. Under the hood, this just calls the imperative iterator’s next result again. That’s all fine except that the pure functional style iterator also needs to close the underlying one. The IteratorResource bundles the pure functional iterator and the releaser together. When bracketing, the whole thing is retained until the bracket release goes, resulting in a memory leak.

Solution implemented here: force the releaser before running the bracket continuation (adjusted for ResourceT “bracketing”). It seems like a sensible thing to do for bracketing in general, but the standard bracket idiom does not do this.

Alternate solutions?

  • Use a weak reference for the pure functional iterator.
  • Abandon trying to give a pure functional iterator.
  • Have the byron-proxy DB expose the imperative iterator, and also a way to derive a pure functional iterator from it, if desired.

Option 1 seems even worse than this, and options 2 and 3 aren’t immediately obvious, since the DB wrapper interface has a legitimate case in which it must construct an empty iterator (lookup by hash that is not in the index) but the ImmutableDB iterator isn’t suitable, because it has an IteratorId field.

Co-authored-by: Alexander Vieth [email protected]

View on GitHub