Cardano Updates

3409: Use real byte size limits r=coot a=coot

Fixes #1727.


Co-authored-by: Marcin Szamotulski <[email protected]>

* MsgRelease
* MsgDone

* MsgRelease
* MsgDone

NOTE: Missing close tests

NOTE: Missing close tests

Disable assert which triggers by failing hot demotions.

- Fix numberOfConnections call to be number of incommingConnections and not countPrunableConnections
- Changed everything to countIncomingConnections
- Applied pruning on:
  - OutboundStateDuplex -> InboundStateDuplex;
  - OutboundStateDuplex^tau -> InboundStateDuplex;
  - OutboundIdleStateDuplex^tau -> InboundStateDuplex;
  - OutboundStateDuplex -> DuplexState
  transitions
- Tweaked DemoteToColdLocal PruneConnections constructor
- Changed prune connections code:
  - for OutboundStateDuplex{^tau} -> InboundStateDuplex and
    OutboundIdleStateDuplex^tau -> InboundStateDuplex transitions, prune
    code is pretty much the same except we have to check for
    (numberOfConns + 1) -- See Summary
  - for OutboundStateDuplex -> DuplexState transition, prune code has to
    check for (numberOfConns + 1) and also forcefully be contained
    inside its toPrune Set -- See Summary
  - for DuplexState -> InboundStateDuplex transition, prune code has to
    take into account if the connection is contained in its toPrune Set.
    If so we shouldn't update the connection to the new state as we
    might go above hardlimit.
- Fix race between acceptLoop and unregisterOutboundConnection
- Fixed bug on unregisterInboundConnectionImpl where UnsupportedState
  was being returned incorrectly.

Summary:

In order for us to never go above hard limit connections we need to make sure that connections that evolve via OutboundDupSt Ticking → InboundIdleSt Duplex  and via OutboundState Duplex -> DuplexState  to use 'numberOfConns + 1' because we want to know if we actually let those connection to evolve if we need to make room for them by pruning.

For OutboundState Duplex -> DuplexState we also need to include it on its own pruneSet, since this connection might be a Sim-open connection and we are above the hardlimit, we really need to prune this connection in order to preserve the hard limit.

Due to the inherent race condition between accept and runConnectionRateLimits in the acceptLoop, we need to, after successfully running accept, check if accepting that connection put us above the hard limit. If it does then we need to cancel the connection.

- Fixed Simulation/Network/Snocket.hs accept function:
  Accept should be careful when finding a connection in Established or
  HalfOpened state, acting accordingly: When in Established state, fail
  the accept call as in the other side, the connect call fell through;
  when in HalfOpened, do not successfully return nor fail, but rather
  just take a new connection from the TBQueue.

  (This should be catched with more thorough Socket tests, see issue #3289)

- multinode_Sim tests refactor

These identity transitions are harmless, triggered by hot miniprotocol
termination.

Rebase fixs

This simulation is not designed to handle errors.

The resource handler for `bracketOnError` must be uninterruptible.

Be careful when overwriting the state.