Built in end of stream marker?

[rklaehn]

We have the following annoyance/footgun in quinn, that leaks out to rpc.

When code that uses a quinn::SendStream is dropped, e.g. because of an error or because a future is dropped due to a timeout, finish is called on drop. If - which is pretty likely, this finish is not called in the middle of sending a length delimited message, from the remote side this is indistinguishable from a normal stream termination.

So e.g. if you have a local impl Stream<Item = Bytes>, you can't just translate this to a irpc request with a rx: spsc::Receiver<Bytes>. E.g. you try to send a stream of 10 blobs, remote for some reason only gets 9, you get a different hash.

The proper translation is to create an enum with an explicit Done case, and then on the server side fail unless you get the Done message. But this is quite tedious, you end up translating from local Stream to rpc protocol message T | Done and back on the recv side.

The following would be alternatives:

• have an explicit built in end of stream message, then translate the lack of this message to a RecvError::SenderClosed.
• change our quinn fork to not call finish on drop, but to call reset

Note that this is only relevant for the case where the update messages are not self-contained and there is no natural end of stream message. It is also only relevant for spsc streams.

[rklaehn]

There would be a third alternative, change the Senders to call reset on drop, and require people to explicitly call finish if they don't want this.

[rklaehn]

@Frando I would be curious what you think of this.

[rklaehn]

An issue with this is that the tokio mpsc stream does not distinguish between drop after successful completion and drop due to error. So if we do this we would have to wrap mpsc messages with an enum, and lose the ability to do a from conversion from a tokio::sync::mpsc::Sender/Receiver to the irpc mpsc channels.

This is a problem. If only the rust stream abstraction had the notion of completion built in like https://docs.rs/ufotofu/latest/ufotofu/...

[matheus23]

There's some discussion of this finish-on-drop vs. reset-on-drop behavior of streams in the quinn channel on discord.