feat[tokio-quiche]: implement server-side support for 0-RTT

[toidiu]

This PR adds server support for 0-RTT. This is done by transitioning the Handshake to Application when we have 0-RTT keys. The PR also adds an extension header IsInEarlyData to forward the early data request status so application can detect when a request was received with 0-RTT keys (and take appropriate security precautions).

[toidiu]

This is what helps us transition to Application space so we can then handle the early_data request.

[antoniovicente]

Thinking about your comment about requests after early data having trouble. Is there something that prevents the handshake from continuing and completing after taking a break to handle early data?

[antoniovicente]

Worth adding some logging on handshake completion?

[toidiu]

Thinking about your comment about requests after early data having trouble. Is there something that prevents the handshake from continuing and completing after taking a break to handle early data?

Thats exactly what I need to debug next. My mental model is that it should invoke the work_loop and continue so need to double check that and compare against the no early_data variation.

[evanrittenhouse]

The primary concern I see here is that work_loop's wait phase will not call wait_for_quiche, which wakes the IOW when a TLS callback progresses, in the RunningApplication phase. It depends solely on the AOQ's wait_for_data implementation.`

I'm not super familiar with early data, is it possible that the 0-RTT request/response finishes before the PK callback? If so, we may delay the handshake until the next received packet/Quiche timer fires, rather than when the callback actually completes. From Boring docs it sounds like there's no guarantee about that:

As a server, if early data is accepted, SSL_do_handshake will complete as soon as the ClientHello is processed and server flight sent. SSL_write may be used to send half-RTT data. SSL_read will consume early data and transition to 1-RTT data as appropriate. Prior to the transition, SSL_in_init will report the handshake is still in progress. Callers may use it or SSL_in_early_data to defer or reject requests as needed.

[toidiu]

If I understand correctly: wait_for_quiche is used to progress the TLS Handshake callbacks. In RunningApplication phase, we do not call the wait_for_quiche. The concern is that we might stall the Handshake we transition to RunningApplication in early data (since technically the Handshake is not complete).

We currently use quiche for two callbacks private_key_callback (not called when we resume) and async_select_cert_callback (called when before processing ClientHello). So for our usecase this the Handshake should not stall.

One way to make this clean would be to and another phase Handshake -> ZeroRtt -> RunningApplication. But that doesn't seem necessary at the moment. I'll leave a comment documenting this behavior and something we might need to visit if we decide to use more callbacks.

[antoniovicente]

Thinking about your comment about requests after early data having trouble. Is there something that prevents the handshake from continuing and completing after taking a break to handle early data?

[evanrittenhouse]

The primary concern I see here is that work_loop's wait phase will not call wait_for_quiche, which wakes the IOW when a TLS callback progresses, in the RunningApplication phase. It depends solely on the AOQ's wait_for_data implementation.`

I'm not super familiar with early data, is it possible that the 0-RTT request/response finishes before the PK callback? If so, we may delay the handshake until the next received packet/Quiche timer fires, rather than when the callback actually completes. From Boring docs it sounds like there's no guarantee about that:

As a server, if early data is accepted, SSL_do_handshake will complete as soon as the ClientHello is processed and server flight sent. SSL_write may be used to send half-RTT data. SSL_read will consume early data and transition to 1-RTT data as appropriate. Prior to the transition, SSL_in_init will report the handshake is still in progress. Callers may use it or SSL_in_early_data to defer or reject requests as needed.

[toidiu]

@evanrittenhouse 👏 helped confirm that we want to record the handshake duration as soon as we resume the early connection.

---

@LPardue @ghedo

@evanrittenhouse and I discovered that we set the handshake duration right after resumption with 0-rtt. Wondering if this is ok or if we want to record the duration when the full handshake finishes.

https://github.com/cloudflare/quiche/blob/master/tokio-quiche/src/quic/io/worker.rs#L787-L788