Received 'Status::internal("Missing request message.")' in case of cancellation by client

[madhurishgupta]

Bug Report

Version

Verified it in tonic 0.12 and 0.14.

Platform

• 24.6.0 Darwin Kernel
• Verified on linux as well.

Crates

tonic

Description

If the cancellation is sent from the client before the full request message reached the server, it leads to Status::internal("Missing request message.").

I reproduced this issue locally by java gRPC client (with hedging) and tonic server with request size of around 1 MB.

Error is thrown here.

Expected Behaviour:

• Case1: If the cancellation is sent from the client before the full request message reached the server, then request future must be dropped and no response/error should be returned to the tower layers.
• Case2: If the cancellation is sent from the client after the full request message reached the server, then request future must be dropped and no response/error should be returned to the tower layers. This is working as expected.

I'm not sure if this is the intended behaviour that needs to be handled by the server. In my opinion, case 1 should drop the future instead of return Status::internal("Missing request message.").

[madhurishgupta]

@LucioFranco @dfawley looking for your inputs.
Thank you!

[dfawley]

@madhurishgupta where are you seeing the "Missing request message" error manifest? The h2 layer should see the cancellation and not send anything back to the client when this occurs. And even if it didn't, the client shouldn't be using any returned status, either, since the client initiated cancelation. To me this scenario seems like something that should only have any effect internally and not be visible to the application. It might be helpful if you could provide a small reproduction example showing the error. Thank you.

[madhurishgupta]

@dfawley thanks for the prompt reply.

Yes, the error Status::internal("Missing request message.") is not sent back to the client. But this error is propagated to all the tower layers on the server side (metrics layer, trace layer etc).

For reproduction, I used java gRPC client with hedging delay of 1 second and tonic server in which handler implementation has sleep of 900 ms. Then I sent the request of size 1 MB to the server. Below is the flow:

• Request 1 starts from the client at time=0 ms.
• Request 1 goes to sleep on the server side for 900 ms.
• Due to hedging, the request 2 starts from the client after time = 1 second.
• After few milliseconds, response of request 1 reaches the client and it cancels the request 2.
• Server receives the cancellation of request 2 but full request 2 didn't reach the server, instead of dropping the request, it throws Status::internal("Missing request message.") here during deserialization of the message body.

Below are the logs of trace layer on server side:

2025-09-05T17:28:10.799777Z DEBUG request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_request: started processing request
2025-09-05T17:28:11.716159Z DEBUG request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_response: finished processing request latency=916 ms response_headers={"content-type": "application/grpc"}
2025-09-05T17:28:11.720559Z DEBUG request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_eos: end of stream stream_duration=4 ms status=0


2025-09-05T17:28:11.737302Z DEBUG request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_request: started processing request
2025-09-05T17:28:11.737920Z DEBUG request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_response: finished processing request latency=0 ms status=13 response_headers={"content-type": "application/grpc", "grpc-status": "13", "grpc-message": "Missing%20request%20message."}
2025-09-05T17:28:11.737998Z ERROR request{method=POST uri=http://localhost:50051/helloworld.Greeter/SayHello version=HTTP/2.0}: tower_http::trace::on_failure: response failed classification=Code: 13 latency=0 ms

My hunch is that tonic is trying to deserialize the RST_STREAM (cancellation from client) which is leading to this behavior.

[ankurmittal]

This error happens because here decoder returns none and when grpc tonic gets no message here, it just returns internal error instead of correctly mapping it to cancelled

[madhurishgupta]

@dfawley @LucioFranco can you please share your thoughts on this?