HTTP body compression for gRPC requests

[jmjolysc]

Does Envoy support HTTP body compression for gRPC requests?

The gzip filter works well for REST requests, but doesn't seem to work for gRPC requests, even when following the filter documentation.

[mattklein123]

I don't see why it wouldn't work, but I'm not sure. Maybe it has to do with content-type or encoding headers. cc @lizan @gsagula

[lizan]

gRPC compression is different from HTTP body compression on wire, it is per message compression, a compression-flag has to be encoded in to gRPC frame header. I don't think it is implemented in gzip filter. See https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md for details.

[jmjolysc]

Thank you @mattklein123 and @lizan! I guess the gzip filter would also have to follow these guidelines to compress data. Would you consider gRPC compression as a feature improvement?

[mattklein123]

@jmjolysc yup switched over to enhancement.

[jmarantz]

@gsagula do you want to take this on?

[gsagula]

@jmarantz I would love too, this seems relevant to the stuff that I've been doing. I just need to get some other compression and ext_authz things out of my way. I guess this could be included in the compression filter that I've been working on but in a separated PR of course.

[davidkilliansc]

So I was also needing this and going back to the OP, I think the problem w/ the gzip filter isn't that it lacks gRPC support, but rather that it lacks HTTP2 support. The suspicion that @mattklein123 had about it being related to length or encoding headers looks spot on. The gzip filter won't compress responses that don't have a content-length header, and content-length and transfer-encoding: chunked headers are both no longer necessary in HTTP2.

I think it'd be ideal to initially just get HTTP2 compression working rather than try to get gRPC's custom compression protocol supported. HTTP2 compression would be more broadly useful, yet would still cover gRPC, plus it's simpler and more hardened than gRPC's custom compression protocol. As a datapoint, grpc-java still has its compression APIs as experimental due to some unresolved complexities and I've rarely seen it used. So ideally Envoy would just take care of all our H1 and H2 compression the same way w/o bringing gRPC's compression scheme into the fold.

[mattklein123]

I think it's reasonable to detect that the connection is h2 and do compression even with the absence of the transfer-encoding: chunked header.