Module skips custom headers

[SkowRon96]

I have prepared a forward proxy as follows:

`http {
log_format main '[$time_local] request="$request" status="$status" bytes="$body_bytes_sent" user="$http_user_agent" request_time="$request_time" x_appname="$http_x_appname" x_appcontext="$http_x_appcontext"';

access_log /var/log/nginx/access-8888.log main;
ignore_invalid_headers on;

server {
listen 8888;
resolver 8.8.8.8 ipv6=off;
proxy_connect;
proxy_connect_allow all;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

location / {
proxy_pass http://$http_host$request_uri;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}`

When my request is via http, the headers are captured fine:
curl --proxy http://localhost:8888 -H "X-AppName: TEST" http://ipinfo.io/json
2024/12/18 12:18:53 [debug] 15#0: *64 http header: "Host: ipinfo.io"
2024/12/18 12:18:53 [debug] 15#0: *64 http header: "User-Agent: curl/7.68.0"
2024/12/18 12:18:53 [debug] 15#0: *64 http header: "Accept: /"
2024/12/18 12:18:53 [debug] 15#0: *64 http header: "Proxy-Connection: Keep-Alive"
2024/12/18 12:18:53 [debug] 15#0: *64 http header: "X-AppName: TEST"

When my request is via https, headers are not visible:
curl --proxy http://localhost:8888 -H "X-AppName: TEST" https://ipinfo.io/json

2024/12/18 12:19:10 [debug] 15#0: *66 http header: "Host: ipinfo.io:443"
2024/12/18 12:19:10 [debug] 15#0: *66 http header: "User-Agent: curl/7.68.0"
2024/12/18 12:19:10 [debug] 15#0: *66 http header: "Proxy-Connection: Keep-Alive"

What could be the problem? Have I missed any settings or is it a bug in the module itself?

[danielboven]

I had the same problem as described above at first, but then realised the problem is based on a misconception of the forward proxy and the request you're making. A proxy request makes an HTTP CONNECT request to a proxy, the forward proxy then blindly forwards to the destination. Any headers that are visible to the forward proxy are seperate from the headers meant for your destination, as the forward proxy blindly transfers them to the destination and doesn't have access to them. See also RFC 7231 section 4.3.6 for more information about this.

The solution is simple: set headers meant for your proxy separately, as they should be included in the HTTP CONNECT request only. You can do this in curl using the --proxy-header option (instead of the -H option, as that is for the destination request).
When using OpenResty, you can get the value of this proxy header in the Lua code using: ngx.var.http_x_appname