X-Forwarded-For & X-Real-Ip handling doesn't properly respect private IPs

[DerRockWolf]

Describe the bug

I hope I've explained everything well enough as this issue is a bit more complex.

In my setup clients can connect externally (original source IP is a public IP) and internally (source IP is a internal/public IP). In both cases the traffic flows via a reverse proxy.
External clients can connect just fine. But internal clients are blocked and Anubis logs: "[misconfiguration] X-Real-Ip header is not set". This even happens when XFF_STRIP_PRIVATE is set to false. I've temporarily fixed this by adding the X-Real-Ip header at the proxy.

This happens because the XForwardedForUpdate handler appends the requests RemoteAddr (in my case the IP of the reverse proxy) to the later flattened list of IPs. In the case where XFF_STRIP_PRIVATE is true this doesn't really matter as it will be skipped, but if it's false the XFF header will be updated to only contain the IP of the proxy.

After that the XForwardedForToXRealIP handler is called and it will use the updated XFF header and set X-Real-Ip to the first public IP. In my setup (regardless if XFF_STRIP_PRIVATE is false) this always results in an empty X-Real-Ip header.

Steps to reproduce

1. Setup Anubis behind a reverse proxy
2. Connect via a internal client
3. Observe the issue in browser and Anubis logs ("[misconfiguration] X-Real-Ip header is not set")

Expected behavior

I'm not really sure 🙃

If you don't think this is a bug and you think that operators with such an setup should just add X-Real-Ip at the proxy, we should at least properly document it.
If you agree with this bug: there are multiple possible solutions that I was able to find.

Execute the XForwardedForToXRealIP handler before XForwardedForUpdate, so that XForwardedForToXRealIP uses the original XFF header. This is not the most complete solution as the XForwardedForUpdate will still append the requests RemoteAddrand if XFF_STRIP_PRIVATE=false then it will be the only IP in XFF (although I'm not sure what the "correct" way would be).

Introduce an setting that allows the user to disable appending the RemoteAddr to the list, so that the update XFF will be the IP the proxy got the request from and not the IP that Anubis got the request from.

For both fixes we must also add support of the XFF_STRIP_PRIVATE setting to the XForwardedForToXRealIP handler. The currently used package just skips any private IPs, so we might need to switch to something else.

Your operating system and its version.

Not relevant

Your browser and its version.

Not relevant

Additional context

Slightly relates to #103

For me it is not quite clear (also not after reading https://anubis.techaro.lol/docs/admin/caveats-xff/) why Anubis flattens the XFF header in the first place.
I also didn't really understand why one would set XFF_STRIP_PRIVATE to false currently. With the above findings it would only be beneficial in cases where clients are directly talking to Anubis without going through a reverse proxy, which from what I can tell isn't even a supported setup, and even then it wouldn't work as the X-Real-Ip header would end up empty.

(Sorry if this isn't so well formulated, I was basically already finished writing this issue and then my browser crashed 🥲)

[Xe]

I am taking a look at this. What reverse proxy are you using so I can replicate things in my lab?

[DerRockWolf]

I am taking a look at this. What reverse proxy are you using so I can replicate things in my lab?

I'm using Envoy Gateway via Gateway API. But I expect the same behavior with any other reverse proxy when connecting via a private IP. So any reverse proxy that sets X-Forwarded-For but doesn't set X-Real-Ip should do the trick.

In my case the X-Forwarded-For header that was sent to Anubis just contained one IP (the original source IP).

[SlyEcho]

The error "[misconfiguration] X-Real-Ip header is not set" is really just to make sure that Anubis is correctly installed. It is assuming that a reverse proxy will be on a private or local network, because that is a common set up. Also, on v6 it's possible that even local network addresses are globally routable. I think there's two solutions to this:

1. Add an option for "trusted IP ranges" that are assumed to be client addresses, even if they are on a private network.
2. Add an option to explicitly define the address or range of the reverse proxy.

Option 2 is also pretty common to see used in various apps and services that are expected to be reverse proxied.

[DerRockWolf]

I understand why Anubis errors with "[misconfiguration] X-Real-Ip header is not set". My issue is that, at least in my setup, the handling (updating XFF and deriving X-Real-Ip) seems wrong, at least if the client is coming from an private IP.