Hitch
In the Varnish ecosystem we have our own TLS terminator: Hitch
It's lightweight, it doesn't even speak HTTP, and just shuffles through the bytes. The main reason why Hitch is so useful, is because it supports PROXY protocol, just like Varnish.
With PROXY protocol enabled, both on Hitch & Varnish, the X-Forwarded-For header is automatically set to the client IP of the original client, regardless of the number of hops in between.
There are even official packages available on https://packagecloud.io/varnishcache/hitch
Enabling PROXY protocol on Varnish
The typical runtime configuration for Varnish usually only supports plain HTTP. Here's an example:
varnishd -a :80 -f /etc/varnish/default.vcl -s malloc,2G
If you want to enabling PROXY protocol, just add an extra listening port. Here's how you do that:
varnishd -a :80 -a :8443,PROXY -f /etc/varnish/default.vcl -s malloc,2G
It's important that Hitch uses port 8443 as a backend, instead of 80
TLS detection
Magento also needs to know what protocol was used in the original client request. The TLV properties of the PROXY protocol information contain these details. It's just a matter of extracting them and turning them into the right X-Forwarded-Proto value.
The proxy VMOD is a module in Varnish 6 that can grab the TLV attributes.
vcl 4.1;
import proxy;
sub vcl_recv {
if (proxy.is_ssl()) {
set req.http.X-Forwarded-Proto = "https";
} else {
set req.http.X-Forwarded-Proto = "http";
}
}
The X-Forwarded-For header is automatically set
What if the problem is unrelated?
I'm pitching Hitch because it's simple, lightweight, it's an official TLS terminator, and it supports PROXY. I'm not saying Nginx is a bad choice, because a lot of people use it.
If it turns out that TLS termination and the X-Forwarded-For & X-Forwarded-Proto headers are not related to the problem, your best choice is to use varnishlog.
Just run the following command if you want to monitor the homepage:
varnishlog -g request -q "ReqUrl eq '/'"
If you want to monitor a different page, just modify the filter.
Then try to trigger the error, and add the varnishlog output here for me to anlyze. Maybe the logging will provide a lot more information about what's going on.
A couple of years ago I wrote an in-depth blog post about varnishlog: see https://feryn.eu/blog/varnishlog-measure-varnish-cache-performance/ for more information.
