2

I have installed a fresh copy of Magento 2 (2.3.5 with sample data) on a VPS which has the following stack: Ubuntu 18.04, Nginx, Mysql, and PHP 7.2 FPM.

This vanilla installation worked.

However, after enabling Varnish cache on Magento 2, some of the pages on the website are now displaying an Error 502 bad gateway message.

I have tried days troubleshooting this issue, without any luck. This is a fresh installation of Magneto 2, using the sample data.

I have come across a few posts online which suggest to increase the proxy_buffer_size within the server block of the nginx configuration file. However, this has not worked in my case and I am still receiving a 502 error on some of the links on the Magneto website.

Does anybody know how I could go about resolving this issue and get Varnish working?

I appreciate any help that you could provide. I have provided below the configuration files, logs, and setup information for Nginx, Varnish, and VPS setup.

Nginx host configuration file:


upstream fastcgi_backend {
        server  unix:/run/php/php7.2-fpm.sock;
}

server {
    server_name www.example.com;
    return 301 https://example.com$request_uri;

}


server {
    listen 80;
    listen [::]:80;

    server_name example.com;
    return 301 https://$server_name$request_uri;
}


server {
    listen 443 ssl http2;
    server_name www.example.com;

    return 301 https://example.com$request_uri;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

}


server {
        listen 443 ssl http2;
        server_name example.com;

    access_log /var/log/nginx/example.com-access.log;
    error_log /var/log/nginx/example.com-error.log;


        ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;


    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    # OCSP stapling
    ssl_stapling on;
    ssl_stapling_verify on;

    # HSTS (ngx_http_headers_module is required) (63072000 seconds)
    add_header Strict-Transport-Security "max-age=63072000" always;


    location / {
        proxy_pass http://127.0.0.1:6081;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Ssl-Offloaded "1";
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Port 443;
        proxy_redirect  http://example.com:8080/  /;
        proxy_http_version 1.1;

    proxy_buffer_size          256k;
    proxy_buffers              4 256k;
    proxy_busy_buffers_size    256k;
    }

}


server {
    server_name example.com;
    listen 127.0.0.1:8080;
    
    set $MAGE_ROOT /var/www/example.com/html;
    include /var/www/example.com/html/nginx.conf.sample;
}

Magento/default vcl config:


root@ubuntu-8gb-nbg1-1:/etc/varnish# cat default.vcl
# VCL version 5.0 is not supported so it should be 4.0 even though actually used Varnish version is 5
vcl 4.0;

import std;
# The minimal Varnish version is 5.0
# For SSL offloading, pass the following header in your proxy server or load balancer: 'X-Forwarded-Proto: https'

backend default {
    .host = "localhost";
    .port = "8080";
    .first_byte_timeout = 600s;
    .probe = {
#        .url = "/pub/health_check.php";
    .url = "/health_check.php";

        .timeout = 2s;
        .interval = 5s;
        .window = 10;
        .threshold = 5;
   }
}

acl purge {
    "localhost";
}

sub vcl_recv {
    if (req.method == "PURGE") {
        if (client.ip !~ purge) {
            return (synth(405, "Method not allowed"));
        }
        # To use the X-Pool header for purging varnish during automated deployments, make sure the X-Pool header
        # has been added to the response in your backend server config. This is used, for example, by the
        # capistrano-magento2 gem for purging old content from varnish during it's deploy routine.
        if (!req.http.X-Magento-Tags-Pattern && !req.http.X-Pool) {
            return (synth(400, "X-Magento-Tags-Pattern or X-Pool header required"));
        }
        if (req.http.X-Magento-Tags-Pattern) {
          ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
        }
        if (req.http.X-Pool) {
          ban("obj.http.X-Pool ~ " + req.http.X-Pool);
        }
        return (synth(200, "Purged"));
    }

    if (req.method != "GET" &&
        req.method != "HEAD" &&
        req.method != "PUT" &&
        req.method != "POST" &&
        req.method != "TRACE" &&
        req.method != "OPTIONS" &&
        req.method != "DELETE") {
          /* Non-RFC2616 or CONNECT which is weird. */
          return (pipe);
    }

    # We only deal with GET and HEAD by default
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Bypass shopping cart, checkout and search requests
    if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") {
        return (pass);
    }

    # Bypass health check requests
    if (req.url ~ "/pub/health_check.php") {
        return (pass);
    }

    # Set initial grace period usage status
    set req.http.grace = "none";

    # normalize url in case of leading HTTP scheme and domain
    set req.url = regsub(req.url, "^http[s]?://", "");

    # collect all cookies
    std.collect(req.http.Cookie);

    # Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") {
            # No point in compressing these
            unset req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            # unknown algorithm
            unset req.http.Accept-Encoding;
        }
    }

    # Remove all marketing get parameters to minimize the cache objects
    if (req.url ~ "(\?|&)(gclid|cx|ie|cof|siteurl|zanpid|origin|fbclid|mc_[a-z]+|utm_[a-z]+|_bta_[a-z]+)=") {
        set req.url = regsuball(req.url, "(gclid|cx|ie|cof|siteurl|zanpid|origin|fbclid|mc_[a-z]+|utm_[a-z]+|_bta_[a-z]+)=[-_A-z0-9+()%.]+&?", "");
        set req.url = regsub(req.url, "[?|&]+$", "");
    }

    # Static files caching
    if (req.url ~ "^/(pub/)?(media|static)/") {
        # Static files should not be cached by default
        return (pass);

        # But if you use a few locales and don't use CDN you can enable caching static files by commenting previous line (#return (pass);) and uncommenting next 3 lines
        #unset req.http.Https;
        #unset req.http.X-Forwarded-Proto;
        #unset req.http.Cookie;
    }

    return (hash);
}

sub vcl_hash {
    if (req.http.cookie ~ "X-Magento-Vary=") {
        hash_data(regsub(req.http.cookie, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1"));
    }

    # For multi site configurations to not cache each other's content
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }

    # To make sure http users don't see ssl warning
    if (req.http.X-Forwarded-Proto) {
        hash_data(req.http.X-Forwarded-Proto);
    }
    

    if (req.url ~ "/graphql") {
        call process_graphql_headers;
    }
}

sub process_graphql_headers {
    if (req.http.Store) {
        hash_data(req.http.Store);
    }
    if (req.http.Content-Currency) {
        hash_data(req.http.Content-Currency);
    }
}

sub vcl_backend_response {

    set beresp.grace = 3d;

    if (beresp.http.content-type ~ "text") {
        set beresp.do_esi = true;
    }

    if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") {
        set beresp.do_gzip = true;
    }

    if (beresp.http.X-Magento-Debug) {
        set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control;
    }

    # cache only successfully responses and 404s
    if (beresp.status != 200 && beresp.status != 404) {
        set beresp.ttl = 0s;
        set beresp.uncacheable = true;
        return (deliver);
    } elsif (beresp.http.Cache-Control ~ "private") {
        set beresp.uncacheable = true;
        set beresp.ttl = 86400s;
        return (deliver);
    }

    # validate if we need to cache it and prevent from setting cookie
    if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
        unset beresp.http.set-cookie;
    }

   # If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass
   if (beresp.ttl = 0s) {
        # Hit within TTL period
        return (deliver);
    }
    if (std.healthy(req.backend_hint)) {
        if (obj.ttl + 300s > 0s) {
            # Hit after TTL expiration, but within grace period
            set req.http.grace = "normal (healthy server)";
            return (deliver);
        } else {
            # Hit after TTL and grace expiration
            return (miss);
        }
    } else {
        # server is not healthy, retrieve from cache
        set req.http.grace = "unlimited (unhealthy server)";
        return (deliver);
    }
}

Error log from /var/log/nginx/error.log:


root@ubuntu-8gb-nbg1-1:/var/log/nginx#
2020/06/18 20:20:03 [error] 20205#20205: *1375 upstream sent too big header while reading response header from upstream, client: 127.0.0.1, server: example.com, request: "GET /men/tops-men/jackets-men.html HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "example.com", referrer: "https://example.com/what-is-new.html"

Varnish live log:

root@ubuntu-8gb-nbg1-1:/var/log/nginx# varnishlog

  • << BeReq >> 32841
  • Begin bereq 32840 fetch
  • Timestamp Start: 1592504403.501162 0.000000 0.000000
  • BereqMethod GET
  • BereqURL /men/tops-men/jackets-men.html
  • BereqProtocol HTTP/1.1
  • BereqHeader Host: example.com
  • BereqHeader X-Forwarded-Host: example.com
  • BereqHeader X-Real-IP: x.x.x.x
  • BereqHeader Ssl-Offloaded: 1
  • BereqHeader X-Forwarded-Proto: https
  • BereqHeader X-Forwarded-Port: 443
  • BereqHeader CF-IPCountry: GB
  • BereqHeader CF-RAY: 5a5705a789e73669-MAN
  • BereqHeader CF-Visitor: {"scheme":"https"}
  • BereqHeader user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0
  • BereqHeader accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
  • BereqHeader accept-language: en-GB,en;q=0.5
  • BereqHeader dnt: 1
  • BereqHeader referer: https://example.com/what-is-new.html
  • BereqHeader upgrade-insecure-requests: 1
  • BereqHeader cookie: __cfduid=d6e220d7c9b748b1fc0725cfa4138edce1592497645; mage-cache-storage=%7B%7D; mage-cache-storage-section-invalidation=%7B%7D; mage-cache-sessid=true; mage-messages=; recently_viewed_product=%7B%7D; recently_viewed_product_previous=%7B%7D; rece
  • BereqHeader CF-Request-ID: 036a41dcb500003669378e9200000001
  • BereqHeader CF-Connecting-IP: x.x.x.x
  • BereqHeader CDN-Loop: cloudflare
  • BereqHeader X-Forwarded-For: x.x.x.x, x.x.x.x, 127.0.0.1
  • BereqHeader grace: none
  • BereqHeader Accept-Encoding: gzip
  • BereqHeader X-Varnish: 32841
  • VCL_call BACKEND_FETCH
  • VCL_return fetch
  • BackendOpen 25 boot.default 127.0.0.1 8080 127.0.0.1 48908
  • BackendStart 127.0.0.1 8080
  • Timestamp Bereq: 1592504403.501272 0.000110 0.000110
  • Timestamp Beresp: 1592504403.791456 0.290293 0.290183
  • BerespProtocol HTTP/1.1
  • BerespStatus 502
  • BerespReason Bad Gateway
  • BerespHeader Server: nginx/1.14.0 (Ubuntu)
  • BerespHeader Date: Thu, 18 Jun 2020 18:20:03 GMT
  • BerespHeader Content-Type: text/html; charset=UTF-8
  • BerespHeader Content-Length: 182
  • BerespHeader Connection: keep-alive
  • TTL RFC -1 10 0 1592504404 1592504404 1592504403 0 0
  • VCL_call BACKEND_RESPONSE
  • TTL VCL -1 259200 0 1592504404
  • TTL VCL 0 259200 0 1592504404
  • VCL_return deliver
  • BerespUnset Content-Length: 182
  • BerespHeader Content-Encoding: gzip
  • BerespHeader Vary: Accept-Encoding
  • Storage malloc Transient
  • ObjProtocol HTTP/1.1
  • ObjStatus 502
  • ObjReason Bad Gateway
  • ObjHeader Server: nginx/1.14.0 (Ubuntu)
  • ObjHeader Date: Thu, 18 Jun 2020 18:20:03 GMT
  • ObjHeader Content-Type: text/html; charset=UTF-8
  • ObjHeader Content-Encoding: gzip
  • ObjHeader Vary: Accept-Encoding
  • Fetch_Body 3 length -
  • Gzip G F E 182 153 80 1144 1154
  • BackendReuse 25 boot.default
  • Timestamp BerespBody: 1592504403.791677 0.290515 0.000221
  • Length 153
  • BereqAcct 1164 0 1164 181 0 181
  • End
  • << Request >> 32840
  • Begin req 32839 rxreq
  • Timestamp Start: 1592504403.501053 0.000000 0.000000
  • Timestamp Req: 1592504403.501053 0.000000 0.000000
  • ReqStart 127.0.0.1 49160
  • ReqMethod GET
  • ReqURL /men/tops-men/jackets-men.html
  • ReqProtocol HTTP/1.1
  • ReqHeader Host: example.com
  • ReqHeader X-Forwarded-Host: example.com
  • ReqHeader X-Real-IP: x.x.x.x
  • ReqHeader X-Forwarded-For: x.x.x.x, x.x.x.x
  • ReqHeader Ssl-Offloaded: 1
  • ReqHeader X-Forwarded-Proto: https
  • ReqHeader X-Forwarded-Port: 443
  • ReqHeader Connection: close
  • ReqHeader Accept-Encoding: gzip
  • ReqHeader CF-IPCountry: GB
  • ReqHeader CF-RAY: 5a5705a789e73669-MAN
  • ReqHeader CF-Visitor: {"scheme":"https"}
  • ReqHeader user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0
  • ReqHeader accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8
  • ReqHeader accept-language: en-GB,en;q=0.5
  • ReqHeader dnt: 1
  • ReqHeader referer: https://example.com/what-is-new.html
  • ReqHeader upgrade-insecure-requests: 1
  • ReqHeader cookie: __cfduid=d6e220d7c9b748b1fc0725cfa4138edce1592497645; mage-cache-storage=%7B%7D; mage-cache-storage-section-invalidation=%7B%7D; mage-cache-sessid=true; mage-messages=; recently_viewed_product=%7B%7D; recently_viewed_product_previous=%7B%7D; rece
  • ReqHeader CF-Request-ID: 036a41dcb500003669378e9200000001
  • ReqHeader CF-Connecting-IP: x.x.x.x
  • ReqHeader CDN-Loop: cloudflare
  • ReqUnset X-Forwarded-For: x.x.x.x, x.x.x.x
  • ReqHeader X-Forwarded-For: x.x.x.x, x.x.x.x, 127.0.0.1
  • VCL_call RECV
  • ReqHeader grace: none
  • ReqURL /men/tops-men/jackets-men.html
  • ReqUnset Accept-Encoding: gzip
  • ReqHeader Accept-Encoding: gzip
  • VCL_return hash
  • VCL_call HASH
  • VCL_return lookup
  • VCL_call MISS
  • VCL_return fetch
  • Link bereq 32841 fetch
  • Timestamp Fetch: 1592504403.791718 0.290665 0.290665
  • RespProtocol HTTP/1.1
  • RespStatus 502
  • RespReason Bad Gateway
  • RespHeader Server: nginx/1.14.0 (Ubuntu)
  • RespHeader Date: Thu, 18 Jun 2020 18:20:03 GMT
  • RespHeader Content-Type: text/html; charset=UTF-8
  • RespHeader Content-Encoding: gzip
  • RespHeader Vary: Accept-Encoding
  • RespHeader X-Varnish: 32840
  • RespHeader Age: 0
  • RespHeader Via: 1.1 varnish (Varnish/5.2)
  • VCL_call DELIVER
  • RespUnset Age: 0
  • RespHeader Pragma: no-cache
  • RespHeader Expires: -1
  • RespHeader Cache-Control: no-store, no-cache, must-revalidate, max-age=0
  • RespUnset Server: nginx/1.14.0 (Ubuntu)
  • RespUnset X-Varnish: 32840
  • RespUnset Via: 1.1 varnish (Varnish/5.2)
  • VCL_return deliver
  • Timestamp Process: 1592504403.791760 0.290707 0.000041
  • RespHeader Content-Length: 153
  • RespHeader Connection: close
  • Timestamp Resp: 1592504403.791802 0.290749 0.000042
  • ReqAcct 1141 0 1141 286 153 439
  • End
  • << Session >> 32839
  • Begin sess 0 HTTP/1
  • SessOpen 127.0.0.1 49160 a0 127.0.0.1 6081 1592504403.501004 22
  • Link req 32840 rxreq
  • SessClose REQ_CLOSE 0.291
  • End

netstat output:


root@ubuntu-8gb-nbg1-1:/etc/nginx/sites-available# netstat -plntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:6081            0.0.0.0:*               LISTEN      19887/varnishd      
tcp        0      0 127.0.0.1:6082          0.0.0.0:*               LISTEN      19887/varnishd      
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      870/mysqld          
tcp        0      0 x.x.x.x:2222   0.0.0.0:*               LISTEN      7853/sshd           
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      20204/nginx: master 
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      20204/nginx: master 
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      18327/systemd-resol 
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      20204/nginx: master 
tcp6       0      0 :::6081                 :::*                    LISTEN      19887/varnishd      
tcp6       0      0 :::80                   :::*                    LISTEN      20204/nginx: master 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           18327/systemd-resol 
udp        0      0 0.0.0.0:68              0.0.0.0:* 

2

proxy_buffer_size 256k;

That size is huge and won't do you good. You can read the guide on how to tune it properly with one important note: fastcg_buffer_size and proxy_buffer_size are essentially the same thing, they are just applied to different upstream module of NGINX.

  • proxy_buffer_size is the buffer for header of HTTP response of NGINX's proxy module
  • fastcgi_buffer_size is the buffer for header of HTTP response of NGINX's fastcgi module

Because it is a header, it makes little sense to put things like 256k for that buffer because even a complete page HTML will hardly reach that size.

When you enable Varnish, Magento code will indeed output more HTTP headers, specifically X-Magento-Tags may be very large depending on catalog size and if looking at a category page.

For this you need to adjust both proxy_buffer_size and fastcgi_buffer_size. Why is because in your setup NGINX uses both modules: it proxies requests to Varnish (TLS termination), and does FastCGI communication to PHP-FPM.

So aside from tuning proxy_buffer_size, you need to do the same for fastcgi_buffer_size and tune it up as needed. You can either do this alongside fastcgi_ directives in /var/www/example.com/html/nginx.conf.sample or add this globally, e.g. /etc/nginx/conf.d/custom.conf and make sure it is loaded from nginx.conf.

Don't forget to tune *_busy_buffers_size as well as per the guide. E.g.:

fastcgi_buffer_size 16k;
fastcgi_busy_buffers_size 16k;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.