very confused about Varnish. Is it running or not? no headers it only says Pregma: cache

Question

Maybe someone can help:

on Ubuntu, nginx, latest magento (2.4.6), varnish 6

This is my Nginx conf setup:

server {

   listen 8080 default_server;
   listen [::]:8080  default_server;


    listen 443 ssl http2;

    server_name example.com www.example.com;

   ssl_certificate      /etc/letsencrypt/live/example.com/fullchain.pem;
   ssl_certificate_key  /etc/letsencrypt/live/example.com/privkey.pem;

    #include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
    ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 24h;
    keepalive_timeout 300s;

    # OCSP stapling
    ssl_stapling on;
    ssl_stapling_verify on;

    # HSTS (ngx_http_headers_module is required) (63072000 seconds)
    add_header Strict-Transport-Security "max-age=63072000" always;
    set $MAGE_ROOT /var/www/magento2;
    set $MAGE_MODE production;
    include /var/www/magento2/nginx.conf.sample;
    access_log /var/log/nginx/magento2-access.log;
    error_log /var/log/nginx/magento2-error.log;
}
  

This is my varnish /etc/varnish/default.vcl

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .first_byte_timeout = 600s;
    .probe = {
        .url = "/health_check.php";
        .timeout = 2s;
        .interval = 5s;
        .window = 10;
        .threshold = 5;
   }
}
sub vcl_deliver {
    if (resp.http.x-varnish ~ " ") {
        set resp.http.X-Magento-Cache-Debug = "HIT";
        set resp.http.Grace = req.http.grace;
    } else {
        set resp.http.X-Magento-Cache-Debug = "MISS";
    }

    # Not letting browser to cache non-static files.
    if (resp.http.Cache-Control !~ "private" && req.url !~ "^/(pub/)?(media|static)/") {
        set resp.http.Pragma = "no-cache";
        set resp.http.Expires = "-1";
        set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
    }

    if (!resp.http.X-Magento-Debug) {
        unset resp.http.Age;
    }
#    unset resp.http.X-Magento-Debug;
#    unset resp.http.X-Magento-Tags;
#    unset resp.http.X-Powered-By;
#    unset resp.http.Server;
#    unset resp.http.X-Varnish;
}

When I do curl -I https://example.com

HTTP/2 200 
server: nginx/1.18.0 (Ubuntu)
date: Tue, 08 Aug 2023 20:13:23 GMT
content-type: text/html; charset=UTF-8
content-length: 199266
vary: Accept-Encoding
set-cookie: PHPSESSID=gge00en22a0548v46nifalmcon; expires=Tue, 08 Aug 2023 21:13:22 GMT; Max-Age=3600; path=/; domain=example.com; secure; HttpOnly; SameSite=Lax
pragma: cache
cache-control: max-age=1209600, public, s-maxage=1209600
expires: Tue, 22 Aug 2023 20:13:22 GMT
x-magento-tags: store,cms_b,cat_c,cat_c_391,cat_c_393,cat_c_394,cat_c_395,cat_c_396,cat_c_397,cat_c_398,cms_p_50,cms_b_sexshop_cart_promotion,cms_b_ox-hotspot-sp-home-slider-01,cat_p_18447,cat_p,cat_p_18439,cat_p_18438,cat_p_18415,cat_p_18379,cat_p_18368,cat_p_15614,cat_p_15413,mfb_p_0,mfb_p_1,cms_b_sexshop2-footer-content-clean,cms_b_
x-magento-debug: 1

I am in developper mode and I do not see anything in my header saying my varnish is working!

What should I do?

Answer 1

Check ports of running processes

Before we look at Varnish and potential ways to debug Varnish, let's first have a look at what process listens on the relevant ports.

Run the following command to inspect the usage of ports 80, 8080 and 443:

sudo netstat -tulpn | grep -E "80|443"

This could be the output:

tcp  0  0 0.0.0.0:80        0.0.0.0:*     LISTEN      34715/varnishd
tcp  0  0 0.0.0.0:8080      0.0.0.0:*     LISTEN      34613/nginx: master
tcp  0  0 0.0.0.0:443       0.0.0.0:*     LISTEN      34613/nginx: master
tcp6 0  0 :::80             :::*          LISTEN      34715/varnishd

That way you already have a good indication whether Varnish actually runs on port 80 and what service handles TLS over port 443.

Doublecheck system settings

To doublecheck the output, we also want to have a look at the Systemd settings of Varnish. Please run the following command:

sudo systemctl cat varnish

This will print the Varnish Systemd unit file. The output can look like this:

[Unit]
Description=Varnish Cache, a high-performance HTTP accelerator
After=network-online.target nss-lookup.target

[Service]
Type=forking
KillMode=process

# Maximum number of open files (for ulimit -n)
LimitNOFILE=131072

# Locked shared memory - should suffice to lock the shared memory log
# (varnishd -l argument)
# Default log size is 80MB vsl + 1M vsm + header -> 82MB
# unit is bytes
LimitMEMLOCK=85983232

# Enable this to avoid "fork failed" on reload.
TasksMax=infinity

# Maximum size of the corefile.
LimitCORE=infinity

ExecStart=/usr/sbin/varnishd \
      -a :80 \
      -a localhost:8443,PROXY \
      -p feature=+http2 \
      -f /etc/varnish/default.vcl \
      -s malloc,256m
ExecReload=/usr/sbin/varnishreload

[Install]
WantedBy=multi-user.target

If the -a option is not 80, Varnish is either not running correctly, or is hosted behind another proxy. The netstat output will paint a clear picture of this.

Reload VCL

I'm not sure if you reloaded the VCL after commenting out the unset of certain headers. If you want to be sure that X-Varnish is correctly set after performing those VCL changes, please either restart Varnish or reload the VCL.

Here's how you reload the VCL:

sudo varnishreload

Check varnishlog

When you think Varnish is configured correctly, but you're still not 100% sure it's handling the HTTP traffic, please run varnishlog.

Here's a command that will show traffic going to the homepage:

sudo varnishlog -g request -q "ReqUrl eq '/'"

When this command doesn't show any relevant output once you start sending traffic to the homepage, there might be something wrong.

Answer 2

According to your vcl you must be recieving X-Magento-Cache-Debug too.

Also, please make sure to delete var/page_cache in Magento's root directory and check the folder again after you hit the page to make sure no cache files are generated there so that we know that your installation is not using Magento's default full page cache.