1

I have configured HTTPs on Ubuntu Nginx. It works perfectly.

However once I implement Varnish,apart from the homepage, all other pages/CSS and Javascript files returns 500 Server Errors.

My nginx. config file

server {
    
        listen [::]:443 ssl http2 ipv6only=on; # managed by Certbot
        listen 443 ssl http2;  # managed by Certbot
        
        server_name example.com  www.example.com;
        index  index.php index.html index.htm; 
        
        ssl_certificate /etc/letsencrypt/live/www.example.com/fullchain.pem; # managed by Certbot
        ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem; # managed by Certbot   
        
         # link to intermediate configuration      
        include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
        
        # curl https://ssl-config.mozilla.org/ffdhe2048.txt > /path/to/dhparam
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
               
         ssl_trusted_certificate /etc/letsencrypt/live/www.example.com.com/chain.pem; # managed by Certbot
         
         # OCSP stapling
       #  ssl_prefer_server_ciphers on;
         ssl_stapling on; # managed by Certbot
         ssl_stapling_verify on; # managed by Certbot  
        
         resolver 8.8.8.8 8.8.4.4 valid=300s;
         resolver_timeout 30s;
         keepalive_timeout 300s;
         
         # HSTS (ngx_http_headers_module is required) (63072000 seconds)           
         add_header Strict-Transport-Security "max-age=31536000" always; # managed by Certbot
         
         
         # location / {
         #      try_files $uri $uri/ =404;
         # }  commented out because the nginx.conf.sample has it
           
          location /sitemap.xml {
             alias /opt/magento/public_html/sitemaps/sitemap.xml;
          }
         
         
         # gzip off; ---- commented out because magento has it in the magento file
         
         location / {
                 proxy_pass http://127.0.0.1:6081;
                 proxy_set_header Host $http_host;
                 proxy_set_header X-Forwarded-Host $http_host;
                 proxy_set_header X-Real-IP $remote_addr;
                 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                 proxy_set_header Ssl-Offloaded “1”;
                 proxy_set_header X-Forwarded-Proto https;
                 proxy_set_header X-Forwarded-Port 443;
                 #proxy_hide_header X-Varnish;
                 #proxy_hide_header Via;
                 proxy_set_header X-Forwarded-Proto $scheme;
                 
                 ## Prevent PHP files from being served as static assets, and fall back to index.php if file does not exist
                 try_files $uri $uri/ /index.php$is_args$args;
         }  
   
    }
    

I suspect that the problem is with the Magento supplied default.vcl.

I.e

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .first_byte_timeout = 600s;
  #  .probe = {
  #      .url = "/pub/health_check.php";
  #      .timeout = 2s;
   #     .interval = 5s;
    #    .window = 10;
     #   .threshold = 5;
   #}
}

I did a curl from my terminal.

i. e

curl -I -v –location-trusted 'example.com'

It returns:

* Failed to convert –location-trusted to ACE; string contains a disallowed character

* Closing connection -1
curl: (3) Failed to convert –location-trusted to ACE; string contains a disallowed character

* Rebuilt URL to: example.com/
*   Trying 46.137.70.167...
* TCP_NODELAY set
* Connected to example.com (46.137.70.167) port 80 (#0)
> HEAD / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< Server: nginx/1.14.0 (Ubuntu)
Server: nginx/1.14.0 (Ubuntu)
< Date: Wed, 06 Jan 2021 08:12:40 GMT
Date: Wed, 06 Jan 2021 08:12:40 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 194
Content-Length: 194
< Connection: keep-alive
Connection: keep-alive
< Location: https://example.com/
Location: https://example.com/

< 
* Connection #0 to host example.com left intact

Any Idea where the problem might be, or how i can debug it

EDITED QUESTION;

I ran the following for one of the pages that returns a 500 server error:

 varnishlog -g request -q "ReqUrl eq '/desio-colored-contacts-attitude-monthly.html'" 

 varnishlog -g request -q "ReqUrl eq 'https://example.com/desio-colored-contacts-attitude-monthly.html'"   

Nothing came up for both.

MY VARNISH LOGS

varnish logs do not return any data for any other page except the home page.

this is log for home page;

26912#26912: *42 rewrite or internal redirection cycle while internally redirecting to "/index.php", client: 79.77.194.203, server: molenses.com, request: "GET /desio-colored-contact-lenses/desio-colored-contacts-attitude-monthly.html HTTP/2.0", host:"example.com", referrer: "https://example.com/"

ALSO, my varnish log from the home page shows this:

sudo varnishlog
*   << Request  >> 65538     
-   Begin          req 65537 rxreq
-   Timestamp      Start: 1609923430.683170 0.000000 0.000000
-   Timestamp      Req: 1609923430.683170 0.000000 0.000000
-   ReqStart       127.0.0.1 37840
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.0
-   ReqHeader      Host: example.com
-   ReqHeader      X-Forwarded-Host: molenses.com
-   ReqHeader      X-Real-IP: 79.77.194.203
-   ReqHeader      X-Forwarded-For: 79.77.194.203
-   ReqHeader      Ssl-Offloaded: “1”
-   ReqHeader      X-Forwarded-Proto: https
-   ReqHeader      X-Forwarded-Port: 443
-   ReqHeader      X-Forwarded-Proto: https
-   ReqHeader      Connection: close
-   ReqHeader      cache-control: max-age=0
-   ReqHeader      upgrade-insecure-requests: 1
-   ReqHeader      user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
-   ReqHeader      accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
-   ReqHeader      sec-fetch-site: none
-   ReqHeader      sec-fetch-mode: navigate
-   ReqHeader      sec-fetch-user: ?1
-   ReqHeader      sec-fetch-dest: document
-   ReqHeader      accept-encoding: gzip, deflate, br
-   ReqHeader      accept-language: en-GB,en-US;q=0.9,en;q=0.8
-   ReqHeader      cookie: private_content_version=29ffccba7f7ce57a95dd89376bb705c9; _fbp=fb.1.1608726914824.1476315738; _ga=GA1.2.1527235712.1608732847; __stripe_mid=d93a05af-d56f-4b12-b411-bb1a8942dbca804aac; AMCVS_8F99160E571FC0427F000101%40AdobeOrg=1; s_cc=true; __atuv
-   ReqUnset       X-Forwarded-For: 79.77.194.203
-   ReqHeader      X-Forwarded-For: 79.77.194.203, 127.0.0.1
-   VCL_call       RECV
-   ReqHeader      grace: none
-   ReqURL         /
-   ReqUnset       accept-encoding: gzip, deflate, br
-   ReqHeader      Accept-Encoding: gzip
-   VCL_return     hash
-   VCL_call       HASH
-   VCL_return     lookup
-   Hit            3 83702.966373 259200.000000 0.000000
-   VCL_call       HIT
-   VCL_return     deliver
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Server: nginx/1.14.0 (Ubuntu)
-   RespHeader     Date: Wed, 06 Jan 2021 08:12:13 GMT
-   RespHeader     Content-Type: text/html; charset=UTF-8
-   RespHeader     Pragma: cache
-   RespHeader     Cache-Control: max-age=86400, public, s-maxage=86400
-   RespHeader     Expires: Thu, 07 Jan 2021 08:12:12 GMT
-   RespHeader     X-Magento-Tags: cms_b_block-slideshow,store,cms_b,cat_c,cat_c_6,cat_c_10,cat_c_14,cat_c_19,cat_c_21,cms_p_7,cms_b_block-contact-header,cms_b_block-about-header,cms_b_newsletter-popup,cms_b_social-newsletter-popup,cms_b_id-1-block-1,cms_b_id-1-block-2,cms
-   RespHeader     X-Content-Type-Options: nosniff
-   RespHeader     X-XSS-Protection: 1; mode=block
-   RespHeader     X-Frame-Options: SAMEORIGIN
-   RespHeader     Vary: Accept-Encoding
-   RespHeader     Content-Encoding: gzip
-   RespHeader     X-Varnish: 65538 3
-   RespHeader     Age: 2697
-   RespHeader     Via: 1.1 varnish (Varnish/5.2)
-   VCL_call       DELIVER
-   RespUnset      Age: 2697
-   RespUnset      Pragma: cache
-   RespHeader     Pragma: no-cache
-   RespUnset      Expires: Thu, 07 Jan 2021 08:12:12 GMT
-   RespHeader     Expires: -1
-   RespUnset      Cache-Control: max-age=86400, public, s-maxage=86400
-   RespHeader     Cache-Control: no-store, no-cache, must-revalidate, max-age=0
-   RespUnset      X-Magento-Tags: cms_b_block-slideshow,store,cms_b,cat_c,cat_c_6,cat_c_10,cat_c_14,cat_c_19,cat_c_21,cms_p_7,cms_b_block-contact-header,cms_b_block-about-header,cms_b_newsletter-popup,cms_b_social-newsletter-popup,cms_b_id-1-block-1,cms_b_id-1-block-2,cms
-   RespUnset      Server: nginx/1.14.0 (Ubuntu)
-   RespUnset      X-Varnish: 65538 3
-   RespUnset      Via: 1.1 varnish (Varnish/5.2)
-   VCL_return     deliver
-   Timestamp      Process: 1609923430.683300 0.000130 0.000130
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Content-Length: 13362
-   RespHeader     Connection: close
-   Timestamp      Resp: 1609923430.683348 0.000178 0.000049
-   ReqAcct        1644 0 1644 396 13362 13758
-   End            

*   << Session  >> 65537     
-   Begin          sess 0 HTTP/1
-   SessOpen       127.0.0.1 37840 a0 127.0.0.1 6081 1609923430.683114 22
-   Link           req 65538 rxreq
-   SessClose      REQ_CLOSE 0.000
-   End            
4
  • Can you please figure out if the Nginx TLS termination vhost is returning an HTTP 500, or if it's the Nginx webserver vhost. Please share the error message that was returned either by Nginx, or by Magento. Please also share a complete varnishlog extract for the page that failed. You can for example use the following command: varnishlog -g request -q "ReqUrl eq '/your-page'". This information will allow me to further guide you to a solution. Jan 6 at 8:35
  • hey @ThijsFeryn . thanks for help. how do i determine if Nginx TLS termination vhost is returning an HTTP 500 ? What commands do i use on my terminal
    – theSeeker
    Jan 6 at 8:50
  • @ThijsFeryn i have answered the rest of your questions above. i.e i edited my question.
    – theSeeker
    Jan 6 at 8:51
  • If your research shows no entries in varnishlog for the failing pages, this probably means the problem starts in the TLS termination vhost. My advice: increase the verbosity of your Nginx error logs, and try to come up with an error. Don't hesitate to share the error here. Jan 7 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.