3

We're running Magento 2.3.1 on AWS EC2(t2.medium) Instance with an ELB. Webserver is nginx. Sometimes, we receive the "Upstream sent too big header while reading response header from upstream."-Error in our logs. After researching this issue, I found out that the most popular solution seems to be adjusting the fastcgi buffer size. I've done that without any effect.

To determine the current fastcgi response sizes, I did:

$ awk '($9 ~ /200/) { i++;sum+=$10;max=$10>max?$10:max; } END { printf("Maximum: %d\nAverage: %d\n",max,i?sum/i:0); }' /var/log/nginx/access.log

Maximum: 1613895
Average: 51433

Due to the huge amount of response sizes, I am not sure if the issue is nginx related. My guess is that the current buffer size rules are fine and Magento might has a bug with huge response sizes?

My current buffer sizes are:

fastcgi_pass   fastcgi_backend;
fastcgi_buffers 8 32k;
4
  • Does this happen only on certain requests? Or sporadically around the website? I have historically seen observers triggering SetCookie for the same key/value pair multiple times which inflated the header size. the solution was to the fix the php code there. Apr 11 '19 at 7:10
  • The cookie thing is something I heard before and makes sense in my opinion. Which fix did you apply? The errors typically occur on catalog sites or search result pages. Apr 11 '19 at 13:53
  • I was able to replicate semi-reasonably on a test environment. I put some logging deep in the magento cookie model where the set cookie logic occurs so that i could log the names/values of each cookie set per request (i think i used uniqid and assigned to a superglobal to ensure i could pick out each request individually in the logs) It was pretty hacky and nothing you'd want to deploy near production. Once i could find out what cookies were being set i was able to add stack traces to the log to figure out what was going on. Apr 12 '19 at 6:44
  • Makes sense. Let me try that. Apr 13 '19 at 7:02
4

In my case fresh magento 2.4.1 (nginx on ubuntu) solution (by David Lambauer) worked:

In terminal:

  • cd /etc/nginx/sites-available/
  • ls

Open your nginx config file:

  • code magento24.local.config

Find "fastcgi_buffers"

I change it like that:

        #fastcgi_buffers 1024 4k;
        fastcgi_buffers 16 256k;
        fastcgi_buffer_size 256k;

Save that config file. Restart nginx :

  • sudo service nginx restart
3

Magento2 always have the issues with the response size. since you can solve it by enabling the cache which all recommended. Can you please try the below setup in nginx configuration to your location block

location / {
    proxy_pass       http://upstream;
    ...

    proxy_buffer_size          128k;
    proxy_buffers              4 256k;
    proxy_busy_buffers_size    256k;
  }

and restart the nginx service.

1
  • 2
    Could you explain why you chose these settings? Apr 11 '19 at 13:51
1

Coming back after a year, I don't have this issue anymore. I can't exactly explain the final solution but regarding the fastcgi parameters, I can tell that the following params work fine and are also proven by our hoster.

fastcgi_buffers 16 256k;
fastcgi_buffer_size 256k;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.