Wait time of 400ms should I look at page caching or something else? (Dedicated Magento from NginX + PHP-FPM)

Question

Firstly could some one help me with what specifically this 'wait time' is. e.g. php render? browser render ? can i do anything about it?

Should i be looking to implement page cache to reduce this 'wait time'?

While my site feels pretty fast, I have been getting more and more concerned about the figures shown on tools.pingdom.com,

I appreciate it would be somewhat difficult to help given i'v chosen not to include my magento domain, but I feel the question is generic enough to apply to all who run magento. I did consider posting in the general webmasters thread, but since this is a dedicated magento store, i felt it best to start here.

External Tests

http://tools.pingdom.com

http://gtmetrix.com/

https://developers.google.com/speed/pagespeed/insights/

Answer 1

Honestly 400ms isn't too shabby for a Magento CE installation although I don't know what kind of page that is (Category / product / cart etc).

If you want a quick way to speed up the shop look into using a Full Page Cache extension like Brim FPC. It's easy to install and configure and it works more effective than the Magento default cache.

Another thing you can do is to switch from file based caching to memory based caching. Since the RAM memory is faster in reading and writing than the disk it will save you some time. Check out this article for that.

And last but not least: the database. In a lot of cases this will prove to be a bottle neck. Check your slow query log or use a tool like mytop to see how busy the MySQL server is. Moving it from the localhost to a seperate server that can be finetuned specifically for MySQL will also give you better performance

Answer 2

After some part time messing around with a copy of the site on a dev server, i finally got to the bottom of the lag issue. What I found totally infuriating is the smaller and far less powerful dev server would return pages faster! Which made no sense at all, as it was a mirror image of the site and db with the same OS and even the same versions of nginx php etc..

But then it hit me, what i didn't rsync over was the cache and sessions folders.. As I had an SSD webserver, i had allowed magento to use "file storage" for the cache and sessions. This had managed to accumulate hundreds of thousands of tiny files in mageDIR/var/session and /cache folders which i thought was no problem.. turns out it's a very big problem if left unchecked for a couple weeks.

Conclusion (for me, on my system) even with dedicated server with SSDs in raid, file storage is never the way to go. It would seem the speed of IO on the storage in this case wasn't an issue, i guess the way in which it handles huge amounts of tiny files is?

In any case Switching to a localhost redis-server for cache and sessions has reduced all lag and has been running very well for a month on the live machine. Page "wait" as reported by the speed test sites was in the 240ms area

Further savings were made when I switched ngx_pagespeed to memcached storage instead of file storage.

Now initial wait is down to 140ms odd plus 20ms odd per browser connection, but more importantly to me google no longer suggests I need to "reduce your server response time" which is great news as that warning from them is what sent me down this rabbit hole.

-more reading:

...the drawback of putting potentially thousands of files in a single directory. If you've been around servers long enough, you know that directories become slower and slower as more files are added...

Redis: Better Session Storage for Magento October 28, 2015 by Scott Hebert