8

We are running Magento 1.9.2.1 with Lesti_Fpc on an adequately sized managed server. Initially, we used the default file cache, which was fine. But after the catalog grew (although I think ~8000 products is not too bad) and crawlers became more aggressive, the site became slow as soon as the cache was getting a bit bigger. When the cache was cleared, everything was running quick again.

We tried to switch to APC as a cache backend via the following entry in the local.xml:

<global>
    <cache>
        <backend>apc</backend>
        <prefix>MYSHOP_</prefix>
    </cache>
</global>

But this made the problems even worse. I then read that Cm_Cache_Backend_File is made for this problem and integrated it via:

<global>
    <cache>
        <backend>Cm_Cache_Backend_File</backend>
    </cache>
</global>

This feels a bit better, but the problem is still the same. To keep the cache small and tidy, I also integrated Aoe_CacheCleaner, but this also does not help. Still, as soon as the cache is cleared, everything is running quick again.

EDIT:
Based on the answer by infabo, I also activated Cm_Cache_Backend_File for the FPC with the file app/etc/fpc.xml and the following content:

<?xml version="1.0"?>
<config>
    <global>
        <fpc>
            <lifetime>86400</lifetime>
            <backend>Cm_Cache_Backend_File</backend>
        </fpc>
    </global>
</config>

I am sure this makes sense, but it also does not solve the problem.

I know that the general solution to this problem seems to be Redis (or maybe alternatively Memcached) as a cache backend, but unfortunately, it is not available on our managed server. Switching to another hosting company is not (yet) an option.

I investigated a lot now, but I do not have any more idea. Maybe anyone else can help?

  • What's the URL of the site? Would be useful to be able to look at how the pages are loading. – Jonathan Hussey Aug 18 '15 at 16:38
  • @JonathanHussey sorry, can't share and as described, this is highly dependent on the cache status, so it would not really help anyway... – Simon Aug 18 '15 at 16:51
  • Sure ok, well without being able to see the issue, very hard to even speculate on what might be wrong. Being able to profile the HTML request would at least tell us if things were sticking on the FPC or not (i.e. still fast, or slow TTFB when there are problems). – Jonathan Hussey Aug 18 '15 at 16:53
  • Long known issue from 2011 fbrnc.net/blog/2011/05/magento-caching-internals – Fiasco Labs Aug 18 '15 at 20:05
  • @FiascoLabs I followed Fabrizio closely, but I cannot see that there is any solution (besides Redis). Can you? – Simon Aug 19 '15 at 7:11
7

I investigated even more and I think I finally solved the issue. So what can you do to analyse such an issue?

  1. In order to have a good idea when the cache becomes too large and if the problem is really the cache size, add a cronjob which calls the following script e.g. each 15 minutes:

    #!/bin/bash
    
    # this script is an attempt to check if the full cache is the real performance problem
    # it should be called regularly as a cronjob
    
    cache_dir="/html/shop/var/cache/"
    log_file="/html/cache_log"
    
    line=$(date)
    line="$line Size of cache directory: $(du -hs $cache_dir)"
    echo "$line" >> $log_file
    
    line=$(date)
    line="$line Total cache tags: $(find $cache_dir'cm-tags/' -type f | wc -l)"
    echo "$line" >> $log_file
    

    Then you can analyse the contents of /html/cache_log to see how the cache size develops, when your page becomes too slow and if the root cause is really the cache.

  2. Analyse your cache files. Therefore, it is quite helpful to write all cache files into a log file with e.g.:

    ls -R /html/shop/var/cache > /html/cachefiles
    

    Have a look at this file and the file names within. What kind of cache files are there? Is there anything suspicious? In my case, there were a whole lot of cache files containing AMSHOPBY in their file name - a reference to the Amasty Improved Navigation (Amasty_Shopby) extension. It was creating a whole lot of cache files. Some of them looked quite weird to me. Disabling the Amasty Improved Navigation cache solved the problem instantly. I contacted their support with a detailed description and the support was really good. The caching strategy was quickly completely overhauled and is now much better. They promised to integrate the patch in the next version of the extension, so every version > 2.8.3 should be fine.

Good luck in finding the root cause for your big cache!

4

Have you tried Cm_Cache_Backend_File as a backend in fpc.xml too? Maybe give that a try. I would give Aoe_Profiler a shot too. If you are able to reproduce the "slowdowns" on a staging-copy - go and profile your slow requests there. Otherwise you can do it on production (i strictly do not recommend it, but if you dare, you can configure the profiler to just be enabled when the GET parameter is set and go ahead)

  • No, I did not yet (did not know about fpc.xml). Interesting idea, will try, thanks! – Simon Aug 19 '15 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.