3

Recently faced a problem of customer saving being too slow in production environment, especially while doing it as a mass-action. Production system has ~200 simultaneous users at the same time.

After some investigation on a mirror system found out the following:

  1. I should switch indexes into "update on schedule" mode.
  2. Looks like the save action of customer is slow (1-2 seconds)
  3. Half of that save time is spend on clearing caches after save, see the code snippet below

    // app/code/core/Mage/Core/Model/Abstract.php
    /**
     * Remove model onject related cache
     *
     * @return Mage_Core_Model_Abstract
     */
    public function cleanModelCache()
    {
        $tags = $this->getCacheTags();
        if ($tags !== false) {
            Mage::app()->cleanCache($tags);
        }
        return $this;
    }
    
    /**
     * Processing object after save data
     *
     * @return Mage_Core_Model_Abstract
     */
    protected function _afterSave()
    {
        $this->cleanModelCache();
        Mage::dispatchEvent('model_save_after', array('object'=>$this));
        Mage::dispatchEvent($this->_eventPrefix.'_save_after', $this->_getEventData());
        return $this;
    }
    

On live system that becomes even worse because cache cleanup on a system with quite some live users takes a lot of time, and actually this time is wasted.

So far I see the only solution is to disable the cache, perform customer mass-action and then enable the cache again. Though that sounds little insane and here comes the question:

Is there a way to speed up cache clean up or configure magento better to optimize model saving in production environment?

UPDATE: We are talking about Magento EE v.1.14.2.4, I believe the corresponding Magento CE have the same problem, I haven't checked though.

UPDATE2: Continued playing around and found out the following: saving 885 customers as a mass-action takes:

  1. with default magento cache - ~7 minutes
  2. with default cache completely disabled - 4.3 minutes
  3. with redis session, cache and fpc configured according to magento examples - 49 seconds

that doesn't solve the issue though, cause if the time proportion still stands we could shorten those 49 seconds to be 25-30 seconds.

see also this SO topic

UPDATE3: all the story is happening on the following machine

Intel® Xeon® E3-1275 v5 Quad-Core Skylake (12 cores visible to OS)
64 GB DDR4 ECC RAM
2 x 512 GB NVMe PCIe Gen3 x4 SSD (Software-RAID 1)
running Debian Wheezy with nginx 1.10 and PHP 5.5.33 via php-fpm
  • disable one by one custom module and check which 3rd part extension responsible for this issue – Amit Bera May 17 '16 at 9:45
  • We are talking about magento core problem - see code snippet in the question, i.e. Magento cleans the model cache everytime customer model is being saved. – Anton Boritskiy May 17 '16 at 9:47
1

This is quite a common issue on the production system in B2B environments. The only way to overcome the issue is to:

  1. Remove cache clean up by cache tags, if you don't use any customer based cache tags on the frontend.
  2. Move cache clean up into background, as any cleaning it in separate thread for each connection poses a bottleneck in highly concurrent environment. To make it possible you need to:
    • Rewrite cleanModelCache() method of a customer model.
    • Send a message to a queue or a custom storage which realizes pseudo-queue
    • Create subscriber for this queue, that will invoke logic of original cleanModelCache() by cron in background
  • Well, this is a valid answer to the question in the way it was asked :) No need to explain why I don't like changing core code via direct patching or rewriting classes I guess ))) At the same Magento code of this feature is written in a way that there is no other option left to fix the problem. – Anton Boritskiy Jun 20 '16 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.