2

I'm in the process of developing a bespoke product import/sync module for Magento 2 and I'm trying to find ways to speed the process up as much as possible.

The full set of products contains over 800k entries, and that data will be processed at least once a day (more likely to be 2-3 times per day).

I've significantly improved the creating/updating of product entities and attributes by using ResourceConnection::insertMultiple/insertOnDuplicate, but using StockRegistry::updateStockItemBySku seems to be taking up a huge portion of the overall time.

According to \Magento\Framework\Profiler the majority of the time taken while setting quantities is taken up with EVENT:clean_cache_by_tags, specifically Magento\Framework\Cache\Frontend\Decorator\Profiler::clean

Is there a practical way to disable cache invalidation/events for the duration of data processing, then invalidate the relevant caches in one batch afterward?

I suppose I could go the same route I have with product entities/attributes and just update/insert records, but without more investigation I'm not sure what other events/handling is taken into account when it comes to stock and availability of products.

Anyone have any ideas/tips or direction that might be helpful?

  • Hi Danzapps, Do you get any solution for this? I have a cronjob which update the product attributes. My issue is when the products attributes getting updated, the debug.log file will get written by the "cache-invalidated" logs each time, this causing the debug.log file size becomes very large. Is there any way to block the "cache-invalidated" log printing in debug.log file. – amesh Dec 5 '17 at 8:15
  • Hi Amesh, I think your issue is a little bit different to mine. I was looking to disable the actual cache systems with intent to rebuild them after processing as the cache is important. I'm not sure how you can reduce the number of "cache-invalidated" messages, but you could configure a cronjob/script to to delete that file often, or, you could configure logrotate to rotate and compress that log file 2-3 times a day perhaps. – Danzapps Dec 7 '17 at 4:53
  • As a temporary solution i have overrided the file vendor/magento/framework/Cache/InvalidateLogger.php and commented out the log writing script in the execute function. – amesh Dec 7 '17 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.