0

So I'm having a serious issue with Magento which is ruining performance since 90+% of the time we don't have caches.

I'm running Magento 2.3.2 and Varnish and have had several issues with them.

  1. Magento sending insanely large headers and crashing Varnish constantly.
  2. Then Magento sent a fix to send these headers in batch but they're improperly cut so Varnish was still crashing.

We've managed to fix these issues ourselves but we've now found out where the large tags are coming from and it seems that one of our indexes in Magento is clearing the cache of every single product and category in our catalog. Is there any reason for Magento to clear the cache of all products/categories constantly, like every 2 minutes or so? Also, this happens even when we have no updates to products, categories or any other actions in the admin area.

We've had situations where this stopped happening when some indexes got 'stuck' and I believe the Product Categories index might be the origin.

Stuck indexes

Here's the backtrace caught before sending the tags to Varnish (I'd link the tag it but it's really large, it's literally every tag of every product and every category in our catalog):

[2019-08-21 09:25:13] report.WARNING: backtrace cache #0 /home/<user>/magento2_prod/vendor/magento/module-cache-invalidate/Model/PurgeCache.php(75): Magento\CacheInvalidate\Model\PurgeCache->splitTags('((^|,)cat_p_421...')
#1 /home/<user>/magento2_prod/vendor/magento/module-cache-invalidate/Observer/InvalidateVarnishObserver.php(65): Magento\CacheInvalidate\Model\PurgeCache->sendPurgeRequest('((^|,)cat_p_421...')
#2 /home/<user>/magento2_prod/vendor/magento/framework/Event/Invoker/InvokerDefault.php(88): Magento\CacheInvalidate\Observer\InvalidateVarnishObserver->execute(Object(Magento\Framework\Event\Observer))
#3 /home/<user>/magento2_prod/vendor/magento/framework/Event/Invoker/InvokerDefault.php(74): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod(Object(Magento\CacheInvalidate\Observer\InvalidateVarnishObserver), Object(Magento\Framework\Event\Observer))
#4 /home/<user>/magento2_prod/vendor/magento/module-staging/Model/Event/Manager.php(97): Magento\Framework\Event\Invoker\InvokerDefault->dispatch(Array, Object(Magento\Framework\Event\Observer))
#5 /home/<user>/magento2_prod/generated/code/Magento/Staging/Model/Event/Manager/Proxy.php(95): Magento\Staging\Model\Event\Manager->dispatch('clean_cache_by_...', Array)
#6 /home/<user>/magento2_prod/vendor/magento/module-target-rule/Model/ResourceModel/Rule.php(170): Magento\Staging\Model\Event\Manager\Proxy->dispatch('clean_cache_by_...', Array)
#7 /home/<user>/magento2_prod/vendor/magento/module-target-rule/Model/Indexer/TargetRule/AbstractAction.php(110): Magento\TargetRule\Model\ResourceModel\Rule->cleanCachedDataByProductIds(Array)
#8 /home/<user>/magento2_prod/vendor/magento/module-target-rule/Model/Indexer/TargetRule/Product/Rule/Action/Rows.php(34): Magento\TargetRule\Model\Indexer\TargetRule\AbstractAction->_reindexByProductIds(Array)
#9 /home/<user>/magento2_prod/vendor/magento/module-target-rule/Model/Indexer/TargetRule/Product/Rule.php(100): Magento\TargetRule\Model\Indexer\TargetRule\Product\Rule\Action\Rows->execute(Array)
#10 /home/<user>/magento2_prod/vendor/magento/framework/Mview/View.php(294): Magento\TargetRule\Model\Indexer\TargetRule\Product\Rule->execute(Array)
#11 /home/<user>/magento2_prod/vendor/magento/framework/Mview/Processor.php(44): Magento\Framework\Mview\View->update()
#12 /home/<user>/magento2_prod/vendor/magento/module-indexer/Model/Processor.php(104): Magento\Framework\Mview\Processor->update('indexer')
#13 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Indexer\Model\Processor->updateMview()
#14 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Indexer\Model\Processor\Interceptor->___callParent('updateMview', Array)
#15 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Indexer\Model\Processor\Interceptor->Magento\Framework\Interception\{closure}()
#16 /home/<user>/magento2_prod/generated/code/Magento/Indexer/Model/Processor/Interceptor.php(39): Magento\Indexer\Model\Processor\Interceptor->___callPlugins('updateMview', Array, Array)
#17 /home/<user>/magento2_prod/vendor/magento/module-indexer/Cron/UpdateMview.php(31): Magento\Indexer\Model\Processor\Interceptor->updateMview()
#18 [internal function]: Magento\Indexer\Cron\UpdateMview->execute(Object(Magento\Cron\Model\Schedule))
#19 /home/<user>/magento2_prod/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(317): call_user_func_array(Array, Array)
#20 /home/<user>/magento2_prod/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(737): Magento\Cron\Observer\ProcessCronQueueObserver->_runJob(1566379500, 1566379506, Array, Object(Magento\Cron\Model\Schedule), 'index')
#21 /home/<user>/magento2_prod/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(243): Magento\Cron\Observer\ProcessCronQueueObserver->processPendingJobs('index', Array, 1566379506)
#22 /home/<user>/magento2_prod/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(272): Magento\Cron\Observer\ProcessCronQueueObserver->Magento\Cron\Observer\{closure}('index')
#23 /home/<user>/magento2_prod/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(244): Magento\Cron\Observer\ProcessCronQueueObserver->lockGroup('index', Object(Closure))
#24 /home/<user>/magento2_prod/vendor/magento/framework/Event/Invoker/InvokerDefault.php(88): Magento\Cron\Observer\ProcessCronQueueObserver->execute(Object(Magento\Framework\Event\Observer))
#25 /home/<user>/magento2_prod/vendor/magento/framework/Event/Invoker/InvokerDefault.php(74): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod(Object(Magento\Cron\Observer\ProcessCronQueueObserver), Object(Magento\Framework\Event\Observer))
#26 /home/<user>/magento2_prod/vendor/magento/module-staging/Model/Event/Manager.php(97): Magento\Framework\Event\Invoker\InvokerDefault->dispatch(Array, Object(Magento\Framework\Event\Observer))
#27 /home/<user>/magento2_prod/generated/code/Magento/Staging/Model/Event/Manager/Proxy.php(95): Magento\Staging\Model\Event\Manager->dispatch('default', Array)
#28 /home/<user>/magento2_prod/vendor/magento/framework/App/Cron.php(86): Magento\Staging\Model\Event\Manager\Proxy->dispatch('default')
#29 /home/<user>/magento2_prod/vendor/magento/module-cron/Console/Command/CronCommand.php(117): Magento\Framework\App\Cron->launch()
#30 /home/<user>/magento2_prod/vendor/symfony/console/Command/Command.php(255): Magento\Cron\Console\Command\CronCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#31 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(58): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#32 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Cron\Console\Command\CronCommand\Interceptor->___callParent('run', Array)
#33 /home/<user>/magento2_prod/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Cron\Console\Command\CronCommand\Interceptor->Magento\Framework\Interception\{closure}(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#34 /home/<user>/magento2_prod/generated/code/Magento/Cron/Console/Command/CronCommand/Interceptor.php(26): Magento\Cron\Console\Command\CronCommand\Interceptor->___callPlugins('run', Array, Array)
#35 /home/<user>/magento2_prod/vendor/symfony/console/Application.php(893): Magento\Cron\Console\Command\CronCommand\Interceptor->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#36 /home/<user>/magento2_prod/vendor/symfony/console/Application.php(262): Symfony\Component\Console\Application->doRunCommand(Object(Magento\Cron\Console\Command\CronCommand\Interceptor), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#37 /home/<user>/magento2_prod/vendor/magento/framework/Console/Cli.php(105): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#38 /home/<user>/magento2_prod/vendor/symfony/console/Application.php(145): Magento\Framework\Console\Cli->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#39 /home/<user>/magento2_prod/bin/magento(23): Symfony\Component\Console\Application->run()
#40 {main} [] []
Improve this question
6
  • Welcome, do you have any custom extensions installed? if so have you checked that they are not executing the cache clear on reindex also have you checked that for any custom server side scripts which you have on a cron
    – Dava Gordon
    Commented Aug 21, 2019 at 10:14
  • @DavaGordon We have several extensions installed yes but we have three other environments with the exact same setup where this doesn't happen. We've thought of extensions before but the backtrace shows no reference to any extension or any other script, and so far we've found no relation to extensions we have. We're continuing to analyze but it's really weird considering our other environments are stable. We also have no custom server side script, and our crontab only has the Magento cron setup.
    – João Lisboa
    Commented Aug 21, 2019 at 10:26
  • have you tried moving this site to a local environment and checking to see if the issue is server related or magento related?
    – Dava Gordon
    Commented Aug 21, 2019 at 10:32
  • Yes, we have 3 other environments which are stable. We also have thought about it possibly being server related but haven't tracked down any reason behind it so far. We're currently trying to figure out in what cases Magento puts a product into an index to be updated to see if we can track down the origin. We're soon going to move to a new server for our production environment and we believe it might be fixed then but until then this has still been a serious issue that we really need to figure out.
    – João Lisboa
    Commented Aug 21, 2019 at 10:41
  • This information could be helpful regarding the indexes magento.stackexchange.com/questions/89579/… and at what stage they are invalidated from what you have said i would recommend you try a alternative server just to test to see if the issue persists if it does then you have you answer that it is a 100% a magento issue and we can help you more
    – Dava Gordon
    Commented Aug 21, 2019 at 10:45

1 Answer 1

Reset to default
0

So after a bit of debugging I found out it was a Related Product Rule in Magento by following the backtrace. The target-rule index, even if it was only updating 10 products, the rule matched all products for some reason.

The rule we had was for all products, and it was basically set to show 7 random related products where the matching rule was within the same category.

Improve this answer
5
  • Glad you figured it out. It's hard to debug an issue like this without acess sorry I couldn't be more help
    – Dava Gordon
    Commented Aug 21, 2019 at 19:54
  • @DavaGordon I understand it's hard. I was mainly looking for pointers for possible known issues or ways this could happen because we couldn't understand what was happening. Thanks for your help either way!
    – João Lisboa
    Commented Aug 22, 2019 at 17:41
  • Hi @JoãoLisboa Would you please let us know what did you do to solve this issue? We are constantly getting the target rule cache invalidate log and our product pages are constantly invalidating. Thanks in advance.
    – Pribhav
    Commented Jul 14, 2021 at 10:32
  • 1
    Hi @Pribhav We didn't solve this issue at the time. We have never used the related products rule again after this. Sorry I can't help further with that if that's your issue. I reported the issue directly to Magento/Adobe but they never even assumed there was an issue that a rule could match all products and invalidate all cache like that when at most it should invalidate 10 products.
    – João Lisboa
    Commented Jul 15, 2021 at 11:10
  • No problem @JoãoLisboa. Thank you for your response. :-)
    – Pribhav
    Commented Jul 15, 2021 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.