Varnish cache_invalidate methods - tags explanation

Question

What is the difference between the these two cache_invalidate methods here?

[2022-07-07T07:49:19.265402+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_p_104358","cat_c_100","cat_c_2","cat_c_256","cat_c_500"],"mode":"matchingAnyTag"}} []

[2022-07-07T07:49:19.264726+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_c_500","FPC"],"mode":"matchingTag"}} []

Is FPC (Varnish) only invalidated on the second and is it the full FPC that are invalidated on the second or only cat_c_500.

If cat_c_2 is a top level category, will then all child categories also be invalidated?

why does this completely invalidate full Varnish cache storage:

[2022-07-07T08:42:11.549272+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_p_104366","cat_c_502","cat_c_701","cat_c_700","cat_c_2","cat_c_256","cat_c_500","amasty_xlanding_page_1","amasty_xlanding_page_2","amasty_xlanding_page_3","amasty_xlanding_page_4","amasty_xlanding_page_5","amasty_xlanding_page_6","amasty_xlanding_page_7","amasty_xlanding_page_9","amasty_xlanding_page_11","amasty_xlanding_page_12","amasty_xlanding_page_13","amasty_xlanding_page_14","amasty_xlanding_page_15","amasty_xlanding_page_16","amasty_xlanding_page_17","amasty_xlanding_page_18","amasty_xlanding_page_19"],"mode":"matchingAnyTag"}} []

From debug.log

[2022-07-07T08:42:11.549326+00:00] main.CRITICAL: Exception: only here for debugging in /home/babygear_dev/public_html/vendor/magento/framework/Cache/InvalidateLogger.php:50 Stack trace: #0 /home/babygear_dev/public_html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(60): Magento\Framework\Cache\InvalidateLogger->execute() #1 /home/babygear_dev/public_html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(48): Magento\Framework\Cache\Frontend\Decorator\Logger->log() #2 /home/babygear_dev/public_html/vendor/magento/framework/App/Cache.php(96): Magento\Framework\Cache\Frontend\Decorator\Logger->clean() #3 /home/babygear_dev/public_html/vendor/magento/framework/App/Cache/Proxy.php(103): Magento\Framework\App\Cache->clean() #4 /home/babygear_dev/public_html/vendor/magento/module-indexer/Model/Indexer/DeferredCacheCleaner.php(76): Magento\Framework\App\Cache\Proxy->clean() #5 /home/babygear_dev/public_html/vendor/magento/module-indexer/Model/Processor/CleanCache.php(50): Magento\Indexer\Model\Indexer\DeferredCacheCleaner->flush() #6 /home/babygear_dev/public_html/vendor/magento/framework/Interception/Interceptor.php(146): Magento\Indexer\Model\Processor\CleanCache->afterUpdateMview() #7 /home/babygear_dev/public_html/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Indexer\Model\Processor\Interceptor->Magento\Framework\Interception{closure}() #8 /home/babygear_dev/public_html/generated/code/Magento/Indexer/Model/Processor/Interceptor.php(41): Magento\Indexer\Model\Processor\Interceptor->___callPlugins() #9 /home/babygear_dev/public_html/vendor/magento/module-indexer/Cron/UpdateMview.php(31): Magento\Indexer\Model\Processor\Interceptor->updateMview() #10 [internal function]: Magento\Indexer\Cron\UpdateMview->execute() #11 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(368): call_user_func_array() #12 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(879): Magento\Cron\Observer\ProcessCronQueueObserver->_runJob() #13 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(840): Magento\Cron\Observer\ProcessCronQueueObserver->tryRunJob() #14 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(280): Magento\Cron\Observer\ProcessCronQueueObserver->processPendingJobs() #15 /home/babygear_dev/public_html/vendor/magento/framework/Event/Invoker/InvokerDefault.php(88): Magento\Cron\Observer\ProcessCronQueueObserver->execute() #16 /home/babygear_dev/public_html/vendor/magento/framework/Event/Invoker/InvokerDefault.php(74): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod() #17 /home/babygear_dev/public_html/vendor/magento/framework/Event/Manager.php(66): Magento\Framework\Event\Invoker\InvokerDefault->dispatch() #18 /home/babygear_dev/public_html/generated/code/Magento/Framework/Event/Manager/Proxy.php(95): Magento\Framework\Event\Manager->dispatch() #19 /home/babygear_dev/public_html/vendor/magento/framework/App/Cron.php(86): Magento\Framework\Event\Manager\Proxy->dispatch() #20 /home/babygear_dev/public_html/vendor/magento/module-cron/Console/Command/CronCommand.php(117): Magento\Framework\App\Cron->launch() #21 /home/babygear_dev/public_html/vendor/symfony/console/Command/Command.php(255): Magento\Cron\Console\Command\CronCommand->execute() #22 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(1009): Symfony\Component\Console\Command\Command->run() #23 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand() #24 /home/babygear_dev/public_html/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun() #25 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun() #26 /home/babygear_dev/public_html/bin/magento(23): Symfony\Component\Console\Application->run() #27 {main} [] [] [2022-07-07T08:42:11.549412+00:00] main.CRITICAL: #0 /home/babygear_dev/public_html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(60): Magento\Framework\Cache\InvalidateLogger->execute() #1 /home/babygear_dev/public_html/vendor/magento/framework/Cache/Frontend/Decorator/Logger.php(48): Magento\Framework\Cache\Frontend\Decorator\Logger->log() #2 /home/babygear_dev/public_html/vendor/magento/framework/App/Cache.php(96): Magento\Framework\Cache\Frontend\Decorator\Logger->clean() #3 /home/babygear_dev/public_html/vendor/magento/framework/App/Cache/Proxy.php(103): Magento\Framework\App\Cache->clean() #4 /home/babygear_dev/public_html/vendor/magento/module-indexer/Model/Indexer/DeferredCacheCleaner.php(76): Magento\Framework\App\Cache\Proxy->clean() #5 /home/babygear_dev/public_html/vendor/magento/module-indexer/Model/Processor/CleanCache.php(50): Magento\Indexer\Model\Indexer\DeferredCacheCleaner->flush() #6 /home/babygear_dev/public_html/vendor/magento/framework/Interception/Interceptor.php(146): Magento\Indexer\Model\Processor\CleanCache->afterUpdateMview() #7 /home/babygear_dev/public_html/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Indexer\Model\Processor\Interceptor->Magento\Framework\Interception{closure}() #8 /home/babygear_dev/public_html/generated/code/Magento/Indexer/Model/Processor/Interceptor.php(41): Magento\Indexer\Model\Processor\Interceptor->___callPlugins() #9 /home/babygear_dev/public_html/vendor/magento/module-indexer/Cron/UpdateMview.php(31): Magento\Indexer\Model\Processor\Interceptor->updateMview() #10 [internal function]: Magento\Indexer\Cron\UpdateMview->execute() #11 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(368): call_user_func_array() #12 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(879): Magento\Cron\Observer\ProcessCronQueueObserver->_runJob() #13 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(840): Magento\Cron\Observer\ProcessCronQueueObserver->tryRunJob() #14 /home/babygear_dev/public_html/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php(280): Magento\Cron\Observer\ProcessCronQueueObserver->processPendingJobs() #15 /home/babygear_dev/public_html/vendor/magento/framework/Event/Invoker/InvokerDefault.php(88): Magento\Cron\Observer\ProcessCronQueueObserver->execute() #16 /home/babygear_dev/public_html/vendor/magento/framework/Event/Invoker/InvokerDefault.php(74): Magento\Framework\Event\Invoker\InvokerDefault->_callObserverMethod() #17 /home/babygear_dev/public_html/vendor/magento/framework/Event/Manager.php(66): Magento\Framework\Event\Invoker\InvokerDefault->dispatch() #18 /home/babygear_dev/public_html/generated/code/Magento/Framework/Event/Manager/Proxy.php(95): Magento\Framework\Event\Manager->dispatch() #19 /home/babygear_dev/public_html/vendor/magento/framework/App/Cron.php(86): Magento\Framework\Event\Manager\Proxy->dispatch() #20 /home/babygear_dev/public_html/vendor/magento/module-cron/Console/Command/CronCommand.php(117): Magento\Framework\App\Cron->launch() #21 /home/babygear_dev/public_html/vendor/symfony/console/Command/Command.php(255): Magento\Cron\Console\Command\CronCommand->execute() #22 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(1009): Symfony\Component\Console\Command\Command->run() #23 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand() #24 /home/babygear_dev/public_html/vendor/magento/framework/Console/Cli.php(115): Symfony\Component\Console\Application->doRun() #25 /home/babygear_dev/public_html/vendor/symfony/console/Application.php(149): Magento\Framework\Console\Cli->doRun() #26 /home/babygear_dev/public_html/bin/magento(23): Symfony\Component\Console\Application->run() #27 {main} [] []

Answer 1

10 months ago but might just put a few lines in anyway for whoever comes next

Tags are attached to pages, context and blocks. This is set by Magento but also by modules, and purge requests sent by actions that change the content.

I use a different cache but some of it is general the tag matching is done by Magento and Magento passes object IDs to varnish to invalidate.

So if something odd happens it's either Magento sending a bigger (or smaller) list than you expect, or a different issue on the varnish side. I suspect a tag is applying to more than you think.

[2022-07-07T07:49:19.265402+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_p_104358","cat_c_100","cat_c_2","cat_c_256","cat_c_500"],"mode":"matchingAnyTag"}}

Any item in the cache with ANY of these tags attached to it will get cleared. This could be blocks as well as pages, and perhaps collections?

My guess is here you changed a product that is on all these categories so all blocks, collections and pages linked to all of these need to be refreshed.

[2022-07-07T07:49:19.264726+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_c_500","FPC"],"mode":"matchingTag"}}

Any item in the cache with ALL of these tags attached to it will get cleared so I guess it would only clear FPC content with the tag.

My guess is here you edited the category itself and only its page needs to be changed. This would not even refresh widgets of this category

why does this completely invalidate full Varnish cache storage:

[2022-07-07T08:42:11.549272+00:00] main.DEBUG: cache_invalidate: {"method":"GET","url":"http:/","invalidateInfo":{"tags":["cat_p_104366","cat_c_502","cat_c_701","cat_c_700","cat_c_2","cat_c_256","cat_c_500","amasty_xlanding_page_1","amasty_xlanding_page_2","amasty_xlanding_page_3","amasty_xlanding_page_4","amasty_xlanding_page_5","amasty_xlanding_page_6","amasty_xlanding_page_7","amasty_xlanding_page_9","amasty_xlanding_page_11","amasty_xlanding_page_12","amasty_xlanding_page_13","amasty_xlanding_page_14","amasty_xlanding_page_15","amasty_xlanding_page_16","amasty_xlanding_page_17","amasty_xlanding_page_18","amasty_xlanding_page_19"],"mode":"matchingAnyTag"}}

On paper this looks like a product and listing pages that product was on and only they should be invalided. If all of it invalidates then one of these happens:

• one of these tags is attached to a lot more content than we think, maybe because a widget of a landing page is added in a sidebar and so on every page and if that block gets purged then every page will? Modules can be very stupid about tags. You should be able to turn on some cache tag debug in logs or headers to track this. My FPC has a magento admin module with options to ignore tags from specific blocks because of this.
• something else happens from this action that purges a lot more (example: reindexing)
• something else (example: magento has a full cache clear in the crontab scheduled for midnight by default?)