XML configuration getting cached somewhere

Question

TL;DR

Moving an XML configuration block from one module to another seems that the old one is cached somewhere (not in the Magento cache silly!!).

How and why?

---

I've just encountered a strange problem while testing our changes for the SUPEE 6788 patch. Here's the scenario:

We used to have a local module which was old and had a bunch of dodgy stuff in it. We also have another local module which is for the same area but newer and safe.

While fixing routes and doing all that fun stuff, I moved a Product_List block from Namespace_OldModule to Namespace_NewModule, and took the rewrite configuration from the old config.xml to the new one.

No problems until we deployed this to test in an environment using Redis for cache and Akamai for frontend CDN - the category lists were all blank (no products) and a PHP error was in the source code:

Fatal error: Call to a member function getLoadedProductCollection() on a non-object in /.../app/code/core/Enterprise/GoogleAnalyticsUniversal/Block/List/Json.php on line 130

Debugging that method, the method $this->getListBlock() returns bool false. I then noticed that an exception in the logs said the following:

2015-11-09T20:27:20+00:00 ERR (3): 
exception 'Mage_Core_Exception' with message 'Invalid block type: Namespace_OldModule_Block_Product_List' in /.../app/Mage.php:595
Stack trace:
#0 /.../app/code/core/Mage/Core/Model/Layout.php(495): Mage::throwException('Invalid block t...')
#1 /.../app/code/core/Mage/Core/Model/Layout.php(437): Mage_Core_Model_Layout->_getBlockInstance('catalog/product...', Array)
#2 /.../app/code/core/Mage/Core/Model/Layout.php(472): Mage_Core_Model_Layout->createBlock('catalog/product...', 'product_list')
#3 /.../app/code/core/Mage/Core/Model/Layout.php(239): Mage_Core_Model_Layout->addBlock('catalog/product...', 'product_list')

So I checked the changes to the XML, and this component:

<global>
    <blocks>
        <catalog>
            <rewrite>
                <product_list>Namespace_NewModule_Block_Product_List</product_list>
            </rewrite>
        </catalog>
    </blocks>
</global>

... had been moved from Namespace/OldModule/etc/config.xml to Namespace/NewModule/etc/config.xml - the only difference being the contents of the tag where the module name changed.

This must have been cached somewhere, because if I add a shortcut class back in place of where it's looking to extend the new one - no problem. The old module isn't referenced anywhere, I've checked. Also the cache has been cleared many times, php-fpm and nginx restarted (hard restart, not just gracefully) and cache busting etc.

# File: app/code/local/Namespace/OldModule/Block/Product/List.php
<?php
class Namespace_OldModule_Block_Product_List extends Namespace_NewModule_Block_Product_List
{
}

Can anyone explain to me why this might be happening?

Answer 1

Magento will always cache some of the XML regardless of cache configuration settings being set to Disabled. Magento will read all of the XML files and combine them into one big XML tree it will store for subsequent loads. Localization, Currency and Configuration data is what is always cached regardless. You can verify this by setting your cache storage to use file and viewing the contents of the files in var/cache.

• https://github.com/OpenMage/magento-mirror/blob/magento-1.9/app/code/core/Mage/Core/Model/App.php#L272-L278

An easy way to see the combined XML tree is to add this to your index.php for example before the Mage::run($mageRunCode, $mageRunType);. You can use it else where but requires no Headers being sent before hand.

header("Content-Type: text/xml");
die(Mage::app()->getConfig()->getNode()->asXML());

This way you can see what values Magento has stored and is using currently, and should be read from any cache data storage, such as Redis.

You can also login to Redis via redis-cli and perform a flushdb or flushall to ensure all cache data in Redis has been purged. The module notes itself also recommend to setup a general garbage clean up of Redis nightly to keep references in tact.

Occasional (e.g. once a day) garbage collection is recommended if the entire cache is infrequently cleared and automatic cleaning is not enabled. The best solution is to run a cron job which does the garbage collection. (See "Example Garbage Collection Script" below.)

• https://github.com/colinmollenhour/Cm_Cache_Backend_Redis

I'm including the Example script for reference, as it could be called via CLI instead of logging into redis-cli and added to a cron task to perform during off peak hours (be sure and uncomment the line for Enterprise if you are using Redis for your FPC cache storage in enterprise.xml):

<?php PHP_SAPI == 'cli' or die('<h1>:P</h1>');
ini_set('memory_limit','1024M');
set_time_limit(0);
error_reporting(E_ALL | E_STRICT);
require_once 'app/Mage.php';
Mage::app()->getCache()->getBackend()->clean('old');
// uncomment this for Magento Enterprise Edition
// Enterprise_PageCache_Model_Cache::getCacheInstance()->getFrontend()->getBackend()->clean('old');

Most likely the cause is stale references in Redis causing the XML not to be invalidated, and still loading the old reference instead.

You could confirm this further with removing Redis out of the equation, via local.xml and performing the same scenario.

Some other notes in regards to automatic cleaning (which isn't recommended) and stating that data eviction may be invalid if memory limits are reached.

• The recommended "maxmemory-policy" is "volatile-lru". All tag metadata is non-volatile so it is recommended to use key expirations unless non-volatile keys are absolutely necessary so that tag data cannot get evicted. So, be sure that the "maxmemory" is high enough to accommodate all of the tag data and non-volatile data with enough room left for the volatile key data as well.
• Automatic cleaning is optional and not recommended since it is slow and uses lots of memory.