How to flush/rebuild config cache on High-Performance sites?

Question

If we flush the config cache, the cache is build up by the customer in the frontend. The problem is that a lot of customer build the same cache and it took a lot of server resources.

Is there a smart way to build the config cache?
Something like, build a parallel config cache and switch it with the real cache keys?
Or just rebuild every config cache key in the adminhtml without flushing it?

We are using Redis as backend cache.

Edit: I figured out, that it's just the case if the cache key CONFIG_GLOBAL_STORES_[A-Z]_[A-Z] is flushed.

Answer 1

We figured out that magento set a cache key prefix based on the path of the magento etc folder. We have a multiple server and each server has a release folder structure for deployment. That means the path of the etc directory is different on every server, so every frontendserver has/build his own config/block cache. The solution was to set a hardcoded prefix in the local.xml (global/cache/prefix) and now the cache flush is very smooth :)

Answer 2

My recommendation is to use something like the following: https://github.com/ctidigital/Configuration-Cache-Lock.

The approach this extension takes is to add a cache_lock key to redis that every process with a cold cache checks before generating the cache themselves. If the cache_lock key already exists then the process sleeps for a while before trying again.

The main problem this avoids is multiple processes each generating their own version of the cache, causing a massive spike in CPU utilisation on cache clears.

There are a number of alterations that we made internally to:

• Add an expiry time to the cache_lock key, incase the process generating the cache crashes for any reason by adding an EX to the set logic
• Sending NX along with the SET to ensure that the key is only set iff the key doesn't already exist during the aquireLock and only returning a true if it was us that obtained the lock to avoid absolutely any duplication in cache generation.