CAUSE:
The line in question returning the exception in CRedis:
NOTE the timeout causing because of the read meta data of the redis process stream. More than likely you've hit timeout & or memory limits possibly.
More details on stream_get_meta_data:
FIRST STEPS:
First make sure you are using the latest versions of Collin's modules for Redis with Magento.
- https://github.com/colinmollenhour/Cm_Cache_Backend_Redis
- https://github.com/colinmollenhour/Cm_RedisSession
One thing to note in Cm_RedisSession is:
A process may lose it's lock if another process breaks it, in which case the session will not be written.
Also setup a garbage collection cron script noted here:
Be sure to review local.xml configurations, and Redis's configurations, particularly timeouts and max limits. Compare with any log data and known hardware limits of the server.
More than likely with such a large set of Customer Groups and 2 websites, the amount of required data to be indexed is going to be quite large as you mentioned. Most likely the indexing process is timing out or hitting a max limit causing the indexing to fail, and with such a long process occurring in Redis for the current session a read timeout occurs.
Ultimately the best solution is to leverage Asynchronous indexing, Class path caching, disable logging tables, and disable Core Magento modules not required. With these in place monitoring the processes closely with the other tools mentioned below to help pinpoint the location of the limits being hit.
PERFORMANCE:
Disable logging and any unused core modules you don't use:
Some modules to consider when needing more insight into the indexing/caching mechanisms:
Improve Index management in Magento. Enables Magento to process indexes asynchronously. I would highly recommend giving this a try as it will chunk up the data needing to be indexed and spawned across separate PHP processes. By Default Magento only uses one CPU Core when indexing. On a large multi-core system this helps tremendously with indexing performance.
Another free option available on GitHub that looks to be more up to date:
There is a paid for solution as well, as Aoe_Index hasn't been updated in over 4 years, and claims to be EE 1.14+ compatible:
With as many files/folders that Magento has, this helps the Autoloader find the exact files its looking for instead of fseek's on your servers file system to locate the files. Not a direct solution to your problem but will greatly speed up Magento's performance.
ANALYSIS:
Utility for managing Redis services directly in Magento's admin:
This module adds some statistics to System -> Index Management for the changelog based indexers introduced in Magento Enterprise Edition 1.13+
Recommended Profiler over Magento's built-in one:
FURTHER READING: