Speed : Magento with both APC and Memcached

Question

We have studied many forums and don't know the answer to the following. We have both APC and Memcache installed on our servers. We are not sure what the correct and best config is.

My question

What are/is the best settings for running Magento using both Memcache + APC at the same time? (Or is this not smart at all)

Background research

Here Memcache and APC are advised as fast and slow cache (but no disk). Sounds like this only works when you have enough RAM (and sure about it)

• http://www.coeusblue.com/blog/48-magento/65-magento-caching

And this article is about Memcache or APC - and we have both

• http://magebase.com/magento-tutorials/speeding-up-magento-with-apc-or-memcached/

And here it states that Memcache only really works when you also have a slow backend defined

• http://www.magentocommerce.com/boards/viewthread/283908/#t393090

And I think this article is saying the same

• http://www.byte.nl/blog/speeding-up-magento-the-burden-of-two-level-cache/

This is my ISP's solution for local.xml

<cache>
  <backend>apc</backend>
  <prefix>sitenamehere__</prefix>
</cache>
<cache>
  <backend>memcached</backend>
  <memcached>
    <servers>
      <server>
        <host><![CDATA[127.0.0.1]]></host>
        <port><![CDATA[11211]]></port>
        <persistent><![CDATA[1]]></persistent>
      </server>
    </servers>
    <compression><![CDATA[0]]></compression>
    <cache_dir><![CDATA[]]></cache_dir>
    <hashed_directory_level><![CDATA[]]></hashed_directory_level>
    <hashed_directory_umask><![CDATA[]]></hashed_directory_umask>
    <file_name_prefix><![CDATA[]]></file_name_prefix>
  </memcached>
</cache>

Situation

Shared hosting Brim FPC installed: http://ecommerce.brimllc.com/full-page-cache-magento.html (this FPC also has a scaleable file cache to make it more complex)

Answer 1

You need to understand the clear distinction between these two products to understand how to use them.

• APC is both an OPCode Cache and Fast Backend
• Memcache is just a Fast Backend

Using APC as an OPCode Cache

Simply install the module on your server

pecl install apc

And enable it in your php.ini

echo "extension=apc.so" >> /usr/lib/local/php.ini       (RedHat/Centos)
echo "extension=apc.so" >> /etc/php5/conf.d/20apc.ini   (Debian)

You then enable and fine-tune the runtime configuration to suit, eg.

apc.enabled
apc.shm_segments
apc.shm_size
apc.optimization
apc.num_files_hint
apc.user_entries_hint
apc.ttl
apc.user_ttl
...

Then restart PHP/Apache

/etc/init.d/httpd restart                               (RedHat/Centos)
/etc/init.d/apache2 restart                             (Debian)

After that, there is nothing else to do. Confirm APC is enabled with a quick phpinfo() - but otherwise, at this point, the OPCode cache portion of APC is active.

Nothing needs to be configured on Magento's side.

Using APC as a Fast Backend

You need to add the following to your ./app/etc/local.xml

<global>
  ...
  <cache>
    <backend>apc</backend>
      <prefix>mystore_</prefix>
  </cache>
  ...
</global>

Then flush your existing store caches. To verify it is working, load a page in the front-end and the ./var/cache directory should remain empty.

Using Memcache as a Fast Backend

You'll need to install Memcache as a PHP extension, and install the respective Memcache Daemon (Memcached) on your server.

pecl install memcache

And enable it in your php.ini

echo "extension=memcache.so" >> /usr/lib/local/php.ini            (RedHat/Centos)
echo "extension=memcache.so" >> /etc/php5/conf.d/20memcache.ini   (Debian)

/etc/init.d/httpd restart                               (RedHat/Centos)
/etc/init.d/apache2 restart                             (Debian)

Then install Memcached on the server. For RH/Centos, adjust the URL to suit your release version and CPU architecture.

rpm -Uhv http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
yum --enablerepo=rpmforge install memcached

apt-get install memcached                               (Debian)

Then modify Magento to use Memcache as a fast backend, change the socket path to a TCP/IP connection to suit.

<cache>
  <slow_backend>database</slow_backend>
  
  <fast_backend>memcached</fast_backend>
  <fast_backend_options>
    <servers>
      <server>
        <host>unix:///tmp/memcached.sock</host>
        <port>0</port>
        <persistent>0</persistent>
      </server>
    </servers>
  </fast_backend_options>
  
  <backend>memcached</backend>
  <memcached>
  <servers>
    <server>
      <host>unix:///tmp/memcached.sock</host>
      <port>0</port>
      <persistent>0</persistent>
    </server>
  </servers>
</cache>

The caveats of Memcache and tagging - what is it storing

Memcache only supports a single level of key-value relationships, so it cannot store the Magento cache tags (that are used to flush cache data independently). As a result, you either need to specify a slow_backend to maintain the cache content tag relationship, or don't define one at all.

If you define a slow_backend, you run the risk of the cache tags growing so large that performance is negated; there is also the inherent problem that you cannot scale across multiple servers if each server is maintaining their own cache tags.

So when using Memcache, the better approach (with the caveat you cannot flush caches independently), is to not bother using the slow_backend.

In which case, we suggest removing <slow_backend>database</slow_backend> and replacing it with:

  <slow_backend>Memcached</slow_backend>
  <slow_backend_options>
    <servers>
      <server>
        <host>unix:///tmp/memcached.sock</host>
        <port>0</port>
        <persistent>0</persistent>
      </server>
    </servers>
  </slow_backend_options>

This will break/disable the 2nd level of caching (and prevent tag storage), but still allow the performance of Memcache.

Which to use

If its a single server deployment - there's no harm just using APC for everything.

If its a distributed set-up - then you'll need to use Memcache as the fast backend (so that all machines can access the common store).

More concerning is that if your hosting provider can't tell you the right set up to use, you are certainly with the wrong host.

---

Attributions: sonassi.com, php.net, repoforge.org

Answer 2

I quite agree with previous answers, but here's a short precision to complete it: Yes, apc can be used as both a cache storage engine and as a PHP byte code optimiser. But two points need to be clarified:

• As a fast backend, the configuration directives used by APC to understand how it must save data are managed through the apc.user_% directives. The other ones concern only the byte code cache (Ex apc.ttl: the expiration duration for the opcode cache, apc.user_ttl: the expiration duration for the data stored in cached by your Magento).

• And as a fast backend, APC has exactly the same behaviour than memcached: it does not manage the cache tags, and for Magento it requires a configured slow backend (or use by default the slow backend file).

From my experience, on websites with huge traffic, if you use apc as byte code optimizer only, you require between 96 and 256Mo in apc.shm_size configuration value. Also increase the apc.num_files_hint from 1000 to 15000: by default the apc cache byte code cache only 1000 files and Magento contains around ~20,000 PHP and PHTML files by default (find . -type f -name "*.php" -o -name "*.phtml" | wc -l). So customize this value with your source code.

If you use APC or memcached as fast backend, it's difficult to provide you some tips on the required memory: it really depends on the cache policy applied on your instance.

For now, your cache configuration works like this:

• every content is stored both in memcached and in file
• The fast backend is always requested prior to slow backend
• if nothing is found in fast backend, magento looks for in the slow backend

Why these two levels management? memcached and other fast backends are memory storages. So it means that data can be corrupted or have disappeared.

How can you increase this configuration performance?

Disable the second writing is probably one of the most efficient options. This is explained in the fourth article you mentioned. But your are not able to use without modifications the slow_backend_store_data source code. In your context, I do not recommend to make this customization for the following reasons: your data stored in cache will never be controlled. You'll store data in memory, will earn performance, but perhaps send to your vistors an invalid content. So you need to find a solution that ensures you to have memory acces (for better performance), write control, and the ability to disable slow_backend_store_data caching. You can reach this context by:

• replace the memcached server by a redis one (redis can control read and write like it is done by a filesystem), and continue to use apc as a byte code optimizer

• *ensure that you are able to use slow_backend_store_data option* either by customizing your source code or either by switching to a database slow backend (yes it increases the load of your database server, but if your cache policy is well defined, it should not be a problem)

• *deactivate slow_backend_store_data option*: in this configuration it's not required anymore, you have read and write control done by redis.

Answer 3

As an additional note to this, we've found that when using APC with Magento (for opcode caching - we use Redis for conventional Magento page and block caching) then it's important to ensure that the stat setting is 0 in production (but 1 in development):

apc.stat = 0

The apc.stat setting is used to determine whether to check a script on each request to determine whether it has been modified (http://www.php.net/manual/en/apc.configuration.php#ini.apc.stat) and so setting this to 0 in a production environment will bring the performance benefit of APC not doing this check with each request.

Worth noting that once apc.stat is set to 0 you will likely have to restart your web server process to pick up file changes (i.e. post-deployment) but this should be part of your post-deployment strategy anyway.

Answer 4

The best thing we did to speed up the backend significantly is to install REDIS as a cache handler. It is now also supported in core from Magento 1.8 and up.

Nothing compares ... now it is click click clickerdy click

http://www.magentocommerce.com/knowledge-base/entry/redis-magento-ce-ee

In addition you could consider adding Redis Session extension to also add sessions to the redis memory server ...

Good luck!

Answer 5

From this local.xml file, Magento will pick-up the last entry and use Memcache. I think there is a confusion between how APC and Memcache can work with Magento.

First, APC has 2 uses:

• opcode caching - compile your php files into opcode, making script execution roughly 25% faster
• key/value storage - can be used by Magento as a cache system.

Memcache on the other hand is just a key/value store. The big advantage of Memcache is that it can work in client-server mode, so multiple frontend servers can use the same cache, which is a must if you have multiple servers that serve the same website.

The most common setup is to install APC to get opcode caching (so you get ~25% faster script execution) and use Memcache as a cache server. I also used APC as a cache system and although in theory it should be a little bit faster than Memcache, you cannot tell the difference.