I'm load testing a Magento site using Gatling. The site is hosted on 4 Amazon EC2 machines behind an application load balancer.

If all of my cron jobs are disabled, the site can take a good amount of traffic. But if I add cron.php to one server's crontab during load testing, as soon as that task kicks in, that server can't cope. The load average jumps up to around 40 and I start getting timeouts. Meanwhile, the load balancer continues to regard the machine as "healthy" because the helloworld.php script that I've set up as the health check, seems to be responding just fine the whole time.

The other 3 servers, where cron is not enabled, continue to run just fine.

In fact there's another php script which does the same thing. It's a quick script we wrote to look for orders stuck in "pending" status. Most times, there are no stuck orders, and the script is finished within one second... and yet this server still starts to fail to handle web traffic as soon as the cron job kicks in.

The website runs on php-fpm, the cron job uses php cli. So what on Earth could cause cron.php to interfere with the web traffic so much?

  • This might be obvious.. but have you considered the database as your bottleneck? and if it's not the database itself, then maybe the number of connections that can be handled at once. Both FPM and CLI will interface with the database and might get queued and end up waiting on each other.
    – Rian
    May 30, 2018 at 14:30
  • I've been looking at that and it doesn't seem to be the issue. Found that if I simply have a PHP script in cron that does nothing but Mage::app() it still does it... the server can handle the cron job fine and it can handle the load fine, but not both together. I'll try a script that connects to the database with native PHP rather than any Mage stuff, just to be sure. May 30, 2018 at 15:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.