I have a series of stock importers, all of which run of mostly the same code, each with a few tweaks, with regards to how they fetch their stock data from different suppliers.
These all run off cron, however, I'm running into issues with one of them - it fetches its data by API which is rate limited to 1 req/s, because of this the data fetching takes about 15 minutes. After this, it attempts to get the size of a previously fetched product collection, for logging purposes. At this point the process crashes out with a 2006 MySQL server has gone away
error.
I've tried increasing various timeouts and changing some MySQL settings to no avail, e.g.
ini_set('mysql.connect_timeout', 14400);
ini_set('default_socket_timeout', 14400);
ini_set('mysql.allow_persistent', 1);
I've been talking to the server hosting company who insist it's a code issue, but it still feels like some form of timeout.
What settings should I be looking into? It always crashes out at the same point if data is fully fetched - if I modify the script to test, e.g. by using dummy data instead of taking the time to fetch it from the supplier, no SQL error occurs.