I have a series of stock importers, all of which run of mostly the same code, each with a few tweaks, with regards to how they fetch their stock data from different suppliers.
These all run off cron, however, I'm running into issues with one of them - it fetches its data by API which is rate limited to 1 req/s, because of this the data fetching takes about 15 minutes. After this, it attempts to get the size of a previously fetched product collection, for logging purposes. At this point the process crashes out with a
2006 MySQL server has gone away error.
I've tried increasing various timeouts and changing some MySQL settings to no avail, e.g.
I've been talking to the server hosting company who insist it's a code issue, but it still feels like some form of timeout.
What settings should I be looking into? It always crashes out at the same point if data is fully fetched - if I modify the script to test, e.g. by using dummy data instead of taking the time to fetch it from the supplier, no SQL error occurs.