I'm in the process of developing a bespoke product import/sync module for Magento 2 and I'm trying to find ways to speed the process up as much as possible.
The full set of products contains over 800k entries, and that data will be processed at least once a day (more likely to be 2-3 times per day).
I've significantly improved the creating/updating of product entities and attributes by using
insertOnDuplicate, but using
StockRegistry::updateStockItemBySku seems to be taking up a huge portion of the overall time.
\Magento\Framework\Profiler the majority of the time taken while setting quantities is taken up with
Is there a practical way to disable cache invalidation/events for the duration of data processing, then invalidate the relevant caches in one batch afterward?
I suppose I could go the same route I have with product entities/attributes and just update/insert records, but without more investigation I'm not sure what other events/handling is taken into account when it comes to stock and availability of products.
Anyone have any ideas/tips or direction that might be helpful?