We sync our products a lot with 3rd parties to update pricing and stock. We also have people editing products daily.
This will trigger indexers to run. The indexers can take 5 hours to run on category products and longer for pricing indexer.
This is unacceptable as it makes our database server maxed out during that entire time. When our database server is that busy it makes a normal 2s uncached load of a bundle item take 38 seconds. We lose customers over this constantly. We have a cache warmer, and cached pages are fine, but it's impossible to precache all pages in our store.
Magento2 is a constant source of pain and suffering. Is there really no fix to this? On top of this, it will cause products to appear out of stock after you get over so many items.
Getting stronger hardware barely makes a dent. We moved from 4GB RAM and 2CPU to 16GB and 4CPU, and yet our indexing is still almost as long. Note our database is a separate server so that is just for the database.
Other ecommerce stores don't use this indexers system and don't have this problem. This is a several year project but this might make us switch to another solution. It's just that terrible.
Is there no way to just remove them entirely or at least make them efficient? I looked up other people, having similar issues online and it's everywhere. People testing a new store with 1.4 million items say it takes their pricing indexer 2.5 hours. People with real stores with much less items taking even longer.
This is not any way we can run a business. Currently the indexers never finish before other edits are made to trigger them again.