0

We sync our products a lot with 3rd parties to update pricing and stock. We also have people editing products daily.

This will trigger indexers to run. The indexers can take 5 hours to run on category products and longer for pricing indexer.

This is unacceptable as it makes our database server maxed out during that entire time. When our database server is that busy it makes a normal 2s uncached load of a bundle item take 38 seconds. We lose customers over this constantly. We have a cache warmer, and cached pages are fine, but it's impossible to precache all pages in our store.

Magento2 is a constant source of pain and suffering. Is there really no fix to this? On top of this, it will cause products to appear out of stock after you get over so many items.

Getting stronger hardware barely makes a dent. We moved from 4GB RAM and 2CPU to 16GB and 4CPU, and yet our indexing is still almost as long. Note our database is a separate server so that is just for the database.

Other ecommerce stores don't use this indexers system and don't have this problem. This is a several year project but this might make us switch to another solution. It's just that terrible.

Is there no way to just remove them entirely or at least make them efficient? I looked up other people, having similar issues online and it's everywhere. People testing a new store with 1.4 million items say it takes their pricing indexer 2.5 hours. People with real stores with much less items taking even longer.

This is not any way we can run a business. Currently the indexers never finish before other edits are made to trigger them again.

Improve this question

1 Answer 1

Reset to default
0

Here I observe two problems with your scenario:

Firstly, increase your configuration. The amount of items you have. Usually we have half of those items and even out staging server has more configuration than your above mentioned one.

Secondly, never run crons on Database server, nor on Frontend server. Seperate your Frontend, Backend, and Database server. This will ensure security as well. Since we don't have much Traffic and Backend/Admin Server, we should run indexer cron on that one.

I can assure you, this will solve your problem.

Thank you.

Improve this answer
1
  • This won't help because the database is already it's own server and is at several times capacity all the time. The problem is how inefficient indexing is and m2's design. Increasing complexity of configuration also tends to create as many problems as it solves. And you gave no clear reason as to why it was a solution. Complexity isn't a solution it's a problem to be constantly tamed and controlled. Which sever runs cron doesn't matter as they would all trigger the db running and that's the bottleneck.
    – user44406
    Oct 30, 2019 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.