4

I have completed the product import process of 60 Million products in Magento 2. But products are not visible on the front-end, while debugging this I found that re-indexing of products is still in progress.

I have tried to optimize the re-indexing so that all the products could be visible on the front-end.

While working on this re-indexing part I have analyzed the following things and now I am confused should I go ahead with Magento or not.

  1. Default re-indexing with 16 core cpu and 32gb ram is too much slow, calculated assumption for re-indexing all category is more than 2 years.
  2. If anyhow I do manage the first time re-indexing by increasing the server size again there will be pain on every product insert or update because Magento do the re-indexing from starting everytime.

I am not sure how the other e-commerces use the magento for millions for products.

Thanks,

1
  • You have to scall up your hosting – Amit Bera Feb 18 at 12:38
1

Congratulations - I think you hold the world record for the most products in a Magento 2 catalog.

Whilst theoretically there is no limit to catalog size there will be performance issues related to such a large catalog especially with database queries.

If there is no way to reduce your catalog size then I think you need to review your hosting and database performance to try and find out where the performance issues are occurring when the indexer is running.

Perhaps getting in touch with a good Magento 2 solutions partner would be a good idea.

enter image description here

1
  • The certificate is a nice touch – Dominic Xigen Feb 18 at 18:57
0

Are you amazon or ebay ?

If you have 60 million product 32gb ram would not be enough, cpu and other resources as well.

0

Have you seen this?

https://devdocs.magento.com/guides/v2.3/extension-dev-guide/indexer-batch.html

I heard this might help, but I haven't tried it myself.

3
  • Thanks for the help i have seen that and implemented all the things but still the process is slow. – Prits Mar 11 at 11:44
  • Magento 2 just doesn't handle that many products, as you may have guessed from the ironic reactions of other members. If you have the choice, look around for other platforms and check for this requirement. – Patrick van Bergen Mar 12 at 14:09
  • An other thing is that perhaps you have many many products that are just variations of a much smaller number of products. If that is the case, you may consider having just these basic products as real products, and use custom code on the store front to make it look like there are many more products. See: reduce the number of products github.com/bigbridge-nl/product-import/blob/master/doc/tips.md I don't have time to explain the techniques involved. – Patrick van Bergen Mar 12 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.