3

Do you have some suggestion how can I improve the performance of the catalogsearch_fulltext indexer for Magento 2.1.X

I already set this indexer to 'Update By Schedule' so only the updated products are refreshed thanks to Mview pattern.

I have ElasticSearch as search engine, but it is not really related to it.

I have around

  • 20k products
  • 7k products visible in search by store
  • 31 stores
  • less than 10 new custom searchable attributes

For a full re-index, Magento took around 10 minutes by stores, so around 5 hours for all stores.

I plan to re-index each store in parallel and increase the server CPU and memory because each catalog search index seems independent by stores.

I try to import products with a differential approach, but sometime my customer needs a full product import.

Edit: I did a POC thanks to https://github.com/amphp/parallel reducing from 5 hours to 14 min using the c5.9xlarge EC2 instance with 36vCPU. But I want to know if there is alternative solution. (>1000$ per month)

  • "For a full re-index, Magento took around 10 minutes by stores, so around 5 hours for all stores." -> 10m * 31 = 2,5 hours in my math – simonthesorcerer Jul 4 '18 at 11:25
  • 3
    10 * 31 = 310 minutes. 310 / 60 = 5.1 hours. Can you please try to be constructive ? – Franck Garnier Jul 4 '18 at 15:01
  • Ok, can you post you server hardware & possibly php config such as memory_limit etc.? – simonthesorcerer Jul 4 '18 at 15:24
  • The server hardware can be increase. It is on a dedicated AWS EC2 m5.large for cron. But the timing is similar on my local environment. I'm trying to implement github.com/amphp/parallel to reindex stores in parallel. – Franck Garnier Jul 4 '18 at 15:37
  • 1
    @SavvasRadevic Yes it is the best approach, but it is not simple to integrate it to the native Magento reindex flow. There are many differencial reindex after an order or after a little product import. And they are a full import maybe 1 per week which took 5 hours. I think I'll try to implement your solution for the big import. But I will need to recreate the whole logic :S – Franck Garnier Jul 19 '18 at 8:13
2
+100

Try to upgrade to the latest version (2.2.5). As I can see in the changelog (https://devdocs.magento.com/guides/v2.2/release-notes/ReleaseNotes2.2.5CE.html#indexing) they optimized the search indexer in order to be multithreaded. Try to upgrade to 2.2.5 your local environment and test if it's worth to upgrade to 2.2.x because we know that it's not simple to upgrade from 2.1.x to 2.2.x if you have a lot of custom modules.

  • Very nice information, so indeed, it is the best approach, yes it is not simple but I need to plan this. I want to give the bounty for this answer because it is the one which follow Magento native but it's expired. – Franck Garnier Jul 23 '18 at 7:07
1

You might reconsider having so many store views and moving towards a golden image type of install with many different very similar M2 stores. That would likely improve your indexing time. I say that without knowing why you have so many, though. But I remember Magento 1 multistores were a mess the larger they got.

Your AWS costs are probably much higher than they should be as a result of trying to parallelize it like that, too.

  • I need these high number of stores because there are POS physical stores. With different catalog / stock / customers / tax I cannot reduce this number. All working fine except for catalogsearch reindexing. – Franck Garnier Jul 19 '18 at 8:02
-1

No need to purchase any plugin or go with hosting and upgrade module you can use elastic search plugin for Magento 2 It's free and easily customizable.

I have also done a project you can look https://custom.pratikshajewelry.com/ have 47k approx product and every hour cron featuring new product from merchants.

I have used https://github.com/Smile-SA/elasticsuite

  • The number of product is not the problem but the number of store. How many stores do you have ? – Franck Garnier Jul 26 '18 at 13:23
  • I have a single store. Elastic have powerful search engine.I can tell you to use. There is no any charges to use that plugin and that is not my plugin but have a power of search. Please try one after you reply with your search experience. – Prince Yadav Jul 26 '18 at 19:27
  • Please upgrades m2.1.14 and how can import multiple stores data. Yes multiple stores take time in reindexing but really in your case, it’s too large time. Are you use command line indexing or by browse level? – Prince Yadav Jul 26 '18 at 19:47
  • I know about this plugin, I was in this company, but we decided to stay near to native and we implemented custom search featured on the top of Magento. So using this plugin is not a valid solution for me. I reindex by cron with Schedule Updates as described in my post. It took around 10 minutes by store. I am already with 2.1.14, I should upgrade to 2.2.5. – Franck Garnier Jul 27 '18 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.