3

Do you have some suggestion how can I improve the performance of the catalogsearch_fulltext indexer for Magento 2.1.X

I already set this indexer to 'Update By Schedule' so only the updated products are refreshed thanks to Mview pattern.

I have ElasticSearch as search engine, but it is not really related to it.

I have around

  • 20k products
  • 7k products visible in search by store
  • 31 stores
  • less than 10 new custom searchable attributes

For a full re-index, Magento took around 10 minutes by stores, so around 5 hours for all stores.

I plan to re-index each store in parallel and increase the server CPU and memory because each catalog search index seems independent by stores.

I try to import products with a differential approach, but sometime my customer needs a full product import.

Edit: I did a POC thanks to https://github.com/amphp/parallel reducing from 5 hours to 14 min using the c5.9xlarge EC2 instance with 36vCPU. But I want to know if there is alternative solution. (>1000$ per month)

Improve this question
7
  • "For a full re-index, Magento took around 10 minutes by stores, so around 5 hours for all stores." -> 10m * 31 = 2,5 hours in my math
    – simonthesorcerer
    Commented Jul 4, 2018 at 11:25
  • 3
    10 * 31 = 310 minutes. 310 / 60 = 5.1 hours. Can you please try to be constructive ?
    – Franck Garnier
    Commented Jul 4, 2018 at 15:01
  • Ok, can you post you server hardware & possibly php config such as memory_limit etc.?
    – simonthesorcerer
    Commented Jul 4, 2018 at 15:24
  • The server hardware can be increase. It is on a dedicated AWS EC2 m5.large for cron. But the timing is similar on my local environment. I'm trying to implement github.com/amphp/parallel to reindex stores in parallel.
    – Franck Garnier
    Commented Jul 4, 2018 at 15:37
  • 1
    @SavvasRadevic Yes it is the best approach, but it is not simple to integrate it to the native Magento reindex flow. There are many differencial reindex after an order or after a little product import. And they are a full import maybe 1 per week which took 5 hours. I think I'll try to implement your solution for the big import. But I will need to recreate the whole logic :S
    – Franck Garnier
    Commented Jul 19, 2018 at 8:13

3 Answers 3

Reset to default
2
+100

Try to upgrade to the latest version (2.2.5). As I can see in the changelog (https://devdocs.magento.com/guides/v2.2/release-notes/ReleaseNotes2.2.5CE.html#indexing) they optimized the search indexer in order to be multithreaded. Try to upgrade to 2.2.5 your local environment and test if it's worth to upgrade to 2.2.x because we know that it's not simple to upgrade from 2.1.x to 2.2.x if you have a lot of custom modules.

Improve this answer
1
  • Very nice information, so indeed, it is the best approach, yes it is not simple but I need to plan this. I want to give the bounty for this answer because it is the one which follow Magento native but it's expired.
    – Franck Garnier
    Commented Jul 23, 2018 at 7:07
1

You might reconsider having so many store views and moving towards a golden image type of install with many different very similar M2 stores. That would likely improve your indexing time. I say that without knowing why you have so many, though. But I remember Magento 1 multistores were a mess the larger they got.

Your AWS costs are probably much higher than they should be as a result of trying to parallelize it like that, too.

Improve this answer
1
  • I need these high number of stores because there are POS physical stores. With different catalog / stock / customers / tax I cannot reduce this number. All working fine except for catalogsearch reindexing.
    – Franck Garnier
    Commented Jul 19, 2018 at 8:02
-1

No need to purchase any plugin or go with hosting and upgrade module you can use elastic search plugin for Magento 2 It's free and easily customizable.

I have also done a project you can look https://custom.pratikshajewelry.com/ have 47k approx product and every hour cron featuring new product from merchants.

I have used https://github.com/Smile-SA/elasticsuite

Improve this answer
4
  • The number of product is not the problem but the number of store. How many stores do you have ?
    – Franck Garnier
    Commented Jul 26, 2018 at 13:23
  • I have a single store. Elastic have powerful search engine.I can tell you to use. There is no any charges to use that plugin and that is not my plugin but have a power of search. Please try one after you reply with your search experience.
    – Prince Yadav
    Commented Jul 26, 2018 at 19:27
  • Please upgrades m2.1.14 and how can import multiple stores data. Yes multiple stores take time in reindexing but really in your case, it’s too large time. Are you use command line indexing or by browse level?
    – Prince Yadav
    Commented Jul 26, 2018 at 19:47
  • I know about this plugin, I was in this company, but we decided to stay near to native and we implemented custom search featured on the top of Magento. So using this plugin is not a valid solution for me. I reindex by cron with Schedule Updates as described in my post. It took around 10 minutes by store. I am already with 2.1.14, I should upgrade to 2.2.5.
    – Franck Garnier
    Commented Jul 27, 2018 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.