0

I'm having one magento 2 website which contains 1 million products. The website is based on MSI and there are multiple CRON jobs running on hourly basis to do the inventory and price updates. So the products will get enabled or disabled based on the inventory update.The issue is that indexer is not working perfectly as all the enabled products on the backend are not listing on the category or brand pages.

Also the manual indexing is taking long hours to complete.

Please help me on indexer batching or optimization?

3
  • Additional DB information request, please. OS, Version? RAM size, # cores, any SSD or NVME devices on MySQL Host server? Post TEXT data on justpaste.it and share the links. From your SSH login root, Text results of: A) SELECT COUNT(*), sum(data_length), sum(index_length), sum(data_free) FROM information_schema.tables; B) SHOW GLOBAL STATUS; after minimum 24 hours UPTIME C) SHOW GLOBAL VARIABLES; D) SHOW FULL PROCESSLIST; E) STATUS; not SHOW STATUS, just STATUS; G) SHOW ENGINE INNODB STATUS; for server workload tuning analysis to provide suggestions. Commented Aug 27, 2023 at 0:25
  • Post TEXT data on justpaste.it and share the links. Additional very helpful OS information includes - please, htop 1st page, if available, TERMINATE, top -b -n 1 for most active apps, top -b -n 1 -H for details on your mysql threads memory and cpu usage, ulimit -a for list of limits, iostat -xm 5 3 for IOPS by device & core/cpu count, df -h for Used - Free space by device, df -i for inode info by device, free -h for Used - Free Mem: and Swap:, cat /proc/meminfo includes VMallocUused, for server workload tuning analysis to provide suggestions. Commented Aug 27, 2023 at 0:25
  • Maybe this can help you ? magento.stackexchange.com/questions/304012/…
    – Robinio
    Commented Sep 14, 2023 at 13:20

1 Answer 1

0

When dealing with a large number of products in Magento 2 (e.g., 1 lakh products) and optimizing the time of the indexer during price updates, you can employ several strategies to improve performance. Here are some recommendations:

1. Indexing Mode:

  • Set the indexing mode to 'Update on Schedule' rather than 'Update on Save.' This way, changes are queued for indexing and processed in batches, reducing the immediate impact on performance during updates.

2. Splitting Updates into Batches:

  • Instead of updating all 2 lakh products in a single bulk update, split the updates into smaller batches.
  • This approach minimizes the load on the system during indexing and allows for better resource allocation.
  • You can use custom scripts or Magento CLI commands to update products in smaller groups, specifying batch sizes that work best for your server resources.

3. Use Asynchronous Indexing:

  • Leverage Magento's built-in asynchronous indexing capabilities, specifically the Magento Message Queue (MQ) system.
  • Instead of waiting for immediate indexing, enqueue the price update messages into the message queue.
  • Separate workers can then process these messages asynchronously, reducing the time taken for indexing and minimizing the impact on real-time customer operations.
  • Make sure you have configured and optimized the Magento MQ system for optimal performance.

4. Disable Unused Indexers:

  • Identify indexers that are not essential for your store's functionality.
  • Disable any unused indexers to reduce the number of indexing processes during price updates.
  • This optimization helps speed up the indexing process for relevant data.

5. Optimize Server Resources:

  • Ensure your server resources are properly allocated and optimized for handling the indexing workload.
  • Allocate sufficient memory, CPU, and disk I/O resources to accommodate the bulk updates efficiently.
  • Fine-tune PHP, database, and Elasticsearch configurations to support high-performance indexing.

6. Implement Caching:

  • Utilize caching mechanisms to reduce the need for frequent reindexing.
  • Enable appropriate caching layers, such as full-page caching or block caching, to minimize the impact of price updates on indexing.
  • This optimization helps serve cached pages to customers during the indexing process, reducing the number of dynamic requests.

7. Utilize Indexer Queues and Priority:

  • Magento allows configuring priority levels for different indexers and defining indexer queues.
  • Prioritize critical indexers that impact customer experience during price updates.
  • Set up separate queues for different indexers and allocate appropriate resources accordingly.
  • This optimization ensures that important indexers are processed first, delivering a better user experience.

By implementing these strategies, you can optimize the time of the indexer when a customer updates prices for a large number of products in Magento 2, minimizing the impact on performance and ensuring smooth operations for your online store.

Please follow the below link to know more about optimization : https://devdocs.magento.com/guides/v2.3/extension-dev-guide/indexer-optimization.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.