I'm working on magento 2.1.9 with 1500 configurable products that leads to ~30k simple products. Unfortunaly our customer doesn't apply a percentage of discount over the products, but set a "new price" amout for every configuration (not simple) so we have ~1000 catalog rules linked to each configurable SKU.
when we had 300 configurable products reindexing the catalog price rules took 3 minutes, and less than 2GB of ram, now, to avoid out of memory we allocated 10GB of ram for the script, and it takes 20minutes to complete. We also had to switch reindexing on "schedule" since saving a single product rule was leading to 20minutes delays (and timeouts)
it seems ram usage is proportionally related to the number of products and rules, but that means when we'll have 5k configurable products we'll have to allocate 40GB of ram, and it seems a bit too much, just for reindexing the price rules...
can someone please tell me if there is something wrong with the reindexing process, or if this beavior is normal?
PHP Fatal error: Allowed memory size of 5293211648 bytes exhausted (tried to allocate 20480 bytes) in /***/magento/module-catalog- rule/Model/Rule.php on line 346