2

I have a CLI command on my module that runs through the products collection & updates some of the attributes.
I noticed that it's extremely slow. Each save() takes about 4-5 seconds, which is a lot when you have thousands...
My first guess was that it happens because some of the indexers are set to 'Update on save' (correct me if I'm wrong).

  1. Is there any way to tell Magento to skip the indexing until the process is done & then reindex all at once?
  2. If it's not the indexers then what else could it be?
2
  • Did you solve your issue?
    – Anthony
    Sep 19, 2019 at 16:19
  • No, I had to change my code in order to avoid saving the products during the process.
    – Pini
    Sep 22, 2019 at 6:45

4 Answers 4

1

Oy,

I know it is old and might be optimized already. But if you only updating attribute values only, it is a little overshoot to save the whole product.

Please see

\Magento\Catalog\Model\ResourceModel\Product\Action::updateAttributes

public function updateAttributes($entityIds, $attrData, $storeId)

example:

$this->_updateAction->updateAttributes(
    $productId,
    [
        'attribute_name_1' => value,
        'attribute_name_2' => value,
        ...
    ],
    $storeId
);

as indexers, you can check if the attributes required to index as

/** @var \Magento\Catalog\Model\ResourceModel\Eav\Attribute $attribute */
$attribute->isIndexable()

and trigger EAV indexer

/** @var \Magento\Catalog\Model\Indexer\Product\Eav\Processor $productEavIndexerProcessor */
$productEavIndexerProcessor->reindexList(array_unique($productIds)))

If you amending product-category connection, you might be required to run

/** @var \Magento\Catalog\Model\Indexer\Product\Category $categoryIndexer */
$categoryIndexer->reindexList(array_unique($productIds));

as well

attribute save happening directly to DB (that why it is required to handle indexer manually)


I know the whole process (with indexer) available as

\Magento\Catalog\Model\Product\Action::updateAttributes

but if you need to set different attribute values for different products it cannot be used as calling this method will reindex automatically over and over again and you will facing performance issues again

0

I solved that using temporary lock file and this code:

namespace Vendor\Module_Name\Model;

use Magento\Catalog\Model\Indexer\Category\Flat\State;
use Magento\Catalog\Model\Indexer\Category\Product;
use Magento\Catalog\Model\Indexer\Product\Category as IndexerCategory;
use Magento\Catalog\Model\Indexer\Product\Flat\Processor;
use Magento\CatalogSearch\Model\Indexer\Fulltext;
use Magento\Framework\App\Filesystem\DirectoryList;
use Magento\Framework\Filesystem\Io\File;
use Magento\Framework\Indexer\IndexerRegistry;

class ReindexLock
{
    const LOCK_FILE = 'reindex.assignment.lock';

    /**
     * @var DirectoryList
     */
    protected $directoryList;

    /**
     * @var
     */
    protected $file;

    /**
     * @var IndexerRegistry
     */
    protected $indexerRegistry;

    /**
     * @var array
     */
    protected $reindexIsScheduledStatus = [];

    protected $indexes = [
        IndexerCategory::INDEXER_ID,
        Product::INDEXER_ID,
        Processor::INDEXER_ID,
        STATE::INDEXER_ID,
        Fulltext::INDEXER_ID
    ];

    public function __construct(
        IndexerRegistry $indexerRegistry,
        DirectoryList $directoryList,
        File $file
    ) {
        $this->indexerRegistry = $indexerRegistry;
        $this->directoryList = $directoryList;
        $this->file = $file;

        $path = $this->directoryList->getPath('var') . '/tmp';

        if (false === is_dir($path)) {
            $this->file->mkdir($path, 0775);
        }

        $this->file->open(['path' => $path]);
    }

    public function __destruct()
    {
        if (count($this->reindexIsScheduledStatus) > 0) {
            foreach ($this->indexes as $index) {
                if (true === isset($this->reindexIsScheduledStatus[$index])) {
                    if (false === $this->reindexIsScheduledStatus[$index]) {
                        $productCategoryIndexer = $this->indexerRegistry->get($index);
                        $productCategoryIndexer->setScheduled(false);
                    }
                }
            }
        }

        $this->file->rm(self::LOCK_FILE);
    }

   /**
    * @return ReindexLock
    * @throws \Exception
    */
    public function changeReindexMode(): ReindexLock
    {
        if (true === $this->file->fileExists(self::LOCK_FILE)) {
            throw new \Exception("The process is already in running.\n");
        }

        if (0 === count($this->reindexIsScheduledStatus)) {
            $this->file->write(self::LOCK_FILE, '', 0666);

                foreach ($this->indexes as $index) {
                    $productCategoryIndexer = $this->indexerRegistry->get($index);
                    $this->reindexIsScheduledStatus[$index] = $productCategoryIndexer->isScheduled();

                    if ($this->reindexIsScheduledStatus[$index] === false) {
                        $productCategoryIndexer->setScheduled(true);
                    }
                }
        }

        return $this;
    }
}

The idea of lock file is to avoid parallel processes which can bring collisions with reindex mode. It saves the current mode state, then temporary sets its to "on schedule" mode and then put it back to the original one.

I'm not sure if it's ok to use it as a not-singleton class.

-1

You can set indexer mode using command

bin/magento indexer:set-mode [mode]

here you can specify mode 'realtime' for Update on save, 'schedule' Update by Schedule. For example following command set 'Category Products' and 'Product Categories' Update by Schedule.

bin/magento indexer:set-mode schedule catalog_category_product catalog_product_category

View more on magento doc

1
  • Not relevant to the question.
    – Pini
    Dec 30, 2018 at 7:41
-2

When you save a product and you've got your indexers configured as "save on update" the indexer will have to reindex after every save obviously. The other downside to this configuration is that when your catalog will grow the time to index will grow with it.

I think the best approach will be to set indexers to "reindex on schedule" which can be achieved by running the following cli command:

bin/magento indexer:set-mode realtime

Make sure you've got the Magento cron installed otherwise indexers will never update (installation command below).

bin/magento cron:install

Or to run the cron manually

bin/magento cron:run
1
  • I know all that. My question was about temporarily disabling the 'update on save' (programatically) & then set it back after the process is done. (*Although it's recommended to set the indexers to 'update by schedule', the module I'm working on might be installed on a lot of websites & some of them might be configured differently & it's not my place, as an extension builder to change their settings)
    – Pini
    Jul 5, 2018 at 6:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.