0

In the following script:

    $pageSize = 1000;
    $productCollection = Mage::getModel('catalog/product')
        ->getCollection()
        ->addWebsiteFilter($website->getId())
        ->addAttributeToFilter('status', Mage_Catalog_Model_Product_Status::STATUS_ENABLED);
    for ($currentPage = 1; $currentPage <= $pages; $currentPage++) {
        $productCollection->setCurPage($currentPage);

        foreach ($productCollection as $productItem) {
            $this->writeProductForStores($ioObject, $productItem, $stores, $defaultStore);
        }

        $productCollection->clear();
    }

Let's say I've processed page #1, now I'm on the page #2, processing products 1001-2000, and I don't need products 1-1000 from the first page, but they're still in the memory (I'm monitoring the script execution from top)! And the memory consumption grows as the execution goes, then falling with exception of surpassing the memory limit.

How do I clear memory after finishing processing the previous page?

  • What $this->writeProductForStores() for? – sv3n Oct 16 '18 at 20:21
2

You need to change how you iterate the collection.

Inchoo did a great write-up on the issue:

https://inchoo.net/magento/working-with-large-magento-collections/

  • I want to use pages. I believe, it's possible to achieve with usage of them, isn't it? I wouldn't like to write my own iterator every time I work with big collection - it seems to be an overwhelming approach. – Nikita Abrashnev Oct 16 '18 at 18:05
1

I want to use pages. I believe, it's possible to achieve with usage of them, isn't it? I wouldn't like to write my own iterator every time I work with big collection - it seems to be an overwhelming approach.

I agree with @ProxiBlue ... example code mentioned in Inchoos write-up work also in "standalone"-sripts ... and performs well on large collections ...

Examples:

  • I agree that this approach is good, but... Can we achieve the same thing doing some magic with clearing products from the processed pages? – Nikita Abrashnev Oct 16 '18 at 21:01
  • @NikitaAbrashnev I'm not sure what you mean with "pages" ? Something like bulk-operations? (I guess this code works also for several thousends of products.) May add some info to ypir question? – sv3n Oct 16 '18 at 21:06
  • Yep, there're pages in my code in the question:) $pageSize = 1000; $productCollection->setCurPage($currentPage); – Nikita Abrashnev Oct 16 '18 at 21:08
  • @NikitaAbrashnev I've seen this code. :) But why? Is it really neccessary? – sv3n Oct 16 '18 at 21:09
  • 1
    as you can see in my question, this method is already used, but it doesn't clean the memory of the running script – Nikita Abrashnev Oct 16 '18 at 21:20
-1

This is what I normally do:

$ids = Mage::getModel('catalog/product')
    ->getCollection()
    ->addWebsiteFilter($website->getId())
    ->addAttributeToFilter('status', Mage_Catalog_Model_Product_Status::STATUS_ENABLED)
    ->getAllIds();
foreach ($ids as $id) {
    $product = Mage::getModel('catalog/product')->load($id);
    $this->writeProductForStores($ioObject, $product, $stores, $defaultStore);
}

getAllIds() fetches all the IDs in one DB read but it requires loading the model n times. This is best when you need all the attributes of the model. The walk() example by Inchoo reads the DB n times but no loading of model. If you have addAtributeToSelect to limit the amount if info, the walk() method is more efficient.

  • Please do not recommend to load products in a loop. – sv3n Oct 16 '18 at 7:24
  • May I know why? – kiatng Oct 16 '18 at 8:24
  • Its one of the baddest things you can do. load($id) loads the whole product model an may lead to heavy memory consumption and load times. – sv3n Oct 16 '18 at 8:46
  • 1
    @kiatng some numbers from antother question ... magento.stackexchange.com/questions/179273/… – sv3n Oct 16 '18 at 20:10
  • 1
    It is one of the biggest performance killers – ProxiBlue Oct 16 '18 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.