3

I'm puzzled why memory usage steadily increase over a foreach loop in a shell script I wrote for Magento to index some data. There isn't any compounding variable (arrays increasing in size, etc.) and I'm re-using the same variables in each iteration. I can't post the entire code, but here are some lines..

For each iteration of the loop, check out the below. About 0.11MB of increased usage is shown at each iteration.

$childProduct = Mage::getModel('catalog/product')->load($childId);

$searchProduct = Mage::getModel('catalog/product')->load($parentId);

$childProduct->getAttributeText($this->_mappedColorAttributeCode);

$url = Mage::getBaseUrl(Mage_Core_Model_Store::URL_TYPE_WEB) . Mage::getModel('core/url_rewrite')->loadByIdPath('product/'.$searchProduct->getId())->getRequestPath();

$imgUrl = (string)Mage::helper('catalog/image')->init($searchProduct, 'small_image')->resize(135);

$colorInfo = array() which subsequently get values assigned to it, then used to created a query, $q to run

$colorInfo is passed to saveOneRow() (shorten version)

public function saveOneRow($data) {
    // Check existence first
    $now = date("Y-m-d H:i:s");
    $data['created_at'] = $now;
    $q = "INSERT INTO {$this->_tableName}...";

    try {
        $this->_writeConnection->beginTransaction();
        $this->_writeConnection->query($q);
        $this->_writeConnection->commit();

    } catch (Exception $e){
        $this->_writeConnection->rollback();
        $detail = "[{$data['sku']}] failed to write";
        $this->_writeConnection->query("INSERT INTO {$this->_logTableName} (`key_value`,`detail`,`created_at`) VALUES ({$data['entity_id']},'$detail','$now');");
    }
}

Which of these could make the usage go up incrementally?

2

I looked into this a long, long time ago but I think the problem hasn't been fixed yet, the Observers attached to the product load leak memory.

To confirm you just would need to measure the memory usage before and after the product load lines - similar to how you have already done it to get the usage for the whole iteration.

Take a look at David's presentation at this year's Imagine conference on ERP integrations. It includes a blueprint on creating a process that keeps within its memory limitations.

2
  • Thanks for this post Fooman. I'll read your thread soon. It looks like loading a product collection evades this problem. It's working out very well for me in this problem (much faster, too). – musicliftsme Sep 11 '13 at 22:59
  • Great to hear you have found something that works for you. Something to bear in mind is when using collections is that the product load observers are not triggered. This for example means that inventory data is not automatically added for you. – Kristof at Fooman Sep 11 '13 at 23:03
1

Per philwinkle answer, the product models were the cause of steadily increasing memory usage.

I recommend using something like below instead of ...->load($id). But I still don't know why loading product models makes the memory usage go up steadily.

Mage::getModel('catalog/product')->getCollection()
    ->addAttributeToSelect('my_atts')
    ->addAttributeToFilter('entity_id', $id)
    ->getFirstItem();
0

Which PHP version are you using? Older version had a bug in the garbage collection and the memory was not freed.

You can call gc_collect_cycles manually to clean your memory at the end of the loop.

3
  • We use 5.4.17. I'll give it a shot next time! – musicliftsme Sep 12 '13 at 15:18
  • This error should be fixed in PHP 5.4 – Fabian Blechschmidt Sep 12 '13 at 17:12
  • I tried this the other day, but I didn't see any memory cleared up. – musicliftsme Sep 19 '13 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.