1

I've encountered a problem where my foreach loop is causing a 504 Gateway timeout error so I did some research and found that for large collections it is better to use the core/resource_iterator and the walk() function.

This has definitely improved performance but I have lost the ability to return the array the I need from when I call my custom function.

$this->getAllDataFromTable()

Here is what I mean: This function will get all the data from the table and return the $array variable which I can do stuff with - but is very expensive and times out with a 504 error.

public static function getAllDataFromTable()
{
    $model = Mage::getModel('custom/table');
    $collection = $model->getCollection();
    $array = array();
    foreach ($collection as $item) {
        $array[] = $item->getData();
    }
    return $array;
}

Using the iterator: The iterator is more efficient but I can only do stuff with the last row, I would like a similar $array returned so I can do stuff with it.

/**
 * Mage_Core_Model_Resource_Iterator
 */
public function getAllDataFromTable()
{
    $ordersCollection = Mage::getModel('custom/table')->getCollection();
    Mage::getSingleton('core/resource_iterator')->walk($ordersCollection->getSelect(),
        array(array($this, 'orderCallback'))
    );
}

public function orderCallback($args)
{
  return $args['row'];
}

1 Answer 1

3

Well you can simply assign a variable to the class and do something like this:

private $data = [];

public function getAllDataFromTable()
{
    $ordersCollection = Mage::getModel('custom/table')->getCollection();
    Mage::getSingleton('core/resource_iterator')->walk($ordersCollection->getSelect(),
        array(array($this, 'orderCallback'))
    );
    return $this->data;
}

public function orderCallback($args)
{
    $this->data[] = $args['row'];
}

Also, depending what you want to do with it (when you say "return the $array variable which I can do stuff with") you can add that logic directly in the callback method and that'll save you one array to fill and return.

6
  • Thanks for that, I've gone back to getting a 504 again! How can I make this more efficient as there's only 215 rows in the table? Commented Dec 8, 2016 at 14:24
  • @user1704524 sounds weird that you get a 504. What's your time limit configuration on your server ? Commented Dec 8, 2016 at 14:31
  • I have it set to ini_set('max_execution_time', 300); It gets even weirderer, I only get the timeout in the office so I assumed that there is a memory leak somewhere in the loop hence wanting to fix it using the iterator to fix the leak. Commented Dec 8, 2016 at 14:38
  • @user1704524 did you manage to find the root cause in the end ? Commented Dec 8, 2016 at 15:22
  • 2
    @user1704524 feel free to open a new question. Also try to use the Varien PRofiler to find out where the leak comes from Commented Dec 8, 2016 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.