5

I have a store with over 100K products. I am using this code to get all my products.

$model = Mage::getModel('catalog/product')->getCollection();
foreach ($model  as $product) {
     $p = Mage::getModel('catalog/product')->load($product->getId());
     echo $p->getName();
     echo $p->getDescription();
     echo $p->shortDescription();
}

But it shows nothing. This code works where i have only less number of products in my store. So how to get all my 100K products? Thank you

3
$model = Mage::getModel('catalog/product')->getCollection()
    ->addAttributeToSelect(array('name', 'descryption','short_descryption'))
    ->setPageSize(1000);
$numberOfPages = $model->getLastPageNumber();

for ($i=1; $i<=$numberOfPages; $i++) {
    $products = $model->setCurPage($i)->load();
    foreach ($products as $product) {
        echo $product->getName() . '<br />'; 
        echo $product->getDescription() . '<br />'; 
        echo $product->shortDescription() . '<br />';
    }
}
3
  • Hi, this code does not work when i set page size to 100K. Any other good idea?? – umair ayub Mar 1 '15 at 11:16
  • Set page size to 1k or 10k and it will be work. – SeStro Mar 1 '15 at 11:43
  • just remove this line ->setPageSize(1000); simple :) – SagarPPanchal Nov 30 '15 at 10:44
4

There are several ways you can handle loading large collection, take a look at Magento: Working with large collection. Take a look at Mage::getSingleton('core/resource_iterator')->walk(), base on what your trying to accomplish I would mysql/query.

public function resourceIteratorMethod()
{
    $productCollections = Mage::getModel('catalog/product')->getCollection();

    Mage::getSingleton('core/resource_iterator')->walk(
    $productCollections->getSelect(),
    array(array($this,'productCallback'))
    );
}
public function productCallback($args)
{
    $cust = Mage::getModel('catalog/product');
    $cust->setData($args['row']);

    echo $cust->getName()."\n";
}
1
  • Hi, i am new in magento. So i am sorry if it is a stupid question. i placed the body of resourceIteratorMethod into some phtml file and also placed productCallback function into the block class of that phtml. But productCallBack function is not called. Am i doing something wrong? – umair ayub Mar 1 '15 at 11:23
2

If by "shows nothing" you mean to say that the Web browser simply returns a blank page, you should know that PHP needs to hold all of $model in memory. You then proceed to loop through it 100k times and call load on every single one of them, which fires off more requests to the database. That requires a massive amount of memory. Increase your memory. Also, be aware that code like this will cripple your store the moment you get just a few people browsing concurrently, unless you plan to install 1TB memory.

2
  • Thanks for reply. But is there any way to optimize the code? – umair ayub Feb 28 '15 at 14:31
  • If you want to display 100k products on a single page, no, there is no optimization that will help you. It's bad and will never even reasonably scale for a single user. If your intention is to display a product listing, use pagination. – danemacmillan Feb 28 '15 at 14:36
1

If you really want to load all products with all information in one run without using up all memory you could do something like

$productIds = Mage::getModel('catalog/product')->getCollection()
        ->getAllIdsCache();
// Returns a array with 100k ids

foreach ($productIds as $productId) {
    // Load one product
    $product = Mage::getModel()->load($productId);
    /** @var $product \Mage_Catalog_Model_Product */

    echo $product->getName();
    echo $product->getDescription();
    echo $product->getShortDescription();

    $product->clearIntance();
}

If you know which attributes to get you could also just add those to your filter and iterate that.

$products = Mage::getModel('catalog/product')->getCollection()
        ->addAttributeToSelect('name')
        ->addAttributeToSelect('description')
        ->addAttributeToSelect('short_description');

foreach ($products as $product) {
    /** @var $product \Mage_Catalog_Model_Product */

    echo $product->getName();
    echo $product->getDescription();
    echo $product->getShortDescription();
}

The first one would take less memory than the second one because it wont hold old information in memory.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.