0

I'm trying to fetch around 500 products from Magento, but the call is really slow.

    $startTime = microtime(TRUE);
    $collection = Mage::getModel('catalog/product_api')->items();

    $endTime = microtime(TRUE);
    $totalTime = $endTime - $startTime;
    $timePerProduct = $totalTime / count($collection);

    $benchmarking = array(      
        "totaltime" => $totalTime,
        "timeperproduct" => $timePerProduct,
        "totalproducts" => count($collection)           
    );

    $response = array($benchmarking, $collection);
    echo json_encode($response);

The total time is ending up being an entire second! I'm not sure if this is normal

{ totaltime: 1.0537550449371, timeperproduct: 0.0018136919878436, totalproducts: 581 },

I'm currently running the server on Windows Azure A2 configuration. (2 cores, 3.5GB memory. PHP is 5.4, Centos 7.)

2 Answers 2

2

The real question is, do you need to get all that data for products or just a few attributes?

You could specify which attributes you need to speed things up. (as you can also see when opening Mage_Catalog_Model_Product_Api) Only there is more data to grab secondhand if you take a closer look, category ids are loaded, etcetera.

$collection = Mage::getModel('catalog/product')->getCollection();
$collection->addAttributeToSelect('name');

To get initial counts you should also use getSize() instead of count() on large collections to speed things up.

$collection->getSize();

Of coarse you should not be surprised by load times of +1 second on large collections. That's where pagination joins in.

For lazy loading you could start by using setPage (getSize will still return the total amount of products)

$pageNum = 1; $pageSize = 25;
$collection->setPage($pageNum, $pageSize);

Contents of Mage_Catalog_Model_Product_Api->items($filters = null, $store = null)

$collection = Mage::getModel('catalog/product')->getCollection()
    ->addStoreFilter($this->_getStoreId($store))
    ->addAttributeToSelect('name');

/** @var $apiHelper Mage_Api_Helper_Data */
$apiHelper = Mage::helper('api');
$filters = $apiHelper->parseFilters($filters, $this->_filtersMap);
try {
    foreach ($filters as $field => $value) {
        $collection->addFieldToFilter($field, $value);
    }
} catch (Mage_Core_Exception $e) {
    $this->_fault('filters_invalid', $e->getMessage());
}
$result = array();
foreach ($collection as $product) {
    $result[] = array(
        'product_id' => $product->getId(),
        'sku'        => $product->getSku(),
        'name'       => $product->getName(),
        'set'        => $product->getAttributeSetId(),
        'type'       => $product->getTypeId(),
        'category_ids' => $product->getCategoryIds(),
        'website_ids'  => $product->getWebsiteIds()
    );
}
return $result;

As you can see, you can just use some best practices in your own controller.

A working example with your code:

$collection = Mage::getModel('catalog/product')->getCollection()
    ->addStoreFilter(Mage::app()->getStore())
    ->addAttributeToSelect('name');

// Set page
$collection->setPage(1, 50);

$result = array();
foreach ($collection as $product) {
    $result[] = array(
        'product_id' => $product->getId(),
        'sku'        => $product->getSku(),
        'name'       => $product->getName(),
        'set'        => $product->getAttributeSetId(),
        'type'       => $product->getTypeId(),
        'category_ids' => $product->getCategoryIds(),
        'website_ids'  => $product->getWebsiteIds()
    );
}

echo json_encode($result);
11
  • The problem is I actually need to add attributes to the ones returning already, rather than take away the default ones returned. (Product ID, SKU, Name, categories, etc..) However, as you said, I think I need to add pagination to speed things up.
    – Pat
    Feb 26, 2015 at 7:24
  • You could apply a sort of lazy loading... Than visually a user gets an initial view, after that you load the rest(if needed), I do not believe the users sees 500+ products in one view(but I could be wrong ;))
    – Jeroen
    Feb 26, 2015 at 7:47
  • Indeed, that seems to be the only reasonable solution. The problem is I'm still mostly new to Magento, and I'm trying to figure out how to get the products with pagination and lazy. :d
    – Pat
    Feb 26, 2015 at 7:51
  • I've updated my answer with the setPage option
    – Jeroen
    Feb 26, 2015 at 8:10
  • Excellent! You just saved me a lot of research! I've read up a bit on it, and it seems I have to override/make my own class for the product API, correct?
    – Pat
    Feb 26, 2015 at 8:16
1

Yes Magento IS that slow if you query the product catalogue with that particular API call. The API call will load the complex product model data and make many, many database calls to retrieve the data you requested. The database is an Entity-Attribute-Value database where there is great flexibility but not a lot of speed.

This lack of speed is solved on the front end by using a 'flat catalog'. This is a series of tables in the database where the complex data that is normally held in untold tables is put in simple rows is placed. There is one flat product catalogue table for each store view.

It might take a while to adjust to Magento and its 'slow speed'.

2
  • I currently don't really have a front end, I'm just adding json responses for my app when it asks for products and such. Unless of course that qualifies as front end as well!
    – Pat
    Feb 26, 2015 at 7:33
  • I tried enabling the flat catalog for products, and it seemed to mess things up for me. Only half the products were being returned, and their skus were being returned as null..
    – Pat
    Feb 26, 2015 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.