11

TL;DR, The requirement is having a stock level of inventory be displayed on the category product listing page with as little additional queries/memory with performance in mind that adheres to Magento's framework.


After reading Vinai Kopp's article on preloading for scalability.

What is the best way to include the inventory stock levels on the category product listing pages (list.phtml) with as few additional queries/loads for performance sakes?

I am aware of a few approaches:

afterLoad() seems to work well with media_gallery inclusion without additional queries, however I've not been successful implementing the same approach with inventory.

$attributes = $_product->getTypeInstance(true)->getSetAttributes($_product);
$media_gallery = $attributes['media_gallery'];
$backend = $media_gallery->getBackend();
$backend->afterLoad($_product);

Direct SQL to gather the required data in parallel to the collection with a product_id key for instance. But looking for more of a means through the framework.

Currently I am simply loading the stock_item object via:

$_product->load('stock_item')->getTotalQty(); Which works, but I notice the addition of more queries to get the inventory stock totals of all products in the collection.

...

__EAV_LOAD_MODEL__ (Mage_Catalog_Model_Product, id: stock_item, attributes: NULL)
__EAV_LOAD_MODEL__ (Mage_Catalog_Model_Product, id: stock_item, attributes: NULL)
__EAV_LOAD_MODEL__ (Mage_Catalog_Model_Product, id: stock_item, attributes: NULL)

...

strangely, this works. The magic happens in Mage_Eav_Model_Entity_Abstract->load($object, $entityId, $attributes). If $attributes is empty it will call loadAllAttribute($object). So $product->load('blah') will load all missing attributes, including 'media_gallery' – William Tran Nov 19 '14 at 4:45

Add needed values to the already loaded collection.

The obvious simple approach of adding the needed data to the top level production collection in the layer/filter, would seem to be the best approach.

I did notice an observer addInventoryDataToCollection() in Mage_CatalogInventory_Model_Observer that sounds like it would achieve such, but adding the method to a custom modules observer doesn't seem to be compatible.

<events>
    <catalog_product_collection_load_after>
        <observers>
            <inventory>
                <class>cataloginventory/observer</class>
                <method>addInventoryDataToCollection</method>
            </inventory>
        </observers>
    </catalog_product_collection_load_after>
</events>

Which results in:

Warning: Invalid argument supplied for foreach() in /app/code/core/Mage/CatalogInventory/Model/Resource/Stock/Item/Collection.php on line 71

1
  • 1
    Good question boomer
    – Amit Bera
    Feb 24, 2015 at 4:37

3 Answers 3

4

The real issue here isn't preloading, it's accuracy. It's relatively easy to obtain the stock amount for a collection of products:

$products = Mage::getModel('catalog/product')->getCollection()
    ->addCategoryFilter($_category);
$stockCollection = Mage::getModel('cataloginventory/stock_item')
    ->getCollection()
    ->addProductsFilter($products);

Now with two queries, you have all the info you need. They're just hard to relate to one another, which can be fixed by using an associative array 'product_id' => 'stock' and writing a getter. Also, addProductsFilter can be optimized:

public function addProductIdsFilter(array $productIds)
{
    if(empty($productIds) {
        $this->_setIsLoaded(true);
    }
    $this->addFieldToFilter('main_table.product_id', array('in' => $productIds));
    return $this;
}

This saves you the type check and array cloning.

The problem now is Block HTML cache. This category page needs to be purged when any stock is updated on a product contained in it. As far as I know, this isn't standard, since only a stock status change purges a category page containing a product (or more accurately, a visibility change). So you'll need to observe at least cataloginventory_stock_item_before_save and maybe a few others and purge block html cache (and FPC cache) for that category page.

4
  • Your final point is the reason we went for the additional request to retrieve the stock data. If you have categories with fast moving products you cache will spend most of its time being flushed/invalidated. the only time we need to purge a category page cache is if a product is removed from that category. There is no one magic solution you have to work out the most efficient implementation on a case by case basis. If you wanted you could cache the stock data so only that has to be rebuilt after flush rather than the entire page.
    – john-jh
    Feb 25, 2015 at 8:18
  • If you implement the stock data the same way you do now, using JavaScript data to update the DOM, you can write that using a block with its own cache key and only invalidate the stock data. This is how I'd do it for your situation and not use an ESI based FPC, but one that can hole punch blocks in the request processor. Not just for the router bit, but also for requiring only one php interpreter per page. When a machine comes under pressure for whatever reason, your php interpreter is the most Cpu intensive resource. This is starting to sound more and more like a nice weekend project ;-)
    – user4351
    Feb 25, 2015 at 8:36
  • 3
    This is the type of stuff that makes the job really interesting where you have to really think about the implementation and the potential issues and bottlenecks. I defiantly see where you're coming from with the single PHP process this at the moment isn't an issue for us but can see how it would impact as you scale. The stock values displaying on the page isn't critical functionality so we load it as a secondary resource after load without impacting the user. It's a shame stock on product list not a default feature.
    – john-jh
    Feb 25, 2015 at 8:57
  • 1
    Yeah, I'm playing with the idea now to have this fetched from Redis directly and cut out php. There's some really interesting work here by Yichun Zhang on open Resty.
    – user4351
    Feb 25, 2015 at 9:14
2

I see you've already accepted and no-doubt implemented something by now, however I'd like to point out how close you were with addInventoryDataToCollection() but it looks like you misquoted the config file or we're using vastly different versions of magento. My copy of CatalogInventory/etc/config.xml has the a different method called for catalog_product_collection_load_after

 <catalog_product_collection_load_after>
    <observers>
        <inventory>
            <class>cataloginventory/observer</class>
            <method>addStockStatusToCollection</method>
        </inventory>
    </observers>
 </catalog_product_collection_load_after>

addInventoryDataToCollection() is called in <sales_quote_item_collection_products_after_load>

The source for addStockStatusToCollection() is:

public function addStockStatusToCollection($observer)
{
    $productCollection = $observer->getEvent()->getCollection();
    if ($productCollection->hasFlag('require_stock_items')) {
        Mage::getModel('cataloginventory/stock')->addItemsToProducts($productCollection);
    } else {
        Mage::getModel('cataloginventory/stock_status')->addStockStatusToProducts($productCollection);
    }
    return $this;
}

You could either set the flag require_stock_items on the collection before it's loaded, probably not that easy for the block behind the category list, or you could call Mage::getModel('cataloginventory/stock')->addItemsToProducts($productCollection) manually on the collection, after it's already loaded. addItemsToProducts() gets all the StockItems for you and attaches them to your ProductCollection

public function addItemsToProducts($productCollection)
{
    $items = $this->getItemCollection()
        ->addProductsFilter($productCollection)
        ->joinStockStatus($productCollection->getStoreId())
        ->load();
    $stockItems = array();
    foreach ($items as $item) {
        $stockItems[$item->getProductId()] = $item;
    }
    foreach ($productCollection as $product) {
        if (isset($stockItems[$product->getId()])) {
            $stockItems[$product->getId()]->assignProduct($product);
        }
    }
    return $this;
}
1

Are you using Varnish or FPC at all, or plan to in the future?

We found with the amount of hole punches/ESI requests required on the product listings it almost wasn't worth having the caching in place so opted for a different approach.

We implemented a solution on one website that uses an AJAX request to a custom controller to retrieve the stock data for the products and the javascript handles the DOM updates. The additional request for the stock data takes ~100ms which doesn't impact on the overall (visible) page load time at all. Couple that with primed FPC dropping the page request down to under 100ms you have one rapid site with low performance overhead for displaying stock data on the product listings.

All you need to do template wise is add the productId to each products wrapper html so your javascript knows which stock data to apply to each product.

If you look at additional caching techniques for the actual stock data you could drop that well below 100ms by not having to init Mage/hit the database on each request.

Sorry if this isn't along the lines of what you're looking for but we found it to be the best approach for scalability and performance against our requirements.

5
  • 2
    Deploy chaos monkey and see what happens when your cache storage falls over. The question specifically mentions not to rely on cache. It's answers like this that make our job of convincing a client that FPC isn't gonna fix their BuiltIn/MomsBasement template so much harder.
    – user4351
    Feb 24, 2015 at 20:23
  • You really misunderstand my answer if you think I'm suggesting FPC/Varnish for performance and as the solution. I stated IF they are using or considering FPC/Vanish they should investigate this approach to reduce hole punches/ESI requests. We are getting the stock data we require in under 100ms without leaveraging cache.
    – john-jh
    Feb 24, 2015 at 22:27
  • And you would get the same data in the same time using ESI. Your approach with ESI is just fundamentally different. And so, without any cache you can still get the same data, in less time. Instead of going through yet another router to send back JSON, you write a stock array in JavaScript that updates the DOM, etc. Hell, put it in JSON if you want, just cut out the router.
    – user4351
    Feb 25, 2015 at 0:14
  • 1
    Caching is going to be used but the question is more along the lines of performance optimizations. Some background: magento.stackexchange.com/questions/13957/… (no cache/hardly any) Thanks for the reply however, appreciate the input!
    – B00MER
    Feb 25, 2015 at 2:40
  • Not sure if there is a "best practice" way of doing this in M2, but your solution is how we've always done it since Magento 1.x.
    – thdoan
    May 11, 2017 at 3:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.