4

I am working in a Magento 2.3 installation with avarage cart size of 100-150 lines. Getting the visible cart items from Quote is slower over time ( as products count rises )

I've search thought Magento core and i've found that the delay is caused when it tries to get the Quote from the database (?)

in $quote = $this->quoteRepository->getActive($this->getQuoteId());

public function getQuote()
{
    $this->_eventManager->dispatch('custom_quote_process', ['checkout_session' => $this]);

    if ($this->_quote === null) {
        $quote = $this->quoteFactory->create();
        if ($this->getQuoteId()) {
            try {
                if ($this->_loadInactive) {
                    $quote = $this->quoteRepository->get($this->getQuoteId());
                } else {
                    $quote = $this->quoteRepository->getActive($this->getQuoteId());
                }

                /**
                 * If current currency code of quote is not equal current currency code of store,
                 * need recalculate totals of quote. It is possible if customer use currency switcher or
                 * store switcher.
                 */
                if ($quote->getQuoteCurrencyCode() != $this->_storeManager->getStore()->getCurrentCurrencyCode()) {
                    $quote->setStore($this->_storeManager->getStore());
                    $this->quoteRepository->save($quote->collectTotals());
                    /*
                     * We mast to create new quote object, because collectTotals()
                     * can to create links with other objects.
                     */
                    $quote = $this->quoteRepository->get($this->getQuoteId());
                }
            } catch (\Magento\Framework\Exception\NoSuchEntityException $e) {
                $this->setQuoteId(null);
            }
        }

        if (!$this->getQuoteId()) {
            if ($this->_customerSession->isLoggedIn() || $this->_customer) {
                $customerId = $this->_customer
                    ? $this->_customer->getId()
                    : $this->_customerSession->getCustomerId();
                try {
                    $quote = $this->quoteRepository->getActiveForCustomer($customerId);
                    $this->setQuoteId($quote->getId());
                } catch (\Magento\Framework\Exception\NoSuchEntityException $e) {
                }
            } else {
                $quote->setIsCheckoutCart(true);
                $this->_eventManager->dispatch('checkout_quote_init', ['quote' => $quote]);
            }
        }

        if ($this->_customer) {
            $quote->setCustomer($this->_customer);
        } elseif ($this->_customerSession->isLoggedIn()) {
            $quote->setCustomer($this->customerRepository->getById($this->_customerSession->getCustomerId()));
        }

        $quote->setStore($this->_storeManager->getStore());
        $this->_quote = $quote;
    }

    if (!$this->isQuoteMasked() && !$this->_customerSession->isLoggedIn() && $this->getQuoteId()) {
        $quoteId = $this->getQuoteId();
        /** @var $quoteIdMask \Magento\Quote\Model\QuoteIdMask */
        $quoteIdMask = $this->quoteIdMaskFactory->create()->load($quoteId, 'quote_id');
        if ($quoteIdMask->getMaskedId() === null) {
            $quoteIdMask->setQuoteId($quoteId)->save();
        }
        $this->setIsQuoteMasked(true);
    }

    $remoteAddress = $this->_remoteAddress->getRemoteAddress();
    if ($remoteAddress) {
        $this->_quote->setRemoteIp($remoteAddress);
        $xForwardIp = $this->request->getServer('HTTP_X_FORWARDED_FOR');
        $this->_quote->setXForwardedFor($xForwardIp);
    }

    return $this->_quote;
}

The delay is on average about 15-25 seconds. Is there a way to store this kind of data in a caching storage like Redis ? I've already set Redis as a storage cache with no luck.

Is anyone else experiencing this kind of issue? Magento Changelog claims that carts should be fine when containing about 300 lines

https://devdocs.magento.com/guides/v2.2/release-notes/ReleaseNotes2.2.0CE.html

The project runs on a dedicated server with 32gb of RAM and 8 Cores XEON Cpu

UPDATE

After some more investigation i've found that the delay is on Quote.php line 1380-1391

public function getItemsCollection($useCache = false)
{
    if ($this->hasItemsCollection()) {
        return $this->getData('items_collection');
    }
    if (null === $this->_items) {
        $this->_items = $this->_quoteItemCollectionFactory->create();
        $this->extensionAttributesJoinProcessor->process($this->_items);
        $this->_items->setQuote($this);
    }
    return $this->_items;
}

The weird thing is that there isn't any traffic on Redis CLI monitor when requesting this collection, shouldn't this kind of data be stored in Redis?

UPDATE 2

There seems to be a "plugin" or something around getItemsCollection function. I've stopped execution in it and the response was immediate, if i let it returns the $this->items is shows the delay. Tried to use xdebug in order to find what causes the delay with no luck. Xdebug followed execution until Collection.php where i lost track. Mysql profiler showed around 1500 queries running in cart page ( i think this is a huge number ) although none of them lasted longer than ~5ms ( there where some around 15ms ).

So i've installed a fresh magento installation with luna theme and imported the sample products using composer. The new vm has MySql 5.7, PHP-FPM 7.2 and redis. Enabled production mode, all caches except page_cache and set the deafult cache to redis Added 46 items in cart and the result for just the Document load was ~13s

enter image description here

I can't accept that Magento can't handle this kind of cart size. This time is not acceptable from the end-users.

No clue from where to start searching.

UPDATE 3

Upgraded server ( CPU + more ram ) with no results. Any idea on how to cache the whole quote ? There seems to be no way of serializing the Object due to Closures. I am trying to figure if a short caching in redis ( or a longer one for the checkout page only ) makes a difference.

Already tried serialize, json_encode and base_64 for the Quote

7
  • Was your fresh luma store mage 2.2? I'd stick with that. From what I understand some of the features in 2.3 add overhead (MSI). Also did you try running any mysql tuner? Using nginx? Opcache? Redis works well with high traffic stores as fetching and storing data rapidly is more efficient. Commented Jul 12, 2019 at 21:54
  • I'm using version 2.3 for all my tests and production. Mysql tuner find my configuration ok, opcache is enabled and tuned. Redis has lots of memory allocated, but i didn't notice any collection data stored in it. Anything else apart from enabling it as the default cache needed?
    – sarakinos
    Commented Jul 12, 2019 at 22:04
  • No that should be enough. Redis is more used for session data. That sorta thing. Nginx or apache? Commented Jul 12, 2019 at 22:24
  • Apache. I don't think that this issue is related with the web server as it's not traffic related
    – sarakinos
    Commented Jul 12, 2019 at 22:25
  • Please post the complete report from MySQLTuner.pl for analysis, let another set of eyes look at your report. Thanks Commented Aug 24, 2019 at 17:18

3 Answers 3

2

Thank you for answering my questions in comments section.

You've got a few options.

More resource on stack. More CPU. But even the you are going to struggle to get close to numbers you are talking about. Using nginx or maybe varnish will offer more improvements.

Redis will help on high traffic stores. So if you've got a lot of traffic on top of large baskets it's safe to leave on and will offer improvment.

You've done SQL tuneup. Which is great.

But caches will only offer improvements. They aren't a solution.

The other option would be to attempt to streamline magento. Something like this.

https://www.integer-net.com/make-magento-2-small-again/

Finally contine to profile and investigate code. Without extensions installed it's going to be reasonably optimal.

It's also worth mentioning that it's likely that the figures from magento are on commerce (not open source) which has load balancing and split database https://devdocs.magento.com/guides/v2.3/config-guide/multi-master/multi-master.html

Good luck.

6
  • Agree with your point.Server configuration and architecture should be change and upgrade.
    – Amit Bera
    Commented Jul 13, 2019 at 3:25
  • Is there a way to enable caching database responses in ram in order to reduce database calls? I will try stripping and increasing server resources. I'll keep looking
    – sarakinos
    Commented Jul 13, 2019 at 8:52
  • Mysql has query caching built in. Chances are it will be on. But there are settings you can tweak there. Also in mysql you can add routines (stored procedures and functions). But I've never created these on a magento store though. Commented Jul 13, 2019 at 10:22
  • @sarakinos Please post TEXT results of SHOW GLOBAL VARIABLES LIKE '%size%'; and SHOW GLOBAL VARIABLES LIKE '%cache%'; and SHOW GLOBAL STATUS LIKE '%open%'; and SHOW GLOBAL STATUS LIKE '%used%; for capacity and workload analysis. Commented Aug 31, 2019 at 18:25
  • @sarakinos Or Not. Commented Sep 10, 2019 at 1:48
0

This is a known issue in Mmagento 2.3 I think. It seems to be fixed in 2.4 version Refer https://github.com/magento/magento2/issues/26231

0

I have a similar issue. We have a Magento 2 Cloud B2B project in which the average cart has ~300 products.

Performance gets worse as you add more items, and the operation which is totally out of control is the reloading of CustomerData section after an AddToCart operation happens.

I do keep complaining to Magento enterprise support, but for now they haven't given me a real solution. Magento 2 seems not to be designed for large carts. Simple as that.

Last August (2019) I found that any cart operation was triggering multiple separate queries for each product to the new Multi Source Inventory tables in the database.

My project does not use inventory at all, so in case this helps anyone, disabling these modules helped me a lot (it went from unusable to just painful to use).

    'Magento_Inventory' => 0,
    'Magento_InventoryAdminUi' => 0,
    'Magento_InventoryAdvancedCheckout' => 0,
    'Magento_InventoryApi' => 0,
    'Magento_InventoryBundleProduct' => 0,
    'Magento_InventoryBundleProductAdminUi' => 0,
    'Magento_InventoryCatalog' => 0,
    'Magento_InventorySales' => 0,
    'Magento_InventoryCatalogAdminUi' => 0,
    'Magento_InventoryCatalogApi' => 0,
    'Magento_InventoryCatalogSearch' => 0,
    'Magento_InventoryConfigurableProduct' => 0,
    'Magento_InventoryConfigurableProductAdminUi' => 0,
    'Magento_InventoryConfigurableProductIndexer' => 0,
    'Magento_InventoryConfiguration' => 0,
    'Magento_InventoryConfigurationApi' => 0,
    'Magento_InventoryDistanceBasedSourceSelection' => 0,
    'Magento_InventoryDistanceBasedSourceSelectionAdminUi' => 0,
    'Magento_InventoryDistanceBasedSourceSelectionApi' => 0,
    'Magento_InventoryElasticsearch' => 0,
    'Magento_InventoryExportStockApi' => 0,
    'Magento_InventoryIndexer' => 0,
    'Magento_InventorySalesApi' => 0,
    'Magento_InventoryGroupedProduct' => 0,
    'Magento_InventoryGroupedProductAdminUi' => 0,
    'Magento_InventoryGroupedProductIndexer' => 0,
    'Magento_InventoryImportExport' => 0,
    'Magento_InventoryCache' => 0,
    'Magento_InventoryLowQuantityNotification' => 0,
    'Magento_InventoryLowQuantityNotificationAdminUi' => 0,
    'Magento_InventoryLowQuantityNotificationApi' => 0,
    'Magento_InventoryMultiDimensionalIndexerApi' => 0,
    'Magento_InventoryProductAlert' => 0,
    'Magento_InventoryRequisitionList' => 0,
    'Magento_InventoryReservations' => 0,
    'Magento_InventoryReservationCli' => 0,
    'Magento_InventoryReservationsApi' => 0,
    'Magento_InventoryExportStock' => 0,
    'Magento_InventorySalesAdminUi' => 0,
    'Magento_InventoryGraphQl' => 0,
    'Magento_InventorySalesFrontendUi' => 0,
    'Magento_InventorySetupFixtureGenerator' => 0,
    'Magento_InventoryShipping' => 0,
    'Magento_InventorySourceDeductionApi' => 0,
    'Magento_InventorySourceSelection' => 0,
    'Magento_InventorySourceSelectionApi' => 0,
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.