13

I have a performance problem with Magento 2.1

In my shop, I have 90.000 products. I added these products to soap. When I did this it took round about 7 seconds for each article (product?) (Request>Response). In summary, it took a few days to init all products.

Now all the products are in the shop. All a few weeks I have to update some things on the articles (products). When I do this with soap again, it takes the same duration. When I do updates the shop is unusable. An example request and response are here: https://pastebin.com/aqnMJk98 https://pastebin.com/UAh0h8Zz

My server has a 12 core CPU, 24 GB RAM, and SSD. Its running Ubuntu 16.04 with Apache2 PHP7-fpm and MYSQL.

When I watch the update process I see that 5 cores with MySQL are at 100% PHP is working a little bit and the rest is idle. The ram is only a small amount used iotop says that my SSD is bored.

I watched the database performance and saw that Magento fired in the adding process thousands of commands to the database. Is this OK?

When I update it with the Magento CSV import it is much faster.

What can I do to speed this Magento2 up? In this situation its impossible to work well with this shop.

  • Which extensions are you using? The problem can be related with one of your extensions. – Bunyamin Inan Oct 24 '17 at 13:28
0

The Problem here is, that the SOAP Calls are very expensive. If you fire up a SOAP Request to add/edit/delete a product, Magento has to join a lot of tables to get all necessary information. This behavior in combination with PHP itself makes it slow for the number of products or data you want to process.

Magento itself recommends a 5 node server cluster (1 x DB, 4 x Webserver) to handle the traffic.

I would recommend scaling horizontally, for example with AWS. This might have a bigger impact on your business but IMHO you won't be able to tweak Magento to perform better.

Within AWS, you should use the Feature of an autoscaling group that is responsible for the number of servers you launch. Every time you add the data, there are new servers launching to handle the pressure. Same can be done with AWS RDS (DB Service) very easy and fast.

In case, that AWS is not an option at all, you can try to tweak Magento and PHP. I would recommend to check the settings for:

0

This link explains how to tweak MySQL memory usage. Since you have so much RAM you may benefit from allocating more to MySQL. That may be an easier and less costly ($) solution that building a dedicated database server.

https://www.percona.com/blog/2016/05/03/best-practices-for-configuring-optimal-mysql-memory-usage/

Also you may find the MySQL Workbench utility useful to help you with this tweaking and also monitoring of what MySQL is doing in general. The utility is made by Oracle/MySQL; its not some random shareware app.

https://dev.mysql.com/downloads/workbench/

0

Bulk import

the key is to use batch import, like you yourself mentioned:

When I update it with the Magento CSV import it is much faster.

Consider rethinking your SOAP API and change it into a bulk mode, e.g. you can create some middleware that will accept your product data and generate a csv file which can later be feed into Magento.

There are some extensions from 3rd parties that help with that.

General performance advice

You can try to speed up Magento itself, but you will get ~1-10% boost from each improvements, when you do multiple of them the benefit is multiplied:

  1. Make sure you don't have inefficient code
    1. Check that your code doesn't spawn notices/errors/warnings
    2. Check that you don't have expensive event listeners in the installed extensions (from 3rd party or from yourself).
  2. Once done removing the notices and warnings - consider disabling logging.
  3. Check which Magento and also 3rd party extensions you are not using and can disable see Which module I can disable in Magento 2? for reference.
  4. In case your database server is on the same machine as Magento application - make sure you are using linux socket instead of TCP/IP connection.
  5. In case your database server is not on the same machine as Magento application - make sure PHP doesn't need to do DNS resolution every time it queries the database.
  6. Check the database performance tuning advice, percona article was mentioned in one of the answers already, check also https://github.com/major/MySQLTuner-perl
  7. Check magento indexing modes, switch to index on schedule and perform a reindex every night or after import is complete, not on product save. https://devdocs.magento.com/guides/v2.2/extension-dev-guide/indexing.html#m2devgde-indexing-modes

In case all the above is done and you need more performance:

  • profile your application with xDebug or blackfire, see where are bottle necks, try to eliminate them.
  • ask around, but be more specific, explain what exactly are you trying to improve at the moment.
  • consider asking a Magento partner or a certified Magento dev to get some help.

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.