2

We have a store with 35000 sku and there are 950 attribute options -

We find that when making any updates to products like price or qty or any other changes it is very very slow - like 5 mins if saving via Admin

We have tried some import/export extension like Firebear but these also seem to be slow

Came across this patch by Magento - MDVA-31021: slow import and export

link to patch page

  1. Has anyone tried this and does it make any difference? with import speeds

With Native Magento Import or With other Import Extensions?

  1. Has anyone tried it with Magento 2.4.3 Adobe Commerce (all deployment methods) 2.3.0 - 2.4.1

UPDATE

the requested database information can be downloaded from here download

I had to export as csv files and put them according to the A B C etc host provided with the dump file as I was not allowed root access.

cloud server with 4GB RAM 2 Core 80GB SSD Hd

Apache Elasticsearch Memcached MySQL MariaDB Nginx PHP FPM Redis Varnish

19
  • Check logs, I have recently imported 400k products with custom attributes data and also categories,, it took me 20-30 minutes per 15-20k skus' data. magento2.4.4 Oct 27, 2022 at 22:03
  • @ Zahid thanks were you importing using native magento import or via any extension? and what should you be looking for in logs because we cant find anything clear which can be causing this. This only happened after we created 900 custom attributes went this slow
    – nem3943
    Oct 27, 2022 at 23:14
  • Yes native magento importer. please can you tell me how much custom attributes are you imporitng at the time? you should import like this : skus 30-40 custom attributes with 20-25k values than it will not take much time. so you must first create chunks of data and run it on daily basis. Oct 28, 2022 at 5:43
  • actually we created 920 custom attributes some are magento own- we have not imported any of the sku with the custom attributes yet - products have already been imported with all the usual attributes fairly quickly - but after we created the additional 920 sku import has gone very very slow we tested with just import 1 attribute like manufacturer (magento attribute) taking like 30 hours- we have cleared and flushed and done reindex and importing via ssh
    – nem3943
    Oct 28, 2022 at 11:09
  • Yes this may be issue with your server or magento, you need to debug it, according to yoru last comment I also faced this same issue with 3 magento websites, may be it is magento bug. when we import lots of products like 700k (I imported in 3 magento 2.3 ) and after 700k every you new entry tokk time more than minutes. But you can try this solution: take 500 skues and import same file 3 times, 3rd time will import fastly. or if you share your admin I may try it on. it is not related with indexes. Thanks Oct 28, 2022 at 11:51

2 Answers 2

0

Magento official share some optimization for MariaDB (and MySQL), let's apply them:

  1. For best efficiency, specify a combination of innodb_buffer_pool_instances and innodb_buffer_pool_size so that each buffer pool instance is at least 1 GB.

  2. Reindexing on MariaDB 10.4 takes more time compared to other MariaDB or MySQL versions. As a workaround, we suggest modifying the default MariaDB configuration and setting the following parameters:

optimizer_switch='rowid_filter=off'

optimizer_use_condition_selectivity = 1

Reference:

0

Rate Per Second = RPS

Suggestions to consider for your my.cnf [mysqld] section with present 4G 2 cores

max_connections=100  # from 600 since max_used_connections was 32 in 5 days
innodb_io_capacity=500  # from 200 to use more of your available SSD IOPS available
innodb_lru_scan_depth=100  # from 1024 to conserve 90% of CPU cycles used for function every second
max_allowed_packet=512M  # from 1G until RAM is doubled
thread_pool_size=2  # from 128 autoconfig to throttle threads_running until more cores available
read_rnd_buffer_size=32K  # from 256K to redud handler_read_rnd_next RPS of 4,131
innodb_change_buffer_max_size=50  # from 25 (percent) to expedite row maintenance

View profile for contact info. Many more opportunities exist to improve performance.

1
  • thanks we are working with host on this will contact you soon
    – nem3943
    Nov 9, 2022 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.