0

I have a CSV file with about 6K SKUs that I need to loop through to update the products in Magento 2. I'm having issue where the product saves become increasingly slow, so much so that after letting it run for 24 hours, it's still not done.

I checked memory_get_usage and memory_get_peak_usage, and it hovers around 275MB for the duration, so it doesn't seem to be memory related.

I'm saving the products using ProductRepository::save. I'm stepping through the stack, and I see saving the product attributes is slow. Dispatching the events seems to be even slower, but have no idea why the saving comes to a crawl only increasingly and not from the start.

I tried production mode. This is a vanilla installation other than the module I'm working on. Any ideas?

Improve this question
2
  • Do you have index on save enabled? If so consider on schedule plus run index process in code at end of data process. I've personally seen huge savings.
    – Dominic Pixie
    Commented Nov 19, 2020 at 0:30
  • @DominicXigen it was the indexer! Thanks. Not sure why it makes it increasingly slow, however.
    – laketuna
    Commented Nov 19, 2020 at 13:01

2 Answers 2

Reset to default
1

Decided to convert to an answer.

Do you have index on save enabled?

If so consider on schedule plus run index process in code at end of data process. I've personally seen huge savings.

At a guess MySQL is utilising a lot of memory moving stuff in and out of tmp tables.

Improve this answer
1
  • This is good enough of an answer, although I still don't know why the speed becomes increasingly slow.
    – laketuna
    Commented Nov 24, 2020 at 4:20
0

you need to check if there any observer implemented on top of product save. many 3rd party extension has this functionality. you can check this by following the command in app folder

find app/ -name events.xml | xargs grep catalog_product_save

this will give you a list of events.xml file which has events for catalog_product_save

Improve this answer
3
  • There are no 3rd party modules installed. Updated posted.
    – laketuna
    Commented Nov 18, 2020 at 15:44
  • try removing media attributes like image small_image thumbnail etc from CSV than check the time difference
    – Amit Samsukha
    Commented Nov 19, 2020 at 7:49
  • It was the indexer that made it slow. Not sure why it makes it increasingly slow, however.
    – laketuna
    Commented Nov 19, 2020 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.