3

We're running a system with ~140,000 products on some good server hardware and wondered how long other sites take to do some basic tasks.

The magento setup:upgrade command takes almost 6 hours for us to complete.

I believe this is to do with it looking at just about every file within the main directory checking for file permissions etc.

We're using an NFS system to store static media on a shared location to allow our load balancers to send requests to two different application servers.

Is our hardware setup wrong? Or is this just how long it takes?

Your setup information and time to run this command would be appreciated.

Improve this question
6
  • Mine is about 10 seconds
    – harri
    Commented Jul 26, 2017 at 20:18
  • Thanks - and how many products (with images) do you have?
    – chrismcb
    Commented Jul 27, 2017 at 8:47
  • I'm not sure if it has an effect on this more indexing but store has about 500, i'm building another site which is going to have about 10,000 at the min so will let you know if it changes.
    – harri
    Commented Jul 27, 2017 at 9:33
  • The time increase we found was in relation to the quantity of image files the script checked for file permissions (please see my answer below). As such, the more products, the more images (of various sizes), the more time it took to check those files' permissions. This quickly adds up with ~140k products, multiple images per product and multiple sizes per image.
    – chrismcb
    Commented Jul 27, 2017 at 13:21
  • Interesting yeah that is a lot of products never knew it did all the images aswell tho. That is such a long time have you sorted it then?
    – harri
    Commented Jul 27, 2017 at 13:25

3 Answers 3

Reset to default
3

So it seems that, as part of the setup:upgradecommand, Magento likes to check every file within the server for permission errors.

This is triggered within installDataFixtures() with a call to $this->checkInstallationFilePermissions(); within file setup\src\Magento\Setup\Model\Installer.php

I have circumvented this command, which now takes less than 10 seconds to complete the entire process whilst I submit a bug to the Git repository to see if they can re-engineer this check to something that is done less frequently.

Improve this answer
1

I can run this command in 12.5 seconds on my local, however I have server that it takes 5.5 minutes to run it on with the same code base and the same database...

Turns out NFS can play a huge roll in how long it takes to run.

Proven by going to where the NFS mount is from and running the same command.

When you go to the server

Improve this answer
1
  • Yes, we also had media on NFS, but brought it back internally.
    – chrismcb
    Commented Jul 27, 2017 at 8:47
0

magento setup:upgrade should not be affected by how many products you have.

How much is your PHP memory limit?

This is set in php.ini and should be at least 2 gigabytes:

memory_limit = 2048M

with more than a hundred modules it takes about a minute.

Here is Magento's official documentation on how to configure your servers: http://devdocs.magento.com/guides/v2.1/install-gde/prereq/php-settings.html

Improve this answer
5
  • Thanks. Our server technician isn't quite comfortable at leaving it at this level all the time. Would you leave it at 2G for production? Or just change it when required? Or do you have a more elaborate setup with a separate development server?
    – chrismcb
    Commented Dec 10, 2016 at 14:20
  • Yes, It would be like this on production (or more). As long as that's not all the RAM your server has. Development servers are usually kept with the same settings as production servers to make sure everything works fine. I just disable caches while developing.
    – dlicheva
    Commented Dec 10, 2016 at 17:21
  • If you are using a shared server, and you are not allowed to make changes, you can ask to increase the memory limit only when doing upgrades. see here: magento.stackexchange.com/questions/90052/…
    – dlicheva
    Commented Dec 10, 2016 at 17:26
  • No, there's plenty of memory and it's private dedicated servers; so we have full control of the configuration. PHP on the CLI has memory set to -1 (e.g. unlimited), but it's still taking hours to complete. It seems to stop at "DDL cache cleared successfully". Any ideas what it is up to at that point?
    – chrismcb
    Commented Dec 10, 2016 at 19:46
  • No, I'm sorry I'm out of ideas. I hope somebody can help. :(
    – dlicheva
    Commented Dec 12, 2016 at 11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.