1

My magento 2.2 site has stopped running because the server instance it is hosted on has run out of storage space, I have looked into what is causing this and found that the pub/media/catalog/product/cache folder is over 40GB large, I am unable to run a catalog reindex as the is no writing space available on my instance. Varnish has been returning a 503 error since.

I have over 20,000 products with each 2-3 images attached to them.

I have disabled cron and have been running manual reindexing and cache clearing.

Is it safe to delete that folder and assume that the cache would be regenerated, or would you have a better approach to handling this? I don't want to lose recent data.

  • Are you using Magento version 2.2.0 ? – Kishan Patadia Mar 15 '18 at 8:30
  • yes @KishanPatadia – Max Mar 15 '18 at 11:32
1

Is something to try, note have NOT used it and not endorsing but others mentioning duplicating product data in 2.x Issues have mentioned it.

Reference:

Blindly just removing the /product/cache directory means that all previously generated image resizing will happen again, and at 20k products that will take awhile. The tool above should help remove duplicates however if there is a core bug in 2.2 or a 3rd party module being used you'll eventually need to run the clean up again. Look to address the source if you do have duplicates.

IF no duplicates your only real option is more server space or using something like:

To optimize your product images that should not effect their quality but removing things like meta data and performing other optimizations.

  • 1
    Thanks, I ran the M2CliTools and had a lot of duplicate images. After doing that and rebooting the server, the site was up and running again. I then went to the backend and ran flush catalog image cache which seems to have fixed it. – Max Mar 15 '18 at 23:50
0
  • Issue: They had issue in increasing media size at very huge amount. This is magento's bug in lower vesion than Magento 2.2.X. This occurs when someone creates duplicate product, it repeatedly creates duplicate images.

  • We did deep investigation and found above issue, so we also cross checked in our other local setups as well. While checking media folder we found that numbers(lakhs) of duplicate images.

  • Duplicating product copies product images couple of hundred times so please check this link

    https://github.com/magento/magento2/issues/9466

    And now this type of issue solve magento 2.2.2 version upgrade Duplicating product copies product images not generate so please first crate staging replicate and check pub/media folder size check this command.

    du -msh media/

  • Solution:

    1. There is only solution to this issue is to upgrade to latest version on Magento 2.2.2. So, we did upgrade for it. To remove unused images (duplicate images) we used third party extension to remove it.

      https://magecomp.com/magento-2-image-clean.html

0

As you have mentioned that you have 20,000 products and each one has 2-3 images so what Magento does it resize catalog image for thumbnails and medium size and stores it to the pub/media/catalog/product/cache so it is obvious that it will occupy space.

You have mentioned that you don't have free space in your instance then your varnish and another process may not work properly because to run website it is required to have some free space which can be used for storing some cache file and temporary files.

Is it safe to delete that folder and assume that the cache would be regenerated, or would you have a better approach to handling this?

I don't think this would be a good solution if you delete then also magento will generate it.

And disabling cron job is not good solution cron job is also useful for many other important operations.

So here in your case solution would be you need to increase the size of your instance because you are having approx 20000*3= 60000 images.

0

I also have a Magento 2.2.5 store with very large product image cache. Running fdupes on product cache dir finds all duplicated file content and turns them into links rather than storing the file multiple times on disk.

cd /var/www/html/pub/media/catalog/product/cache
fdupes -r -1 . | while read line; do j="0"; for file in ${line[*]}; do if [ "$j" == "0" ]; then j="1"; else ln -f ${line// .*/} $file; fi; done; done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.