2

I have a Mage CE 1.9.3+ store that hooks up to an ERP. The connector has pushed duplicate images to every product (3k+) - so we now have between 20-50 redundant images per product.

I've written something that I can call via the browser to clean up these surplus images category by category but its incredibly, incredibly slow. The user enters the category id & the function. At the moment I'm calling 'surplus' to remove these surplus images.

I'm using the media api. I suspect that the product is being saved after each image is removed & possibly re-indexed?

I'm sure this can be improved & speeded up by someone who knows their Magento better than I do.

// do an IP check
if (substr($REMOTE_ADDR != "ip")) {
    echo "You are not allowed!";
    exit();
}

include('app/Mage.php');
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);
error_reporting(E_ALL | E_STRICT);
Mage::setIsDeveloperMode(true);//not sure why I've set this!
ini_set('display_errors', 1);
ini_set('max_execution_time', 600);
ob_implicit_flush (1);
$mediaApi = Mage::getModel("catalog/product_attribute_media_api");
//take parameters
if (isset($_GET['function'])) {
    $method=$_GET['function'];
}
if (isset($_GET['categoryId'])) {
    $categoryID=$_GET['categoryId'];
}

$category = Mage::getModel('catalog/category')->setId($categoryID);
$attributes = Mage::getSingleton('catalog/config')->getProductAttributes();
$_products = Mage::getResourceModel('catalog/product_collection')
    ->addCategoryFilter($category)
    ->addAttributeToSelect($attributes);

$i =0;
$total = count($_products);
echo 'PRODUCT count = '.$total .' <br/>';
$position = array(0,1,2,3,4,5,6,7);
$count = 0;
$mediaApi = Mage::getModel("catalog/product_attribute_media_api");
foreach($_products as $_prod) {
    $prodID = $_prod->getId();
    $_product = Mage::getModel('catalog/product')->load($prodID);
    $items = $mediaApi->items($_product->getId());
    if (count($items) > 0) {
        foreach($items as $item) {
            if ($method == 'surplus') {
                if (in_array($item['position'], $position)) {
                } else {
                    removeSurplusImages($mediaApi, $_product, $item);
                    $count++;
                }
            } elseif ($method=='alts') {
                removeImagesWithAltTag($mediaApi,$_product, $item);
                $count++;
            }
        }
        echo "PRODUCT ". $_prod->getSku() . " done \n";
    } else {
        echo "<br>PRODUCT ". $_prod->getSku() . " has no images \n";
    }
}//EOF foreach

echo "<br/> finished removed product images $count  ";

/**
 * @param $mediaApi
 * @param $_product
 * @param $item
 * Deletes all images that have a position > 7. Checks that main or base
   image is not deleted
 * There may be > 1 image with position 6, for example. Such images will not
   be deleted
 */

function removeSurplusImages($mediaApi,$_product, $item)
{
    if ($_product->getSmallImage() != $item['file']
        && $_product->getThumbnail() != $item['file']
        && $_product->getImage() != $item['file']) {
        $mediaApi->remove($_product->getId(), $item['file']);
    }
    return;
}

/**
 * @param $mediaApi
 * @param $_product
 * @param $item
 * Deletes all images that have a label/alt tag.
 */
function removeImagesWithAltTag($mediaApi,$_product, $item)
{
    if ($item['label'] !='') {
        $mediaApi->remove($_product->getId(), $item['file']);
    }
    return;
}

Thanks for any help.

1

The main problem in your case is that you're loading products in a loop by calling:

$_product = Mage::getModel('catalog/product')->load($prodID);

in your foreach loop. This is very very bad in terms of performance. However, according to your code, you're only using the $_product variable to retrieve the image, small_image and thumbnail.

However, I'm pretty sure those 3 attributes are automatically added to select when you call:

$attributes = Mage::getSingleton('catalog/config')->getProductAttributes();

And then uses the $attributes variable to add the attribute to select.

My suggestion is to remove this line from your code:

$_product = Mage::getModel('catalog/product')->load($prodID);

And test the code that checks the image attributes.

If you see that an attribute is missing you can simply add the following line to your collection generation.

For example, if the base image is missing you can replace:

$_products = Mage::getResourceModel('catalog/product_collection')
   ->addCategoryFilter($category)
   ->addAttributeToSelect($attributes);

With:

$_products = Mage::getResourceModel('catalog/product_collection')
   ->addCategoryFilter($category)
   ->addAttributeToSelect($attributes)
   ->addAttributeToSelect('image');
| improve this answer | |
  • Hi, thanks for your help - I applied your code & now images are being deleted at about 0.7 seconds each image. Still not great but usable for most of our categories & the loading of categories, etc seems faster.. I'm going to disable reindexing & see how that helps. – Freshwebs Sep 22 '17 at 13:48
  • @Raphel, I am facing this issue, can you help me out please, magento.stackexchange.com/questions/259796/… – jafar pinjar Jan 30 '19 at 12:22
0

If you wanr to remove all images with position > 7 - without further checks - this can work too:

<?php
require_once('./app/Mage.php');
error_reporting(E_ALL);
ini_set('display_errors', 1);

umask(0);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);

$resource = Mage::getSingleton('core/resource');
$read     = $resource->getConnection('core_read');
$write    = $resource->getConnection('core_write');

$galleryTable       = $resource->getTableName('catalog_product_entity_media_gallery');
$galleryValueTable  = $resource->getTableName('catalog_product_entity_media_gallery_value');

$categoryId = 41;
$category   = Mage::getModel('catalog/category')->setId($categoryId);
$productIds = Mage::getResourceModel('catalog/product_collection')
    ->addCategoryFilter($category)
    ->getAllIds();

# get all image ids
$query = $read->select()
    ->from($galleryTable, array('value_id'))
    ->where('entity_id IN (?)', $productIds);
$values = $read->fetchCol($query, 'value_id');

# get all image ids w/ pos > 7
$query = $read->select()
    ->from($galleryValueTable, array('value_id'))
    ->where('position > ?', 7)
    ->where('value_id IN (?)', $values);
$values = $read->fetchCol($query, 'value_id');

$condition  = array(
    $write->quoteInto('value_id IN (?)', $values)
);

$write->delete($galleryTable, $condition);
$write->delete($galleryValueTable, $condition);

or ... for all products

# get all image ids w/ pos > 7
$query = $read->select()
    ->from($galleryValueTable, array('value_id'))
    ->where('position > ?', 7);
$values = $read->fetchCol($query, 'value_id');

$condition  = array(
    $write->quoteInto('value_id IN (?)', $values)
);

$write->delete($galleryTable, $condition);
$write->delete($galleryValueTable, $condition);

Edit: another bad thing for performance is this line ... it loads the product collection.

$total = count($_products);

Better use

$total = $_products->getSize();

Edit 2 to keep base/small image and thumbnails you can add this:

$eavAttributeTable  = $resource->getTableName('eav_attribute');
$productImageTable  = $resource->getTableName('catalog_product_entity_varchar');

After getting your $productIds add

# keep "frontend" images
# filter by frontend model should work for custom image attributes too
$query = $read->select()
    ->from($eavAttributeTable, array('attribute_id'))
    ->where('frontend_model = ?', 'catalog/product_attribute_frontend_image');
$attributeIds = $read->fetchCol($query, 'attribute_id');

$query = $read->select()
    ->distinct()
    ->from($productImageTable, array('value'))
    ->where('attribute_id IN (?)', $attributeIds)
    ->where('entity_id IN (?)', $productIds);
$keepImages = $read->fetchCol($query, 'value');

Add this to # get all image ids query

    ->where('value NOT IN (?)', $keepImages)
| improve this answer | |
  • Hi, thanks for your contribution. I didn't try your method because I was keen to limit myself to the existing APIs if I could . Also, I couldn't see where I would actually remove the image from disk in your approach. – Freshwebs Sep 22 '17 at 13:46
  • @Freshwebs you're right - this doesn't remove images from disc, but there are some standalone script that do this. Using APIs may be the cleaner solution :) Just tried to find "fastest" solution ... for a category with 750 products it takes ~0,5s. – sv3n Sep 22 '17 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.