0

Magento 2.4

I am importing around 80000 products from an external source in an external script (had to do it this way). They come from an API which uses tokes that are only valid for 2 hours. So every 2 hours the connection is closed and I have to restart the import and I have to check if a product already exists in order to skip it.

My approach right now is this:

$product = $objectManager->create('Magento\Catalog\Model\Product');

try {
    $product = $productRepository->get($articelNumber);
} catch(Exception $e) {}
    
    
// If the product already exists, skip it
if($product->getId()) {
    continue;
}

This takes a while. even for 10000 products and the more products I have the longer it takes to check them all. What would be way faster then this approach? Maybe direct SQL?

Thanks :-)

3
  • Instead making a request per product you can gather product ids in array using for / foreach and then make one reuqest for all ids. Using product repository launches some observer and plugins, so it may take longer. One approach could be an sql using Magento\Framework\App\ResourceConnection. Second approach could be using Product Collection instead of repository. Both options should be quicker and "lighter" on db. I used the direct sql option when we were updating thousands products each hour and product repository was killing the db.
    – Sebastian
    Jan 12, 2023 at 7:59
  • So what you are suggesting is to use direct SQL to find the SKU in the database? Jan 12, 2023 at 9:12
  • yes, I've posted my answer with code samples
    – Sebastian
    Jan 12, 2023 at 9:41

2 Answers 2

1

First, instead making a request per product you can gather product ids in array using for / foreach and then make one (or few, depending on the numbers you can make batches) request for all ids. As for the method / class - using product repository launches some observers and plugins, so it may take longer. If you do not need all the additional data etc, just to check if the product with some ID exists you can:

  • use Magento\Framework\App\ResourceConnection for direct sql
/** @var Magento\Framework\App\ResourceConnetcion $resourceConnetion */
$select = $resourceConnetion->getConnection()->select()
->from('catalog_product_entity')
->where('entity_id IN(?)', $productIds);
$products = $resourceConnection->fetchAll($select);
  • use Magento\Catalog\Model\ResourceModel\Product\CollectionFactory
/** @var Magento\Catalog\Model\ResourceModel\Product\CollectionFactory $collectionFactory */
$collection = $collectionFactory->create();
$collection->addAttributeToFilter('entity_id', ['in' => $productIds]);

Both options should be quicker and "lighter" on db. I used the direct sql option when we were updating thousands of products each hour and product repository was killing the db.

0
0

The quickest way to check if a product exists in Magento 2 is to use the built-in product repository. The product repository provides methods for loading, saving, and deleting products. To check if a product exists, you can use the getById() method and check the returned product object. If the product object is not null, then the product exists.

$product = $this->productRepository->getById($productId);
if ($product) {
    // product exists
} else {
    // product does not exist
}

You can also check the product existence by using get($sku) method

try {
    $product = $this->productRepository->get($sku);
    // product exists
} catch (NoSuchEntityException $e) {
    // product does not exist
}

If you want to check product existence by using product collection then you can use below code snippet:

$collection = $this->productCollectionFactory->create();
$collection->addFieldToFilter('sku', $sku);
$product = $collection->getFirstItem();

if ($product->getId()) {
    // product exists
} else {
    // product does not exist
}

It's important to note that you may need to inject the appropriate classes and dependencies into your class before you can use them, such as

\Magento\Catalog\Api\ProductRepositoryInterface,
\Magento\Catalog\Model\ProductFactory,
\Magento\Catalog\Model\ResourceModel\Product\CollectionFactory
1
  • I am already using the product repository and its get($sku) function. Still too slow :-(. Jan 12, 2023 at 9:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.