1

The Context:

By decision of the client, the Magento Administrator cannot add products by himself, all he can do is edit them, therefore the addition of products must be done programmatically. There are some "attributes" that he cannot edit as well, only programmatically (sku, available qty) and there are attributes that are merely columns in catalog_product_entity and they are not shown in Product Edit Page. Therefore i've created the following method that creates or updates the products (this products are managed by a third party in their own db).

protected function processProducts($products) {

    $dbProducts = json_decode($products, true);
    $catalogConfig = $this->_objectManager->create('Magento\Catalog\Model\Config');
    $attributeSetId = $catalogConfig->getAttributeSetId(4, 'Default');

    foreach ($dbProducts as $dbProduct){

        $product = $this->productFactory->create();
        $productId = $product->getIdBySku($dbProduct["id"]);
        if(!$productId) {
            $inventoryQty = ($dbProduct["inventory"] > 0) ? $dbProduct["inventory"] : 0;
            $isInStock = ($dbProduct["inventory"] > 0) ? 1 : 0;

            $product->setTypeId(\Magento\Catalog\Model\Product\Type::TYPE_SIMPLE);
            $product->setAttributeSetId($attributeSetId);
            $product->setWebsiteIds([$this->_storeManager->getWebsite()->getId()]);
            $product->setStatus(\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_DISABLED);
            $product->setQuantityAndStockStatus(['qty' => $inventoryQty, 'is_in_stock' => $isInStock]);
            $product->setStoreId(\Magento\Store\Model\Store::DEFAULT_STORE_ID);
            $product->setVisibility(\Magento\Catalog\Model\Product\Visibility::VISIBILITY_BOTH);
            $product->setName(utf8_encode($dbProduct["name"]));
            $product->setSku($dbProduct["id"]);
            $product->setUrlKey($dbProduct["id"]);
            $product->setLastPrice($dbProduct["lastCost"]);
            $product->setLastPriceDate($dbProduct["lastCostDate"]);
            $product->setProductGroup($dbProduct["group"]);
            $product->setProductSubgroup($dbProduct["subgroup"]);
            $product->setWarehouseQty($dbProduct["inventory"]);
            $product->setTransitQty($dbProduct["transit"]);
            $product->setSoldQty($dbProduct["sold"]);
            $product->setIsdb(true);
            $product->setIsMassupdate(true);
            $product->setExcludeUrlRewrite(true);
            $product->save();
        } else {                
            $inventoryQty = ($dbProduct["inventory"] > 0) ? $dbProduct["inventory"] : 0;
            $isInStock = ($dbProduct["inventory"] > 0) ? 1 : 0;

            $product->load($productId);
            $product->setStatus(\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_DISABLED);
            $product->setLastPrice($dbProduct["lastCost"]);
            $product->setLastPriceDate($dbProduct["lastCostDate"]);
            $product->setProductGroup($dbProduct["group"]);
            $product->setProductSubgroup($dbProduct["subgroup"]);
            $product->setWarehouseQty($dbProduct["inventory"]);
            $product->setTransitQty($dbProduct["transit"]);
            $product->setSoldQty($dbProduct["sold"]);
            $product->setIsMassupdate(true);
            $product->setQuantityAndStockStatus(['qty' => $inventoryQty, 'is_in_stock' => $isInStock]);
            $product->setIsMassupdate(true);
            $product->save();
        }
    }

    $indexer = $this->_indexerFactory->create();
    $indexerCollection = $this->_indexerCollectionFactory->create();
    $ids = $indexerCollection->getAllIds();

    foreach ($ids as $id){
        $idx = $indexer->load($id);
        if ($idx->getStatus() != 'valid'){
            $idx->reindexRow($id);
        }
    }
}

The problem:

When the algorithm executes the if-part of the statement the products are created without problems or errors and they can be edited in Admin Panel (i.e. change product name, visibility, meta title, etc). Nonetheless, when they are updated programmatically (when the algorithm executes the else-part of the statement) the new product's columns are updated without problem but after product->save(), updating attributes like name, visibility, meta title in 'Product Edit Page' doesn't work anymore.

The question is: How to correctly update products programmatically?

PD:

I don't know if it's relevant or not, but the creation of the new product's "attributes" (new columns in catalog_product_entity) was done through the following UpgradeSchema.php::upgrade method.

/**
 * {@inheritdoc}
 * @SuppressWarnings(PHPMD.ExcessiveMethodLength)
 */
public function upgrade(SchemaSetupInterface $setup, ModuleContextInterface $context)
{
    $setup->startSetup();

    if (version_compare($context->getVersion(), '1.0.4', '<=')) {

        $table = $setup->getTable('catalog_product_entity');
        $columns = [
            'last_price' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_FLOAT,
                    'nullable' => true,
                    'comment' => 'Last Price',
                ],
            'last_price_date' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_TEXT,
                    255,
                    'nullable' => true,
                    'comment' => 'Last Price Date',
                ],
            'product_group' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_TEXT,
                    255,
                    'nullable' => true,
                    'comment' => 'Group',
                ],
            'product_subgroup' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_TEXT,
                    255,
                    'nullable' => true,
                    'comment' => 'Subgroup',
                ],
            'warehouse_qty' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_INTEGER,
                    'nullable' => true,
                    'comment' => 'Warehouse Qty',
                ],
            'transit_qty' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_INTEGER,
                    'nullable' => true,
                    'comment' => 'Transit Qty',
                ],
            'sold_qty' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_INTEGER,
                    'nullable' => true,
                    'comment' => 'Sold Qty',
                ],
            'is_db' =>
                [
                    'type' => \Magento\Framework\DB\Ddl\Table::TYPE_BOOLEAN,
                    'nullable' => true,
                    'comment' => 'Is External Db Product',
                ]
        ];

        $connection = $setup->getConnection();

        foreach ($columns as $name => $definition) {
            $connection->addColumn($table, $name, $definition);
        }
    }

    $setup->endSetup();
}

Thanks,

2

After debugging the code and using the suggestion of @David i realized that the problem wasn't the way i used to create products attributes but the i was using the Create method of the Object Manager (this create duplicated entries of website id in the db) in the update-part (else-part of the statement). After using Get method of the Object Manager, the product could be saved but this couldn't be used in the foreach as it would take only on Product (instance) and ignore the rest. So for updating products' attributes i used a collection instead. Below is the working code.

protected function processProducts($products) {

    $dbProducts = json_decode($products, true);
    $catalogConfig = $this->_objectManager->create('Magento\Catalog\Model\Config');
    $attributeSetId = $catalogConfig->getAttributeSetId(4, 'Default');
    $productsToUpdate = [];
    $productsIds = [];
    foreach ($dbProducts as $dbProduct){
        try {
            $tempProduct = $this->productRepository->get($dbProduct["id"]);
            $productsIds[] = $tempProduct->getId();
            $productsToUpdate[$tempProduct->getSku()] = $dbProduct;
        } catch (\Magento\Framework\Exception\NoSuchEntityException $e) {
            $product = $this->productFactory->create();
            $inventoryQty = ($dbProduct["inventory"] > 0) ? $dbProduct["inventory"] : 0;
            $isInStock = ($inventoryQty > 0) ? 1 : 0;
            $product->setTypeId(\Magento\Catalog\Model\Product\Type::TYPE_SIMPLE);
            $product->setAttributeSetId($attributeSetId);
            $product->setWebsiteIds([$this->_storeManager->getWebsite()->getId()]);
            $product->setStatus(\Magento\Catalog\Model\Product\Attribute\Source\Status::STATUS_DISABLED);
            $product->setQuantityAndStockStatus(['qty' => $inventoryQty, 'is_in_stock' => $isInStock]);
            $product->setStoreId(\Magento\Store\Model\Store::DEFAULT_STORE_ID);
            $product->setVisibility(\Magento\Catalog\Model\Product\Visibility::VISIBILITY_BOTH);
            $product->setPrice(0);
            $product->setWeight(1);
            $product->setName(utf8_encode($dbProduct["nombre"]));
            $product->setSku($dbProduct["id"]);
            $product->setUrlKey($dbProduct["id"]);
            $product->setCustomAttribute('last_price', $dbProduct["lastCost"]);
            $product->setCustomAttribute('last_price_date', $dbProduct["lastCostDate"]);
            $product->setCustomAttribute('is_db', true);
            $product->setIsMassupdate(true);
            $product->setExcludeUrlRewrite(true);
            $product->save();
        }
    }

    if(sizeof($productsToUpdate)>0) {
        $this->updateProducts($productsIds, $productsToUpdate);
    }

    $indexer = $this->_indexerFactory->create();
    $indexerCollection = $this->_indexerCollectionFactory->create();
    $ids = $indexerCollection->getAllIds();

    foreach ($ids as $id){
        $idx = $indexer->load($id);
        $idx->reindexRow($id);
    }
}

private function updateProducts($productsIds, $productsToUpdate) {
    $collection = $this->productCollectionFactory->create()->addIdFilter($productsIds);
    foreach ($collection->getItems() as $product) {
        try {
            $productSku = $product->getSku();
            $dbProduct = $productsToUpdate[$productSku];
            $inventoryQty = ($dbProduct["inventory"] > 0) ? $dbProduct["inventory"] : 0;
            $isInStock = ($inventoryQty > 0) ? 1 : 0;
            $stockItem = $this->stockRegistry->getStockItemBySku($dbProduct["id"]);
            $stockItem->setQty($inventoryQty);
            $stockItem->setIsInStock($isInStock);
            $this->stockRegistry->updateStockItemBySku($dbProduct["id"], $stockItem);
            $stockItem->save();
            $product->setData('last_price', $dbProduct["lastCost"]);
            $product->getResource()->saveAttribute($product, 'last_price');
            $product->setData('last_price_date', $dbProduct["lastCostDate"]);
            $product->getResource()->saveAttribute($product, 'last_price_date');                
        } catch (Exception $e) {
            $e->getMessage();
        }
    }
}
1

product attributes in magento are added using the EAV model.

Basically, this means, that they are more dynamic in the database layer and also, that new attributes are stored in several tables, not only the catalog_product_entity table.

Your script just adds new columns to the catalog_product_entity table, but this does not work for EAV models.

To add a new Attribute in your setup script, there should be several tutorials by now. One Example is here: https://www.atwix.com/magento/adding-attribute-programatically-magento2/

To get an insight, what's going on in your database, you should have a look at the eav_attribute table. All EAV attributes are stored here (bound to the product model by the entity_type_id, which should be 4 for the entity_type catalog_product in a default setup).

They have a specific backend type which defines, in which table the values of the attribute are stored (for example catalog_product_entity_varchar)

  • Thanks for your answer. Sadly this wasn't the problem, nonetheless i took your suggestion of using EAV Attributes. – A. Martz Feb 1 '17 at 19:14
  • glad I could help at least ;) – David Verholen Feb 1 '17 at 19:17
0

I'm working on an import/update script as well. Is it normal it takes an eternity to process 10.000+ products? The speed of the processing also gets slower when the script runs longer.

I looked at your solution A. Martz, could you explain the following code:

 $this->stockRegistry->updateStockItemBySku($dbProduct["id"], $stockItem);
 $stockItem->save();

It looks to me you're saving it twice or am I wrong?

  • How much does it take for you? When developing in a local machine the import/update of 1000-2000 products took about 30-40 minutes but in an AWS instance in production takes 10m at most (yet, that might be a lot). You must ensure to use the $product->setIsMassupdate(true); as it instructs the app not to index the db every time a product is saved. Regarding the bit of code, it saves the stock information of the product. – A. Martz May 21 '17 at 4:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.