I'm trying to upload the following CSV file of images programmatically but it's failing to import. The weird thing is that it works from admin
sku,base_image,base_image_label,small_image,small_image_label,thumbnail_image,thumbnail_image_label,additional_images
20303050001,https://cdn.example.com/43835/e_43835_1.jpg,e_43835_1.jpg,https://cdn.example.com/43835/e_43835_1.jpg,e_43835_1.jpg,https://cdn.example.com/43835/e_43835_1.jpg,e_43835_1.jpg,https://cdn.example.com/43835/e_43835_2.jpg
20303050002,https://cdn.example.com/43836/e_43836_2.jpg,e_43836_2.jpg,https://cdn.example.com/43836/e_43836_2.jpg,e_43836_2.jpg,https://cdn.example.com/43836/e_43836_2.jpg,e_43836_2.jpg,https://cdn.example.com/43836/e_43836_2.jpg~https://cdn.example.com/43836/e_43836_1.jpg
20303050003,https://cdn.example.com/43837/e_43837_3.jpg,e_43837_3.jpg,https://cdn.example.com/43837/e_43837_3.jpg,e_43837_3.jpg,https://cdn.example.com/43837/e_43837_3.jpg,e_43837_3.jpg,https://cdn.example.com/43837/e_43837_2.jpg
20303050004,https://cdn.example.com/43838/e_43838_2.jpg,e_43838_2.jpg,https://cdn.example.com/43838/e_43838_2.jpg,e_43838_2.jpg,https://cdn.example.com/43838/e_43838_2.jpg,e_43838_2.jpg,https://cdn.example.com/43838/e_43838_1.jpg
20303050005,https://cdn.example.com/43839/e_43839_2.jpg,e_43839_2.jpg,https://cdn.example.com/43839/e_43839_2.jpg,e_43839_2.jpg,https://cdn.example.com/43839/e_43839_2.jpg,e_43839_2.jpg,https://cdn.example.com/43839/e_43839_2.jpg~https://cdn.example.com/43839/e_43839_1.jpg
20303050009,https://cdn.example.com/43843/e_43843_1.jpg,e_43843_1.jpg,https://cdn.example.com/43843/e_43843_1.jpg,e_43843_1.jpg,https://cdn.example.com/43843/e_43843_1.jpg,e_43843_1.jpg,https://cdn.example.com/43843/e_43843_1.jpg~https://cdn.example.com/43843/e_43843_2.jpg
20303050010,https://cdn.example.com/43844/e_43844_1.jpg,e_43844_1.jpg,https://cdn.example.com/43844/e_43844_1.jpg,e_43844_1.jpg,https://cdn.example.com/43844/e_43844_1.jpg,e_43844_1.jpg,https://cdn.example.com/43844/e_43844_1.jpg~https://cdn.example.com/43844/e_43844_2.jpg
20303050011,https://cdn.example.com/43845/e_43845_1.jpg,e_43845_1.jpg,https://cdn.example.com/43845/e_43845_1.jpg,e_43845_1.jpg,https://cdn.example.com/43845/e_43845_1.jpg,e_43845_1.jpg,https://cdn.example.com/43845/e_43845_2.jpg
20303050012,https://cdn.example.com/43846/e_43846_1.jpg,e_43846_1.jpg,https://cdn.example.com/43846/e_43846_1.jpg,e_43846_1.jpg,https://cdn.example.com/43846/e_43846_1.jpg,e_43846_1.jpg,https://cdn.example.com/43846/e_43846_1.jpg
20303050013,https://cdn.example.com/43847/e_43847_1.jpg,e_43847_1.jpg,https://cdn.example.com/43847/e_43847_1.jpg,e_43847_1.jpg,https://cdn.example.com/43847/e_43847_1.jpg,e_43847_1.jpg,https://cdn.example.com/43847/e_43847_1.jpg
20303050014,https://cdn.example.com/43848/e_43848_1.jpg,e_43848_1.jpg,https://cdn.example.com/43848/e_43848_1.jpg,e_43848_1.jpg,https://cdn.example.com/43848/e_43848_1.jpg,e_43848_1.jpg,https://cdn.example.com/43848/e_43848_1.jpg
20303050015,https://cdn.example.com/43849/e_43849_1.jpg,e_43849_1.jpg,https://cdn.example.com/43849/e_43849_1.jpg,e_43849_1.jpg,https://cdn.example.com/43849/e_43849_1.jpg,e_43849_1.jpg,https://cdn.example.com/43849/e_43849_1.jpg
20303050016,https://cdn.example.com/43850/e_43850_1.jpg,e_43850_1.jpg,https://cdn.example.com/43850/e_43850_1.jpg,e_43850_1.jpg,https://cdn.example.com/43850/e_43850_1.jpg,e_43850_1.jpg,https://cdn.example.com/43850/e_43850_1.jpg
So I'm trying to import using the following relevant code
$csv = Writer::createFromFileObject(new \SplTempFileObject());
$csv->insertOne($csvHeader);
$csv->insertAll($csvData);
$content = $csv->getContent();
$importFolder = $this->directoryList->getPath(
DirectoryList::MEDIA
) . DIRECTORY_SEPARATOR . 'import';
$timestamp = $runDate->getTimestamp();
$importPath = $importFolder . DIRECTORY_SEPARATOR . $timestamp . 'import_images.csv';
file_put_contents(
$importPath,
$content
);
$importFile = pathinfo($importPath);
$import = $this->importFactory->create();
$import->setData(
[
'entity' => 'catalog_product',
'behavior' => Import::BEHAVIOR_APPEND,
Import::FIELD_NAME_VALIDATION_STRATEGY => 'validation-stop-on-errors',
Import::FIELD_FIELD_SEPARATOR => ",",
Import::FIELD_FIELD_MULTIPLE_VALUE_SEPARATOR => '~',
Import::FIELDS_ENCLOSURE => '',
Import::FIELD_EMPTY_ATTRIBUTE_VALUE_CONSTANT => Import::DEFAULT_EMPTY_ATTRIBUTE_VALUE_CONSTANT,
Import::FIELD_NAME_IMG_FILE_DIR => '',
]
);
$import->setData('images_base_directory', $this->imagesDirProvider->getDirectory());
$read_file = $this->readFactory->create($importFile['dirname']);
$csvSource = $this->csvSourceFactory->create(
[
'file' => $importFile['basename'],
'directory' => $read_file,
]
);
$this->io->writeln("Validating data...");
$validate = $import->validateSource($csvSource);
if (!$validate) {
$this->errIo->error('Unable to validate the CSV');
$csvSourceCopy = clone $csvSource;
foreach (
$import->getErrorAggregator()->getAllErrors() as $error
) {
$csvSourceCopy->rewind();
$csvSourceCopy->seek($error->getRowNumber());
$current = $csvSourceCopy->current();
$this->errIo->error(
sprintf(
"An error occured processing column %s at line %s with message %s",
$error->getColumnName(),
$error->getRowNumber(),
$error->getErrorMessage()
)
);
$this->errIo->error(
sprintf(
"Offending line %s",
json_encode($current)
)
);
}
rename($importPath, $importPath . '.err.csv');
$this->errIo->error(
sprintf("Erroneous file saved at path %s", $importPath . '.err')
);
return false;
}
$this->io->writeln(sprintf("Importing data from file %s", $importPath));
try{
$result = $import->importSource();
if ($result) {
$import->invalidateIndex();
}
unlink($importPath);
}
catch(Exception $exception)
{
$this->cliLogger->error(sprintf("An error occured while processing the file %s", $importPath));
$this->cliLogger->error($exception->getMessage());
}
The exception I receive when running $import->importSource() is
SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign
key constraint fails (`db_example`.`catalog_product_entity_media_gallery_value_to_entity`, CONS
TRAINT `CAT_PRD_ENTT_MDA_GLR_VAL_TO_ENTT_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` FOREIGN KEY (`entity_id`)
REFERENCES `catalog_p), query was: INSERT INTO `catalog_product_entity_media_gallery_value_to_en
tity` (`entity_id`,`value_id`) VALUES (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?,
?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?, ?), (?,
?), (?, ?) ON DUPLICATE KEY UPDATE `value_id` = VALUES(`value_id`)
In Mysql.php line 91:
SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign
key constraint fails (`db_example`.`catalog_product_entity_media_gallery_value_to_entity`, CONS
TRAINT `CAT_PRD_ENTT_MDA_GLR_VAL_TO_ENTT_ENTT_ID_CAT_PRD_ENTT_ENTT_ID` FOREIGN KEY (`entity_id`)
REFERENCES `catalog_p)
I've confirmed that my importModel has the same data as the importModel when running from the admin, and the file is the one I use in both cases as I take the generated file from my code then upload it via the admin page.
What could be the problem causing this error?
After further debugging I notice that the only difference is in the skuProcessor for \Magento\CatalogImportExport\Model\Import\Product and the \Magento\CatalogImportExport\Model\Import\Product\MediaGalleryProcessor
Apparently skuProcessor is populated during validation of the data, however although it's populated for \Magento\CatalogImportExport\Model\Import\Product it's not for \Magento\CatalogImportExport\Model\Import\Product\MediaGalleryProcessor causing the error because the entity_id cannot be found.
However when doing the import from the admin page, the skuProcessors are in perfect sync for the two classes.
The magic di strikes again it seems
