Import: Column names have duplicates

Question

Import shows error message "Column names have duplicates" but I don't have any duplicates.

Who can I fix this?

"_type","sku","has_options","name","image","small_image","thumbnail","url_key","url_path","price","manufacturer","status","tax_class_id","visibility","description","short_description","gift_message_available","qty","min_qty","is_qty_decimal","backorders","min_sale_qty","max_sale_qty","is_in_stock","notify_stock_qty","manage_stock","qty_increments","enable_qty_increments"
"simple","1174407","0","BCI-15BK Tintenpatrone schwarz","","","","","","22.000","Canon","Enabled","none","Catalogue,Search","BCI-15BK Tintenpatrone schwarz, 2er-Pack, für I70                    NSTintenpatronenOriginalML","BCI-15BK Tintenpatrone schwarz, 2er-Pack, für I70                    NSTintenpatronenOriginalML","","0","","","0","1","","","","","","0"

Download file

I made sure all of my fields also exist in exported csv. Am I missing some required rows? I tried to check but this wiki page from 2012 requires columens that are not even in the exported file. So I am not sure what are the required columens.

PHP Version is 5.2.17

Answer 1

The exception with the error message you stated is triggered in exactly one place (code slightly truncated for readability):

final public function __construct($source)
{
    $this->_source = $source;

    $this->_init();

    // validate column names consistency
    if (is_array($this->_colNames) && !empty($this->_colNames)) {
        $this->_colQuantity = count($this->_colNames);

        if (count(array_unique($this->_colNames)) != $this->_colQuantity) {
            Mage::throwException(Mage::helper('importexport')->__('Column names have duplicates'));
        }
    }
}

This means, count(array_unique($this->_colNames)) must be unequal to $this->_colQuantity

The $_colNames array is set in Mage_ImportExport_Model_Import_Adapter_Csv::rewind() (which is called during _init().
Once again, the code is slightly truncated for readability:

public function rewind()
{
    // rewind resource, reset column names, read first row as current
    rewind($this->_fileHandler);
    $this->_colNames = fgetcsv($this->_fileHandler, null, $this->_delimiter, $this->_enclosure);
}

The $_delimiter is set to ,, the $_enclosure is set to ".

To try to reproduce the issue, if I copy the CSV extract from your question into a file called test.csv and run the following code:

$f = fopen('test.csv', 'r');
$names = fgetcsv($f, null, ',', '"');    
$qty = count($names);
$uniqueQty = count(array_unique($names));
printf("%d records, %d unique records\n", $qty, $uniqueQty);

it produces the following output:

29 records, 29 unique records

This means, your CSV file is fundamentally okay. The difference must be somewhere else.
Looking at how array_unique works, there is a note how the type handling of array elements changed in PHP 5.2.9: http://php.net/manual/en/function.array-unique.php#refsect1-function.array-unique-changelog

To reproduce the issue, I re-ran my test-script with the SORT_REGULAR option set, but this still gives the same result (which is logical, since reading a file only can give string values).

At this time I believe the difference must be in the CSV file you are using. The Unix and the Windows newline characters (\n and \r\n) are both recognized by the fgetcsv() command, but the old MacOS style newline character (\r) would actually lead to the behavior you are experiencing.

I can't know if that is the reason why you are experiencing the issue, but I suggest you check the CSV file (again). It might also help if you provide a download link somewhere to the unmodified file (no pastebin), so any non-printable characters are preserved.

It might also help if you post the PHP version you are using.

Answer 2

I was receiving this error when editing and saving using LibreOffice on my Mac. Problem was solved by editing the file using Google Drive instead, then the error went away.

I assume it is due to Mac line endings.

Answer 3

If you checked the new-line encoding and still have problems, double-check that you don't actually have duplicate column name in your CSV. In Excel you can quickly highlight duplicates with this technique:

1. Select the range of cells you wish to test (in this case all the cells in the header row).
2. On Excel's Home tab, choose Conditional Formatting, Highlight Cells Rules, and then Duplicate Values.
3. Click OK within the Duplicate Values dialog box to identify the duplicate values.
4. Duplicate values in the list will now be identified.

In my case I had dragged a formula (unintentionally) into the header row and messed up my header.

Answer 4

You can also save the .csv file (using a Mac) as a Windows Formatted .csv. This will fix the duplicate columns error message. I work on a Mac and this is how I get around these import errors. Hope it works for you.