Magento 1: most efficient way to bulk add entities

Question

So I'm working on a CRUD module that creates entities based on an XML feed.

Right now, I'm using Magento system, so I have an array of data that I use to populate my object then call the save() method for my entities. Code example:

foreach ($datas as $data) {
    $entity = Mage::getModel('module/entity');
    $entity->setData($data);
    $entity->save();
}

Now, I want to know if there is a most efficient way of doing it (in terms of performance). Things that come to my mind are:

• Direct SQL queries ? Assuming I deal with one single table here.
• Collection walk method ? Just an idea I'm not sure if it's even possible but it would be something like populate a collection of data and use the walk iterator to generate the entities.

Feel free to provide a code example based on my snippet above.

Of course, this list is not exclusive, if you have more ideas I'd be willing to try them.

Answer 1

If your entities are not linked to anything else, and the module is for your use only, you can try it with direct inserts. It's certainly the fastest way to do it.

$resource = Mage::getSingleton('core/resource');
$adapter = $resource->getConnection('core_write');
$table = $resource->getTableName('table/name_here');
$adapter->insertOnDuplicate($table, $dataToInsert);

Where $dataToInsert looks like this:

$data = array(
    array(
       'column1' => $value11,
       'column2' => $value12,
       ....
       'columnN' => $value1N,
    ),
    array(
       'column1' => $value21,
       'column2' => $value22,
       ....
       'columnN' => $value2N,
    ),
    ....
    array(
       'column1' => $valueM1,
       'column2' => $valueM2,
       ....
       'columnN' => $valueMN,
    )
);

Using insertOnDuplicate you also make sure that if you have unique keys the new values will override the old values.

If your module is for commercial use you would want to go through the save method in order to have events dispatched if needed.