0

I need to do some test and for my purpose I need a store with more that 50K products so samples data are not enough.

I wonder if there is any way to auto populate a magento store with fake products, even only simple products, what I really need is that the products are working in the front end and so:

  1. assigned to a category ( any one )
  2. enabled
  3. are visible in catalog/search
  4. have an url and a sku
  5. have a random name/sku

because I need many product any direct SQL approach is welcome

3

I've build 2 small scripts for myself a while back to do just that. The project is on Github.

It's only simple products but the rest of the data is random. Images are pulled from a random image API so please take in consideration the amount of bandwidth 50K * ... images will use up. Might be smart do add a small sleep function in there to not overtask the network.

You can use it and if you make any improvements please fork it!

  • 1
    Ha. I saw this script and was just looking for it on github. +1. And about the images, I think it can be changed easily to take a random image from a local folder to avoid all that network issue. One question though. How lonk do you think It will take to call 50k times $product->save(). – Marius Mar 26 '14 at 9:59
  • thank you for the flash answer :) can you provide some basic instruction on how to use it ? – Fra Mar 26 '14 at 9:59
  • I'm getting this error "Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (MAGE_18EX.catalog_product_entity, CONSTRAINT FK_CAT_PRD_ENTT_ATTR_SET_ID_EAV_ATTR_SET_ATTR_SET_ID FOREIGN KEY (attribute_set_id) REFERENCES eav_attribute_set (attribute_set_id" – Fra Mar 26 '14 at 10:07
  • ok it need to set an existing 'attribute_set_id' – Fra Mar 26 '14 at 10:21
  • @Marius It is definitely a good question, on my system (SSD+8GBram+i5) about 1 sec x product so about 13 hours :( ( maybe I can dump the query and create an SQL version of the scritp ) – – Fra Mar 26 '14 at 11:29
2

The Idea
Since the import that magento has built in is much faster than $product->save() how about short-circuiting the import process?

The approach
I mean generate an array with 50k (or more) elements in the format that the import needs it. (not sure exactly what that is yet but I tried with one in the code below).
This should be easy enough, specially if you change a bit the script recommended by @SanderMangel.
Then you can insert it directly in the table importexport_importdata. That is where the data is stored when you press Check file in the import process.
You can insert it all at once but I think inserting chunks of it would work better.
After the insert is done trigger the import process.

The code

Step 1. Build the array.

$toImport = array();
$totalRows = 50000;
for ($i = 0;$i<$totalRows;$i++) {
    $data = array();
    $data['store'] = '';
    $data['_attribute_set'] = 'Default'; //or any attribute set name
    $data['_type'] = 'simple'; //insert only simple products
    $data['_product_websites'] = 'base'; //code for the base website
    //add some required data for the product like stock, tax class & ...
    $data['qty'] = 10;
    $data['min_qty'] = 0;
    $data['use_config_min_qty'] = 1;
    $data['is_qty_decimal'] = 0;
    $data['use_config_backorders'] = 1;
    $data['min_sale_qty'] = 1;
    $data['use_config_min_sale_qty'] = 1;
    $data['max_sale_qty'] = 10000;
    $data['use_config_max_sale_qty'] = 1;
    $data['is_in_stock'] = 1;
    $data['notify_stock_qty'] = 0;
    $data['use_config_notify_stock_qty'] = 1;
    $data['manage_stock'] = 1;
    $data['use_config_manage_stock'] = 1;
    $data['stock_status_changed_auto'] = 1;
    $data['use_config_enable_qty_inc'] = 1;
    $data['qty_increments'] = 1;
    $data['use_config_qty_increments'] = 1;
    $data['enable_qty_increments'] = 0;
    $data['is_decimal_divided'] = 0;
    $data['tax_class_id'] = 2;
    //set the visibility to catalog & search
    $data['visibility'] = Mage_Catalog_Model_Product_Visibility::VISIBILITY_BOTH;
    //set status as enabled
    $data['status'] = Mage_Catalog_Model_Product_Status::STATUS_DISABLED;
    $data['price'] = 9.99; //you can have a function that generates a random price
    $data['weight'] = 1.00; //you can have a function that generates a random weight (can use the same as for price even)
    //add product specific data
    $data['name'] = 'Some name'. $i; //you can generate a random string 
    $data['sku'] = 'some sku'.$i; //you can generate a random SKU
    $data['description'] = 'Some description'.$i; //you can generated a random description
    $data['short_description'] = 'Some short description'.$i; //you can generated a random short description 

    //continue with all the other required attributes if you have any
    //add categories
    $data['_category'] = '...';//not sure how this should look like
    //add images if needed. The images must be placed in `media/import`
    $data['image'] = '/path/to/image.png';
    $data['small_image'] = '/path/to/image.png';
    $data['thumbnail'] = '/path/to/image.png';
    $toImport[] = $data; //add the row to the import data

}

Step 2. Insert the array in the importexport_importdata table

//get an instance of the import model
$importModel = Mage::getResourceSingleton('importexport/import_data');
//clear all data that is already in the table for import
$importModel->cleanBunches();
//add your new data for import. `$toImport` is the array from step 1.
//you can even split the `$toImport` array into pieces of 2000 or something like that using `array_chunk`
$importModel->saveBunch('catalog_product', 'append', $toImport);

Step 3. Trigger the import script

Mage::getModel('importexport/import')->importSource();

I've used a similar script for an import and it seams to work, but I haven't used it on a live environment yet.

The conclusion
I'm not sure this will work for 50K products but it seams like a good place to start.
A reindex may be required when the import is done. Here you may hit an other problem. Indexing 50k products is not that fast.

  • yes I was thinking about using the import functionality as well ... I will give a try to your script, thanks :) – Fra Mar 26 '14 at 12:33
2

For M2 You could try this: http://devdocs.magento.com/guides/v2.1/config-guide/cli/config-cli-subcommands-perf-data.html

Running the command:

magento setup:perf:generate-fixtures /your/www/path/setup/performance-toolkit/profiles/ce/extra_large.xml

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.