2

I'm magrating to Magento, and I have a large amount of customers to import. I am using the dataflow profiles because more fields are available. The problem of importing dozens of thousands of customers by running the profile in a popup is that it will time out and fail.

I am then using a script to run a profile from the command line. It works fine and is MUCH faster than using the browser, however, I am facing a strange problem. Running the profile from the browser reads the "is_subscribed" fields and subscribes the customer to the newsletter is the value equals 1, but it won't subscribe the customer if I run the profile from the command line. Everything else imports fine from the command line. All the data is there, except the newsletter subscription which seems to be ignored.

I tried to change the file format of the import from CSV to XML, ran the script using root access, ... nothing, it will never subscribe the customer to the newsletter if running the script from the command line.

Here is a sample of the csv file with one customer that imports fine from a browser but not from the command line : website,email,group_id,firstname,lastname,billing_street1,billing_street2,billing_city,billing_region,billing_country,billing_postcode,billing_telephone,created_in,is_subscribed,group base,johndoe@gmail.com,group01,John,Doe,555 west street,,Los Angeles,CA,US,90001,5551237894,en,1,group01

Any input on what to do/look for ?

Thanks

Code:

<?php
/* ONLY WORK VIA CRON OR SSH: /usr/bin/php -f /var/www/magento/cron_import.php */
/* http://www.magentocommerce.com/boards/viewthread/35865/ */
if ($_SERVER["SERVER_ADDR"] == $_SERVER["REMOTE_ADDR"]) {

  $profileId = 7;
  $logFileName= 'import-profile-7.log';  
  $recordCount = 0;

  require_once 'app/Mage.php';

  Mage::app();

  Mage::log("Starting",null,$logFileName);  

  $profile = Mage::getModel('dataflow/profile');
  $userModel = Mage::getModel('admin/user');
  $userModel->setUserId(0);
  Mage::getSingleton('admin/session')->setUser($userModel);

  if ($profileId) {
    $profile->load($profileId);
    if (!$profile->getId()) {
      Mage::getSingleton('adminhtml/session')->addError('ERROR: Incorrect Profile');
    }
  }

  Mage::register('current_convert_profile', $profile);

  $profile->run();

  $batchModel = Mage::getSingleton('dataflow/batch');
  if ($batchModel->getId()) {
    if ($batchModel->getAdapter()) {
      $batchId = $batchModel->getId(); 
      $batchImportModel = $batchModel->getBatchImportModel();
      $importIds = $batchImportModel->getIdCollection();  
      $batchModel = Mage::getModel('dataflow/batch')->load($batchId);      
      $adapter = Mage::getModel($batchModel->getAdapter());
      foreach ($importIds as $importId) {
        $recordCount++;
        try{
          $batchImportModel->load($importId);
          if (!$batchImportModel->getId()) {
             $errors[] = Mage::helper('dataflow')->__('WARNING: Skip undefined row');
             continue;
          }
          $importData = $batchImportModel->getBatchData();
          try {
            $adapter->saveRow($importData);
          } catch (Exception $e) {
            Mage::log($e->getMessage(),null,$logFileName);          
            continue;
          }      
          if ($recordCount % 50 == 0) {
            Mage::log($recordCount . ' completados',null,$logFileName);
          }
        } catch(Exception $ex) {
          Mage::log('Fila ' . $recordCount . ', SKU ' . $importData['sku']. ' - ERROR: ' . $ex->getMessage(),null,$logFileName);        
        }
      }
      foreach ($profile->getExceptions() as $e) {
        Mage::log($e->getMessage(),null,$logFileName);          
      }
    }
  }
  printf("OK\n");
  Mage::log("Completed",null,$logFileName);
}
?> 
  • What script / code are you using to run this from command line? – Melvyn Nov 27 '14 at 7:49
  • Hi Melvyn, I tried to copy/paste script here but it is too long. I'm checking on how to post it correctly on this forum. – Calman Nov 27 '14 at 17:36
  • Edit your post or use something like pastebin. – Melvyn Nov 27 '14 at 17:41
  • Melvyn, you can see the script here for now : Hi Melvyn, you can download the script here : onedrive.live.com/… I run it using the plain "php myscript.php". Thanks – Calman Nov 27 '14 at 17:46
  • I couldn't tell you why it works from the backend. I can't find any special handling defined for is_subscribed there and Mage_Customer_Model_Convert_Adapter_Customer::saveRow simply sets it on the customer model, which looks to be wrong. I don't have a is_subscribed attribute in eav_attributes table in a stock installation with sample data, and when you look at app/code/core/Mage/Customer/Block/Account/Dashboard/Info.php then subscription info is handed off to Mage_Newsletter_Model_Subscriber, while this passes it on to Eav attributes. I'll see if I can come up with a work-around. – Melvyn Nov 27 '14 at 20:20
0

Ok, found the answer after discovering subscribeCustomer().

In the newsletter module we see the customer_save_after event defined for two scopes: frontend and adminhtml. The code tries to mimic this by instantiating an admin session, but from the looks of it, it's not enough for the event to fire.

Aside from some cleaning and type hints, see if the following works for you and carefully read the comment I put in:

<?php
/* ONLY WORK VIA CRON OR SSH: /usr/bin/php -f /var/www/magento/cron_import.php */
/* http://www.magentocommerce.com/boards/viewthread/35865/ */
$profileId = 7;
$logFileName= 'import-profile-7.log';
$recordCount = 0;

require_once '../app/Mage.php';

Mage::app('admin');
Mage::log("Starting",null,$logFileName);

/** @var Mage_Dataflow_Model_Profile $profile */
$profile = Mage::getModel('dataflow/profile');

if ($profileId) {
    $profile->load($profileId);
    if (!$profile->getId()) {
        Mage::throwException('ERROR: Incorrect Profile for id ' . $profileId);
    }
}

Mage::register('current_convert_profile', $profile);

$profile->run();

/** @var Mage_Dataflow_Model_Batch $batchModel */
$batchModel = Mage::getSingleton('dataflow/batch');
if ( ($batchId = $batchModel->getId()) ) {
    if ($batchModel->getAdapter()) {
        /** @var Mage_Dataflow_Model_Batch_Import $batchImportModel */
        $batchImportModel = $batchModel->getBatchImportModel();
        $importIds = $batchImportModel->getIdCollection();
        /** @var Mage_Dataflow_Model_Batch $batchModel */
        $batchModel = Mage::getModel('dataflow/batch')->load($batchId);
        /** @var Mage_Customer_Model_Convert_Adapter_Customer $adapter */
        $adapter = Mage::getModel($batchModel->getAdapter());
        foreach ($importIds as $importId) {
            $recordCount++;
            try{
                $batchImportModel->load($importId);
                if (!$batchImportModel->getId()) {
                    $errors[] = Mage::helper('dataflow')->__('WARNING: Skip undefined row');
                    continue;
                }
                $importData = $batchImportModel->getBatchData();
                try {
                    $adapter->saveRow($importData);
                    if( array_key_exists('is_subscribed', $importData)  )
                    {
                        /** @var Mage_Newsletter_Model_Subscriber $subscription */
                        $subscription = Mage::getModel('newsletter/subscriber');
                        /*
                         * Theoretically this should work, but the code sets
                         * "importMode()" but does not really do anything with
                         * it, so chances are email is sent.
                         * You may want to disable email communications
                         * while running mass imports.
                         */
                        $subscription->subscribeCustomer($adapter->getCustomer());
                    }
                } catch (Exception $e) {
                    Mage::log($e->getMessage(),null,$logFileName);
                    continue;
                }
                if ($recordCount % 50 == 0) {
                    Mage::log($recordCount . ' completados',null,$logFileName);
                }
            } catch(Exception $ex) {
                Mage::log('Fila ' . $recordCount . ', SKU ' . $importData['sku']. ' - ERROR: ' . $ex->getMessage(),null,$logFileName);
            }
        }
        foreach ($profile->getExceptions() as $e) {
            Mage::log($e->getMessage(),null,$logFileName);
        }
    }
}
printf("OK\n");
Mage::log("Completed",null,$logFileName);
  • I will test it tomorrow Melvyn. Not sure where you are, but it's Thanksgiving today, and I have to go! I'll keep you posted and thank you again. – Calman Nov 27 '14 at 20:53
  • Melvyn I gave it a try, and the imported customer still is unsibscribed. It subscribes the customer if importing from the backend (same file, same profile, no change here). As far as the email, it only sends an email if you subscribe the customer from the "Manage Customer" on the backend, then edit the customer, and click the "Subscribe to Newsletter" and save. Importing from the backend does not send an email, and if this script can be fixed and the only thing I have to do is to deactivate the email to subscriber, I know how to do this. Thank you again. – Calman Nov 27 '14 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.