4

We have a program where we are trying to import Customers and Orders using php into Magento from a non-Magento system via csv files.

In a loop we call this function to get a customer model of previously imported customers so that we can attach the current order (from the csv file) to the correct customer:

$customers = Mage::getModel('customer/customer')
               ->getCollection()
               ->addAttributeToSelect('*')
               ->addAttributeToFilter('old_customer_id', $data['SessionID'])
               ->load();

The load of the customer model happens fine, that is not the issue. Our previously imported customers are all there. The issue is that the customer model load above takes about one second each time it's called in the loop that reads the order records from a csv file, which is slow. But the second time we run the program, any customer that was loaded during a previous run loads very quickly, orders of magnitude faster.

Does Magento cache customers during a load() operation or index them or something that causes the second load to be much faster than the first?

If so, can we pre-cache or index customers by the attribute we are searching on 'SessionID' in order to make the customer model load faster?

There are about a million order records to process in the csv file so we need to get better performance to make this approach viable.

2

No, magento doesn't cache. If you want a guess: query cache of mysql

Whatever SessionID is. I assume it is a customer attribute. What you can do is add it as static attribute, then it is added as column to customer_entity and then just put an index on it.

| improve this answer | |
2

This is due to the caching of data from the DB itself. You can test it if you like by disabling the query cache altogether:

SET GLOBAL query_cache_size = 0;

If you get the actual query that is being passed you can issue the query yourself to the DB and test if it is indeed cache by adding SQL_NO_CACHE:

SELECT SQL_NO_CACHE entity_id FROM customer;
| improve this answer | |
  • Yes, 'SessionID' is an attribute that was added programmatically before the customer import: $this->addAttribute('old_customer_id', 'text', 'varchar'); I assumed that it was added using the EAV schema and not just a column in the customer_entity. – drichter Jan 16 '14 at 15:42
  • I also tried turning caching off at the beginning of the script but this had no effect on previously loaded customers continuing to load quickly and non-previously loaded customers continuing to load much more slowly. I tried all of these: // Clear cache. Mage::app()->cleanAllSessions(); Mage::app()->getCacheInstance()->flush(); Mage::app()->cleanCache(); Mage::app(‘admin’, ‘store’, array(‘global_ban_use_cache’=>true)); I will try turning caching off at the database level. – drichter Jan 16 '14 at 15:46
  • I recommend disabling caching at the db level only for testing/benchmarking purposes. Best of luck. – philwinkle Jan 16 '14 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.