27

I'm currently seeing that for a database table with a very simple schema (about 5 fields), it's inserting new records at a rate of just under ~50 inserts / second, in my local development environment (SSD drive) - this is with no observers on the model populating associated tables.

Using direct SQL I'm seeing quite an improvement - ~1800 inserts / second. We're thinking about attempting to optimize the performance of our models, but of course we don't want to lose all of the nice stability and flexibility that the Magento core gives us.

I'm wondering whether someone has gone this route before and whether there are some easy wins in terms of components of the model layer that can be bypassed relatively safely that will give significant performance boosts.

Things like:

  • Class name resolution
  • before and after save events
  • Event dispatches
  • Transactions
  • etc.

UPDATE: I lied, there actually were some additional queries firing off of observers or afterSave()'s, which I saw when I inspected the database query log. Benchmarking against a totally simple entity actually gives me ~300 rows / second with Magento models - only MySQL overhead are transactions.

  • 1
    Have you tried reusing the model object to write the data? I.e. clear it, setData and then save. This would avoid the calls to getModel and the object instantiation overhead inherent to PHP. – davidalger Jan 23 '13 at 19:22
  • Also, I'm going to guess that the bottleneck here is in your CPU and not the drive...as all the necessary code files will be loaded on the first pass through. – davidalger Jan 23 '13 at 19:23
  • Thanks, David! I'll try that as well. Actually I think we're still I/O bound by the number of queries that are being executed. We have about 20 queries that are running for a given model save - some of which we need to keep (populating associated tables, SELECT's to check for existence prior to save), and others that we can probably remove (extraneous session saves, additional load()'s that can be avoided in application logic) – kalenjordan Jan 24 '13 at 0:56
  • You could easily find out. Mount the whole root dir and MySQL DB on a RAM disk. But I would strongly doubt I/O is an issue in server grade equipment. You'll probably see more benefits merely disabling "Index on Save". – Ben Lessani - Sonassi Jan 30 '13 at 16:37
17

One thing which can speed up the whole site is to remove all references to Varien_Profiler on your production site. Even if the profiler is disabled, it always checks whether it is enabled so every call to Varien_Profiler:: will result in an additional if statement. Of course, removing all these calls comes at the cost of not being able to use the profiler anymore. However, this can speed up the whole site by maybe 5% or so (this is a subjective eperience, but there are MANY calls to Varien_Profiler all over Magento). I actually wrote a little shell script to automatically comment out these calls in all the files and I will add this to my post tomorrow when I am at work and have my code ready.

As promised now the code to comment out these calls:

grep -l "Varien_Profiler" * -R > profiler.txt 
for x in `cat profiler.txt` 
do 
sed -i '/Varien_Profiler/s/^/\/\//' $x
done

This should be run in the linux console both in the app/ as well as in the lib/ folder. You might need to adjust the file /lib/Varien/Profiler.php manually afterwards. Also note that you should test this thoroughly in a safe environment before taking it live - but I guess this should be obvious ;)

  • Wow! I would have never imagined anything even close to 5% just for the Varien_Profiler calls when disabled. I'll check that out, thanks! – kalenjordan Jan 24 '13 at 0:24
  • @sparcksoft As promised, I added the code now. – mpaepper Jan 24 '13 at 9:23
  • 1
    That is where C precompiler conditionals are really nice. It's too bad that PHP doesn't have them, but of course, that would mean it would have to have it's own method of pre-compiling and caching built-in. :) – davidalger Jan 24 '13 at 14:23
  • 2
    You can also write it find . -type f -exec grep -qF 'Varien_Profiler' {} \; -exec sed -i '/Varien_Profiler/d' {} \; if you prefer a fast oneliner. – kojiro Jan 24 '13 at 19:51
14

When executing a lot of saves on Magento models it's best to disable the Magento indexer which slows the process down:

$processes = Mage::getSingleton('index/indexer')->getProcessesCollection();
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_MANUAL));
$processes->walk('save');

And enabling it when your done:

$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_REAL_TIME));
$processes->walk('save');
  • Ah gotcha, nice. So that would be like if you were saving a lot of customer_entity records and wanted to avoid customer indexing happening for each save? In my case I'm actually doing this against a custom entity that doesn't have any indexing - well at least for the benchmark I did. We do have some custom indexes as well that I will probably use this tip on! – kalenjordan Jan 23 '13 at 19:04
  • I don't think there's any customer indexing but it will sure help you when modifying a lot of products and such. Either way it's worth a try! – Rick Kuipers Jan 23 '13 at 19:07
  • Sorry, yes EAV product data and such for example. Thanks. – kalenjordan Jan 24 '13 at 0:26
  • Hey! You might have to (partially) reindex after that? – Alex Jan 24 '13 at 10:02
  • @Alex Yes, the indexers get reset to MODE_REAL_TIME so it will reindex by schedule. You can, of course, force it if you want. – Rick Kuipers Jan 24 '13 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.