1

What's the best way of storing calculation results in Magento ?

Example: I need to calculate total number of active products in each category and show the results in the menu, for that purpose, I used some scripts that extract enabled,in stock, visible and available products within database. So for all categories (say 100 categories in total with thousands of different products) it takes time (something about 2-3 seconds) for each page load.

Say we have ten thousand of viewers every day in the site and speed matters. There are some scenarios I find out:

1. Using cache:

foreach( category )
{
   if( number of products in this category is not available in cache )
   {
       calcualte it and store the result in cache;
       show the calculated number;
   }
   else
   {
       show the calculated number from cache;
   }
}

In this solution, for each category it takes about 0.01s for the calculation and just after that we are saving the results in the cache. Other users won't recalculate it every time and for simultaneous users I myself think we won't be in trouble. What's your opinion ?

2. Using cron jobs:

Sometime we can do the calculation in a cron job and for each category save the results in a table and then retrieve the calculation from there in the front-end. Cron job will be executed every 10 minutes for example.

I think this solution is ok for heavy calculations , but for calculations like products count its not the case. What do you think ? Is it necessary to do this kind of tasks in a cron job even if it takes 0.01s seconds ? Will it makes trouble for simultaneous users if it calculated on the fly ?

3. Using event observers

We can write an event to re-calculate number of products every time we save, update, create or delete a product/category and then store the results in the table. What do you think about this idea ?

I myself currently don't have any idea how should I implement this for this example but I think it could be a little challenging. Any suggestions about how to do it is welcome.

Let me know if there is any other solutions available out there.

2

This is kind of opinion based. each approach has advantages and disadvantages.

  1. Using cache

    • +: Short implementation time, do it once and don't care about it
    • -: Some of the customers might get slow pages. Each time you clear the cache this needs to be rebuilt.
  2. Using cron jobs

    • +: Transparent to the user. You can recalculate the values at points of day when the traffic is low.
    • -: Users might see wrong results between cronjob running.
    • 0: Medium implementation time.
  3. Using event observers

    • +: Transparent to the user. Results are always correct.
    • -: It takes a little more to implement than the other 2. Could slow down the work for admins.

I used all 3 of these approaches (for different things) and I found that the first one was a disaster. Mostly because once you tell the client to clear the cache for something he will do it for everything. So each time the client changes a static block or page and sees an invalid cache he will clear all of them...you see where I'm going with this.

The observers and crons are good but depend on each case.
For example I used the cron approach because I needed to update some values once a day, after a big import was done. It makes no sense to have an observer here. It will really slow down the import process.

For actions taking a few seconds I would use observers...until the admin complains that the backend is slow :).
Now seriously, for actions that are done multiple times in a short period I would use cronjobs (or better yet, command line indexing). After you are done with the backend changes just run a script (either cron or cli).

For actions that are done rarely and the recalculation does not take ages, I would use observers.

  • +1 Great explanation, What do you suggest for the example I just wrote ? – Michel Gokan Jul 15 '14 at 14:17
  • @Miko. I would go with the cron (or command line indexing) if there are not many changed to the products and categories during one day. You can schedule the cron at night. – Marius Jul 15 '14 at 14:18
  • But isn't the event observer a better solution ? As we may change products 2 or 3 times during the day ? And isn't it more clean way to do it ? – Michel Gokan Jul 15 '14 at 14:20
  • @Miko Like I said in the answer, the best approach can be different from case to case. If you say it does not take long to process the data (2-3 seconds) you can do it in an observer also. You know better what fits you best. Actually to summarize my answer in one sentence..."Do not use the cache approach" :). – Marius Jul 15 '14 at 14:23
  • Do you know any other solutions other than the ones I just mentioned ? – Michel Gokan Jul 15 '14 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.