1

We have a third party app that allows users to manage their catalog. The app uses the Magento API to fetch catalog data but it is very slow and it offers no pagination.

I want to cache the catalog database in this app's local database to avoid making API calls on every page load. The local database should remain in sync with the magento database but I do not require all of the catalog tables (and certianly not the EAV structure) to be copied. Because of this I'm thinking the sync process should be event-based and not rely on database replication tools. Both apps have APIs that can be called to update info on both sides.

Edit: I would be updating one product at a time and not the entire catalog. When a product changes on one said, I'd make an API call to the other side, sending information of what changed.

These are the events that I think I would need to watch to assure 100% sync:

In Magento (afterward, send update to external app)

  1. Product is created/edited in the Magento Admin Panel.
  2. A purchase is made and inventory is modified.

In external app (afterward, send update to Magento)

  1. Product is created/edited from the app.

I think this should be enough to keep the information synced. Otherwise I'll have to do some cron jobs from both apps to compare against the other app every now and then. Or something of the sort.

Question: Can anyone see something potentially critical that I've overlooked? Are there more events I should be aware of? Has anyone done something like this?

Improve this question

1 Answer 1

Reset to default
1

I don't know how often this remote database needs to be updated, but I would be careful with the events you want to put in place. Make sure those calls are asynchronous, for example a cron would probably be your best option. You don't want your backend to slow down, and even less your frontend!

If you want a more "user-friendly" solution, you can create a new "Index", and have it triggered whenever you like.

Hope it helps!

Improve this answer
5
  • Well I would be updating one product at a time, not the entire catalog. I do understande the concern regarding the frontend though, but the only time the frontend would be affected would be on add to cart. Maybe that should be the only one that's asynchronous. We don't mind adding an extra second or two to the same operations in the backend as long as they're immediate. I feel that updating the entire catalog on scheduled tasks will be cumbersome, consume more resources and bandwidth, and remove that "immediate" effect.
    – Vic
    Commented Aug 18, 2015 at 14:22
  • 1
    I wasn't talking about the whole catalogue either, don't worry. Although, any call to a remote server can slow you down by an awful lot (depends on how responsive the other end is). That's why, even in the backend, I don't think it would be your best option. It wouldn't be as "immediate", but if you have a cron running every 5 minutes, it could really do the job
    – Fran Mayers
    Commented Aug 18, 2015 at 19:22
  • Oh I see. I'll look into this further then. Thanks for your insight. I'll update if I figure out how to make this.
    – Vic
    Commented Aug 18, 2015 at 20:45
  • I imagine I will have to keep a table that will act as a queue to hold the products that have been updated recently and that need to be synced? Or was something else on your mind?
    – Vic
    Commented Aug 18, 2015 at 20:46
  • Yes that's exactly what I would do
    – Fran Mayers
    Commented Aug 20, 2015 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.