6

How can I transfer all products from one shop to another after clicking a button?

We have a shop which is a copy of our real shop. We need to keep the products, prices, etc. synchronized in both shops.,

The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live

It would be best if we could just press a button which starts the import process.

Improve this question
10
  • sounds like a use-case for using the multi-store capability of magento rather than having two separate instances that need to be synced.
    – BlueC
    Commented Jun 14, 2019 at 12:37
  • @BlueC, thats not possible since both shops are using different code (different git branches checked out)
    – Black
    Commented Jun 17, 2019 at 7:40
  • 1
    What's different about the code? Can you unify the changes using themes/child themes etc? Magento is built to do this.
    – BlueC
    Commented Jun 17, 2019 at 12:37
  • 1
    Hmmm, templates and scripts can all be utilised per store quite easily. Extensions can be a pain because they can conflict. Observers could/should be written to be configurable per store view. I am not saying its impossible to synchronize the products, but it doesn't seem a very elegant solution, and longer term having a single store with unified products might be preferable. Does sound like it might be tricky to untangle though.
    – BlueC
    Commented Jun 17, 2019 at 14:54
  • 1
    Reading below it seems you don't actually want or need 2 storefronts, but you just want a staging and production site and to be able to sync from production to staging. This wasn't clear in your question, and in this case using the multi-store setup in Magento is not appropriate.
    – BlueC
    Commented Jun 19, 2019 at 8:09

3 Answers 3

Reset to default
7
+25

(Reading the comments, it seems that you're trying to create a way to easily and reliably synchronize your production database to your staging site. Perhaps you could update the question to make this more clear.)

This is a common problem and having a tool to do this reliably is really beneficial. I have written my own bash script to do it and this is the basic process you should be trying to replicate:

  1. Dump the production database to a gzip tarball
  2. Download the dumped tarball to your staging server
  3. Drop and recreate the staging DB
  4. Import the production dump into the staging DB
  5. Adjust any configuration as appropriate for your staging server (mainly changes in core_config_data such as web/unsecure/base_url and web/secure/base_url etc)

I would suggest as a first step to get this down to a set of commands that you can reliably run to give you the desired result. Then once you have the process refined you can think about automating it with a simple script.

The tool that is really going to help you here is n98-magerun which is a priceless CLI tool for the Magento developer.

Breaking down the steps above into n98-magerun commands:

1) Dump the production database to a gzip tarball

On your production server:

n98-magerun db:dump -c "gzip" ~/latest-production-db.sql.gz;

Consider also using the --strip option to remove sensitive data such as customer and order records. Depends on your use-case. See n98-magerun docs for Stripped Database Dumps.

2) Download the dumped tarball to your staging server

On your staging server:

Use rsync or scp or whatever CLI tool is most convenient or familiar to download the tarball created above.

Example using scp:

scp user@production:~/latest-production-db.sql.gz ~/latest-production-db.sql.gz

3) Drop and recreate the staging DB

On your staging server:

n98-magerun db:drop && n98-magerun db:create

4) Import the production dump into the staging DB

On your staging server:

n98-magerun db:import -c "gzip" ~/latest-production-db.sql.gz

5) Adjust any configuration as appropriate for your staging server

On your staging server:

n98-magerun config:set web/secure/base_url https://staging.domain.tld/
n98-magerun config:set web/unsecure/base_url https://staging.domain.tld/

Use the same config:set command for any other configs that need setting. Note that config:set can be scoped to a store or view. See the n98-magerun readme. Once you start to automate this process I highly recommend the HarrisStreet-ImpEx module for n98-magerun for managing your environment configs.

Improve this answer
6
  • Thx, I already know this method, but it is not what I want, it should not drop the whole database, I need to keep a part of it (CMS sites, sliders, static blocks etc.), I only need to synch the products, not the whole database. And I want this all to happen after a single button click, no manual steps are allowed, since people without developer experience should also be able to trigger the sync e.g. our marketing or testing team.
    – Black
    Commented Jun 19, 2019 at 9:21
  • Sure I could create another test environment which is only for our marketing so they can create CMS sites, static blocks, sliders etc, but unfortunatelly the team does not want another site.
    – Black
    Commented Jun 19, 2019 at 9:24
  • 1
    You could use --strip on the n98-magerun db:dump command to strip out any tables you don't want to sync. You'd then need to replace dropping and recreating the staging database with a MySQL command that drops only the tables you want to drop. All this could fairly trivially be automated into a bash or php script and then create some way of triggering it from the admin UI. You'd probably want to implement a lock file so it can't be run more than once simultaneously.
    – BlueC
    Commented Jun 19, 2019 at 9:33
  • 1
    You could also use --include on the n98-magerun db:dump command to explicitly include only the tables you want. See n98-magerun db:dump --help
    – BlueC
    Commented Jun 19, 2019 at 9:38
  • 1
    I'd probably recommend excluding the one's you don't want to overwrite rather than including only the ones you do want. It will be easier to maintain and cleaner. I do wonder whether you might come unstuck if some of the data you want to keep uses any of the EAV tables because you're going to need these tables for product data. However, I think that the CMS stuff is quite self-contained in the DB. I would also recommend writing this as some type of extension that shows a notice in admin and frontend when a sync is in progress, because the store will be in a volatile state until it completes.
    – BlueC
    Commented Jun 19, 2019 at 10:30
6

We've got a similar setup and scenario. We went with a database level sync of product based tables. It works by dumping and importing tables.

The copy site is down for about 15 mins in early hours.

It was trial and error over which tables to sync.

Watch for eav tables though. There is a table which stores increment ids for orders, invoices etc. You dont want to sync this.

It was a very bespoke scenario and that worked for us. That's about as much info as I can give you at this stage. Good luck.

Improve this answer
6

not sure how your copy store works, but you can use many options:

  1. Magento Sync Products paid extension
  2. Odoo connector Example free connector/ Odoo bridge Example Webkul paid extension
  3. Magmi Magento Mass Importer
  4. Store manager for magento
  5. Magento Dataflow
Improve this answer
2
  • Thx! The copied store is just a staging server where we deploy new features/bugfixes so we can test it before it goes live.
    – Black
    Commented Jun 19, 2019 at 6:55
  • so technically you dont really need to sync content back to staging, you need to sync/track file changes, like using git to redeploy code updates and track database changes to extract only changes related to your specific modifictions. using percona-toolkit
    – MagenX
    Commented Jun 23, 2019 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.