I plan to undertake a large project within the next 30 days. I will be dealing with around 8 Million simple products. I think my best plan is to work out the infastructure I'm using, and address any bottlenecks within the stack. Once I recieve the products, I plan to spin up a small Mage instance to see what kinds of issues I run into, then create the final Mage install on a larger server.

I plan to base this on Magento 1.9 - having used Magento 2 I feel it's not ready for such a project as I don't want to have to deal with issues relating to an upcoming platform.

The products won't have images for the time being, but in the near future this will be a needed expansion. My reasoning for this is I will be listing products on large marketplaces (Amazon, Ebay...etc) and will not be running a storefront for the time being.

My main areas for concern is DB load due to the amount of products, PHP being single threaded, and CPU IOPS. I know I can offset some of this using Redis as a database store, caching, SSDs...etc. I am able to utilise an hourly stock feed, which will only include products which require an update, however with that many products I'd imagine there is a lot of change in stock - which means the stock feed will still be large.

I'm leaning towards using AWS, due to it's scalability and ability to split up different sections of my setup, depending on what needs more resources. I haven't had a chance to use AWS infastructure before, but from reading documentation from RDS and S3 buckets, it looks like something that would tick most of the boxes.

If you were in my position, would your go-to be AWS or another provider? If you would use AWS, would you use S3/RDS and why?

  • i would close this as off-topic, as you are going to use third-party extensions. by default magento dont have any ability to offload its products to external marketplaces. – MagenX Apr 8 '17 at 14:39
  • @MagenX How is this off topic? Asking a question about Magento 1 and it's bottlenecks with a large product range (like my scenario) surely relates to Magento, does it not? I don't see the significance in what extensions I plan to use – PublicDisplayName Apr 8 '17 at 14:42
  • Why would you choose 1.9 over 2.1? There was a B2B site with this amount of SKU's featured at Imagine. What in version 2.1 is not ready? – brentwpeterson Apr 24 '17 at 13:43
  • @brentwpeterson I had a nightmare importing configurable products into M2.1.2, not to mention other issues with the platform. I'd much rather go with something tried and tested – PublicDisplayName Apr 24 '17 at 16:40
  • You can see setup a Magento2 store and import configurable products all in under 45 minutes. magento.com/resources/… (At Imagine 2016) – brentwpeterson Apr 24 '17 at 17:12

Browse other questions tagged or ask your own question.