22

We are preparing to launch 2 e-commerce sites using CE ver 1.9. Each site will begin with around 10000 sku's.

What are the key things we need to look for from a web host...are things such as database size, number of images, etc.. important?

Obviously, page load speed is very important also.

How do we best select a proper host?

Thank you

  • Welcome water on MageOverflow. Unfortunately Magento hosting is a complex thing and there are a lot of companies out there which invite you to discuss with them your requirements. There is no way to recommend you anything with out a lot of more informations. And with all the informations the question is too specific to be helpful. – Fabian Blechschmidt Nov 13 '14 at 14:45
  • @FabianBlechschmidt - I don't think is a duplicate question of the one mentioned. And its a fairly reasonable question at that. I've wrote up a fairly thorough answer about sizing requirements (I'll bulk it up with more detail too) and its generic enough for this to serve as a really good reference question for architecture sizing. – Ben Lessani - Sonassi Nov 13 '14 at 19:00
  • You are the expert, so I voted for reopening :-) – Fabian Blechschmidt Nov 13 '14 at 21:23
31

Disclaimer: Sonassi is a Magento host


TL;DR - A good host will tell you what exactly what you need (and justify it). You shouldn't be doing this yourself.

When a customer approaches us, we ask a few questions to understand what the current requirements, peak requirements and future requirements are for their store. Then we'll provide a recommendation on that basis.

Given we host thousands of Magento stores, we have a fairly solid understanding of exactly what infrastructure is required for a given set of customer requirements, so its pretty easy for us to give a recommendation (and prediction of exactly what resources will be used) when they actually go live.

The detail to note here is that whilst we can certainly tell you the requirements for our own infrastructure running MageStack - that almost certainly won't quite be the same as a set up elsewhere - so just bear that in mind.

Let someone else size your requirements

Store and Catalogue

  1. Magento version?
  2. Number of simple products in catalogue?
  3. Number of categories in catalogue?
  4. Number of attributes in catalogue?
  5. Number of attribute sets in catalogue?
  6. Number of Magento store views (Admin > System > Manage Stores)?
  7. Transactions per day?
  8. Peak transactions in a single hour?

Traffic and Bandwidth

  1. What is the level of daily unique visitors?
  2. What is the highest peak in a single hour of unique visitors?
  3. Number of page views per visitor?
  4. What country are visitors predominantly from?
  5. Do you anticipate the site traffic growing over the next 12 months, if so, by how much?
  6. Do you operate high-traffic campaigns/newsletters (that have significant uptake) on a regular basis?
  7. Does your site offer digital downloads?
  8. Current bandwidth usage?
  9. Do you require a dDOS filtering service?

Hardware Requirements

  1. Current disk space usage?
  2. Do you require long term log storage (PCI-DSS compliance)?
  3. Do you require off-site backup storage?
  4. Do you need to run any specialist/alternative software on the server?
  5. Do your PCI compliance regulations dictate the use of a hardware firewall?
  6. Do you need a fault-tolerant, highly available or load-balanced solution?

Staging/Development Practice

  1. Do you need a separate dedicated environment for staging/development?
  2. Do you require a “Test Live” environment to match the “Live” environment (for final pre-live testing)?

Budget

  1. Do you have a monthly budget?

Then using this information, send it to your hosting provider of choice and see what they recommend.


Sizing up your own requirements

As a rule of thumb, it is possible to

  1. Directly correlate your traffic levels to your CPU requirements
  2. Directly correlate your CPU requirements to RAM requirements
  3. Directly correlate your MySQL storage to RAM requirements

CPU Selection

First start by sizing up your traffic.

  • A standard Magento demo store is capable of delivering roughly 230 uniques per GHz, per hour.
  • A typical web store, with admin user activity, development activity, product addition/deletion can see this degrade by around 100%, to 115 uniques per GHz, per hour.
  • A store with a poorly built/heavy template can further reduce the figure by another 100-200%, to 50 uniques per GHz, per hour.

Using these numbers, you can work out exactly what you need in terms of CPU resource.

Eg. If you have 4,025 sustained unique visitors/day - you need 28GHz total CPU resource (ie. 8 Cores @ 3.5GHz, or 12 Cores @ 2.3GHz)

The other important detail is the speed of the CPU, it is possible to have:

  • A slow page load time and low concurrency support (low clock speed CPU (GHz), few cores)
  • A fast page load time, but low concurrency support (high clock speed CPU (GHz), few cores)
  • A slow page load time, but high concurrency support (low clock speed CPU (GHz), lots of cores)
  • A fast page load time and high concurrency support (high clock speed CPU (GHz), lots of cores)

RAM Selection

For a standalone server (the best configuration for Magento), the rule here is 2GB RAM/CPU Core. So if you have 8 cores, then 16GB RAM should be the minimum.

To work out what you need in addition to this, you need to factor in the size of your catalogue. Doing this is easy, multiply your number of store views against the total catalogue size.

Eg. In your case, 1 store view * 10,000 products = 10,000

Our recommendation is,

<1,000    = 8GB   RAM
<50,000   = 16GB  RAM
<100,000  = 32GB  RAM
<100,000  = 32GB  RAM
<250,000  = 48GB  RAM
<500,000  = 64GB  RAM
<750,000  = 96GB  RAM
>999,999  = 128GB RAM   

You want to take the higher of the two numbers (ie. from the CPU Core:RAM ratio and catalogue requirements), or a sensible mix of the two.

HDD Selection

Lets make something clear, for the average Magento store (ie. <50k unique visitors/day), Magento is not bound by I/O - SSDs will not make your Magento store faster; it won't improve concurrency and it won't improve TTFB. Similarly, using striped RAID levels (eg. RAID10) will also provide no advantage (as almost every file on disk is going to be smaller than the stripe size anyway, so only a single disk will be used).

The only time storage is a bottleneck is on contended services (ie. cloud/VPS).

However, there are certainly advantages to fast I/O beyond the store operation itself. With SSDs, file operations (eg. Git/SVN status, backup/restore, copying directories etc.) are substantially faster. Your developer's life will be substantially easier (with routine tasks being much quicker).

Also note that not all disk drives are created equal.

  • Cheap/desktop-grade SSDs will perform slower than conventional HDDs
  • Cheap/desktop-grade HDDs will perform slower than enterprise HDDs
  • Cheap/desktop-grade HDDs will have poor NRE rates (usually 10^14) compared to enterprise disks (usually 10^16)

So be sure to actually pick disk drives that deserve to be in a server, Ie. the Intel DC S3700.

Picking capacity is easy, you just need two commands,

For the Magento document root

cd /path/to/magento/installation 
du -hsL . \
--exclude="/var/log" \
--exclude="/var/session" \
--exclude="/var/cache" \
--exclude="/var/full_page_cache" \
--exclude="/var/report" \
--exclude="/var/tmp" \
--exclude="/includes/src/"    

For the MySQL DB

SELECT 
IFNULL(B.engine,'Total') "Storage Engine", CONCAT(LPAD(REPLACE(FORMAT(B.DSize/POWER(1024,pw),3),',',''),17,' '),' ',SUBSTR(' KMGTP',pw+1,1),'B') "Data Size", CONCAT(LPAD(REPLACE(FORMAT(B.ISize/POWER(1024,pw),3),',',''),17,' '),' ',SUBSTR(' KMGTP',pw+1,1),'B') "Index Size", CONCAT(LPAD(REPLACE(FORMAT(B.TSize/POWER(1024,pw),3),',',''),17,' '),' ',SUBSTR(' KMGTP',pw+1,1),'B') "Table Size" FROM (SELECT engine,SUM(data_length) DSize,SUM(index_length) ISize,SUM(data_length+index_length) TSize 
FROM information_schema.tables 
  WHERE table_schema NOT IN ('mysql','information_schema','performance_schema') 
  AND engine IS NOT NULL 
GROUP BY engine 
WITH ROLLUP) B,(SELECT 3 pw) A 
ORDER BY TSize;

.

Who to pick

Well that's up to you. I can't give a recommendation [although it would be obvious :)] - but you can certainly form your own opinion,

  1. If they have the hardware to match your minimum requirements
  2. If they have a solid heritage/reputation in Magento (ie. participating on sites like this, and demonstrating real-world experience)
  3. If they can provide a demonstration of your store, on their hosting (ie. so you can see exactly how it performs)
  4. If they can answer any and all of your Magento related questions (ie. try testing them with questions you have had in the past and see how they answer it)
  5. If their pricing matches your budget
  6. Ensure their support matches your expectation (ie. are you willing to do all the due diligence yourself before contacting support, or do you have an expectation they can debug your Magento issues on your behalf)
  7. Make sure the scope of works included matches your needs (ie. fully managed, bare metal etc.)
  8. If the solution is scalable (ie. is it single-server platform - can it scale to be a multi-server solution).
  9. If the solution features all the software you need for your Magento store (Eg. ElasticSearch/Sphinx/SOLR, Redis/Memcache, Varnish etc.)

There's something to add here, you'll get two types of hosts

  • Managed (eg. where the server is completely set up and taken care of)
  • Unmanaged (eg. where you set up the entire server yourself)

Which is most appropriate for you really depends on what your skillset is. In my honest opinion, developers are not system administrators - just because you can run apt-get install nginx - it doesn't make you an experienced sysadmin. Otherwise any store owner that could use Magento Connect could class themselves as a developer!

Understanding your role (and limitations) as a developer and letting someone with proper experience in managing the server has extreme value. Anyone can install a series of packages, but tuning the software configuration, ensuring it is stable, bug free, secure and high performance are a completely different thing.

I would only ever recommend the whole DIY route to people who have a massive amount of Magento server configuration experience, who have deployed thousands of servers before, with known tested and stable configurations. Otherwise, go with a managed Magento host that knows what they are doing and allow you to do your job uninterrupted.


Sources:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.