Magento Stack Exchange is a question and answer site for users of the Magento e-Commerce platform. It only takes a minute to sign up.
Sign up to join this community
I was asked how to save all the magento media assets in Amazon S3. I thought about using a PHP Stream Wrapper like this one: https://github.com/punkave/aS3StreamWrapper and just set the media folder to s3:\mybucket\mediaFolder
Does anyone has experience with this?
Or better use something like this: https://github.com/sstoiana/magento-s3 But this extension doesn't look good. Any other recommendations?
it seems these guys find a solution: ( http://thinkglobal.co/resources/moving-the-magento-media-directory-to-s3/ )
There are 2 possible approaches, one being to have Magento push all media assets to S3 as they are created and the other being to simply mount the media directory to an S3 bucket and let Magento think that it’s writing to the local disk. We chose the latter.
We ended up using a tool called s3fs, which uses a broader technology called fuse. With some fairly straightforward configuration (described below), you can get s3fs up and running. Be careful not to hardcode S3 bucket names or API access credentials if you are using AWS or other PAAS provider.
Make sure you replace the following variables with values for your specific configuration:
Here is the setup:
yum install -y gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap
cd /usr/local/src
wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.3/fuse-2.9.3.tar.gz
tar -xzf fuse-2.9.3.tar.gz
rm -f fuse-2.9.3.tar.gz
mv fuse-2.9.3 fuse
cd fuse/
./configure –prefix=/usr
make
make install
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig
modprobe fuse
pkg-config –modversion fuse
wget http://s3fs.googlecode.com/files/s3fs-1.74.tar.gz
tar -xzvf s3fs-1.74.tar.gz
rm -f s3fs-1.74.tar.gz
mv s3fs-1.74 s3fs
cd s3fs
./configure –prefix=/usr
make
make install
echo “{{S3_BUCKET}}:{{API_PUBLIC_ACCESS_KEY}}:{{API_SECRET_ACCESS_KEY}}” > ~/.passwd-s3fs
chmod 600 ~/.passwd-s3fs
echo “user_allow_other” > /etc/fuse.conf
echo “s3fs#{{S3_BUCKET}} {{MOUNTED_DIRECTORY}} fuse use_cache=/tmp,allow_other,uid={{UID}},gid={{GID}} 0 0″ >> /etc/fstab
s3fs -o allow_other -o uid={{UID}} -o gid={{GID}} -ouse_cache=/tmp {{S3_BUCKET}} {{MOUNTED_DIRECTORY}}
OnePica's CDN extension is very handy out of the box for this functionality and supports more than just S3. Quote: "Currently, the extension integrates with Amazon S3, Coral CDN, Mosso/Rackspace Cloud Files, and any CDN, server or service that supports FTP, FTPS, or SFTP."
OnePica's CDN extension is not the solution it only works for product images not to mention you need to set the correct TTL for its own cache life time which will timeout much earlier than default Magento. All you cms wysiwyg image will be in media folder but not using S3, you will end up uploading all your media image to S3.
Use cloudfront will be good but you will still need a NFS to share your media folder across all your frontend nodes and backend nodes. Or you need to build your own interface to save file on S3, and you also need to handle errors (think about someone upload extreme large file whether your code can handle that before S3 API timeout)
Other way just save media storage in your extra RDS (database) and individual resize cache image will still save in your app server local media folder, but it is ok as cloudfront will cache them already.
I prefer having a separate "admin" node(admin.example.com) and keeping media folders there. This way you would save yourself with NFS cost as well as issue with syncing etc. any CDN (I use cloudfront) could then be connected with admin node media folders, and be configured for all stores.
Other benefits of using admin node is, you could move all crons(reindexing and other crons) , scheduled tasks(product imports, inventory & order sync etc) at admin node and save frontend nodes cpu processing.
Edit: As last answer didnt actually addressed OP's question. Magento Media Assets in Amazon S3? Well I believe Magento already has got so much processing overhead, that I think it'd be good idea to keep it from s3 syncing/streaming, and use local media folder instead.
Well, now that a couple years have gone by I think it's important to note that there are multiple plugins available that will move and serve assets from S3 / CDN.
Google searches turn up plenty of options.
Here's one on GitHub:
Here's one on Magento's Connect marketplace:
http://www.magentocommerce.com/magento-connect/aws-s3-cdn-for-product-images.html
I have not tried these particular extensions yet but will be for an upcoming project. There are others out there, too.
I'd have to disagree with @Sonassi, using separate storage to allow the cheapest server configuration still with good performance is really considered the best practice today. For example, simply use S3 to move all static content off your web server helps the server use far less bandwidth and allow more concurrent users and less overall server power needed. And, if you're in a single datacenter like most people these days, and say it's in Dallas, then there's no doubt that people in Western Europe will be served much faster with a Global CDN like Cloudfront. So, using storage separate from your web server to lower bandwidth usage allows you to use only 1 server when you would need more sooner without this approach, and using CDN will greatly increase global load times.
