For all the Magento people running multiple web servers, how do you handle media folder?
- each server has its own media folder and keep it in sync (lsyncd?)
- other "smarter" fs (glusterfs)
NFS and nothing else for all our servers. Writes and reads are so very, very limited that there is no reason to use anything else. Varnish caches almost everything so activity over NFS is negligible.
Performance is quick, reliable and secure. No reason at all to use anything else.
In the past we've used deviations based around
rsync - but only in odd circumstances where we felt NFS wasn't appropriate (geographically diverse servers).
There is absolutely no use case for Gluster or any other complex distributed file system. It shouldn't even be considered.
I used NFS on two occasions and it failed for me during high IO load, similary Glusterfs is a big fat guy, not recommended, it is slower then NFS, IMHO I would recommend using Amazon S3 with combination of CloudFront or some other cloud based file storage, NFS and Gluster FS would give you tough time during high IO.
If you have small number of files, in thousands may be rsync would be a better option for you
Since I could not comment on the original post, I am editing my current answer.
There is already an extension available https://github.com/magefm/cdn I tried that myself and most of the parts just works fine, I had to enhance add a few features according to my requirements, it can be slow but once the cached files has been created, it is even more efficient, Its not an overhead but rather convenient IMHO. If you can then write a bash script to generate thumbnails and small images overnight or once a week, that too will make things efficient for you.
Note** If you have millions of files, I strongly recommend using NOSQL instead of MYSQL to store cached image information, and then query that for fileExiss
to summerize if you have millions of files that you would want to sync, NFS, rsync, GlusterFS all these local distributed storage systems cannot be an option, Object Storage can be the only better solution.