Monday, 28 July 2008
Computing in the clouds
This website and the others I run have always been a 'do-it-yourself' affair. Partly this is out of my own passion for technology, and partly as a showcase of my ability to design and implement an enterprise-level architecture (a J2EE app server and DB2 database running on Linux, with single-sign on, SSL, JAAS security, etc., etc.—it all makes me drool!). Even the server hardware I assembled by hand out of the individual parts.
The one thing I haven't been able to do, though, is host video. On the software level there is no difficulty to doing this: just put the video files in the right folder. On the other hand, the bandwidth costs (in terms of performance, but also in terms of money if I were to try to address it directly) are quite high. So heretofore, my home movies have been shot in a beautiful hi-resolution 16:9, and then horribly downscaled and sometimes even distorted by Google video or Youtube. I have not been happy with the results, but this was the only way to share my videos.
Now, however, my tech interests and video hosting needs have come together in an elegant solution, the programming of which was the most fun I'd had coding in a while. The buzzword these days in the tech press is "Cloud computing"—a phrase which usually is just a fancy way of saying you pay to subscribe to something. But in the case of Amazon's Simple Storage Service, generally known as "S3", they really deliver on the promise of the hype.
S3 puts the massive computing resources of Amazon.com in the hands of anyone who wants to use their storage and bandwidth, for any purpose. You can encrypt and store offsite backups, share documents, or... host high-bandwidth content for your website such as videos. Amazon handles the redundancy issues, and Amazon has the capacity to deliver on however large a scale you need. And because of economies of scale, the cost is very cheap.
What S3 does not do, is provide much in the way of tools with which to use their service. There are more and more third party tools availible, but I don't necessarily trust the third parties with my data. (Besides home movies, I also plan to use S3 for off-site backups.) So I have coded up my own file manager like application to handle uploading and downloading content to S3, and keeping my private data separate from material I want them to host for me publicly. (It also encrypts the private data—more out of principle than because I don't trust Amazon. That also added to the technical challenge which made writing it more fun!). As a result I am now able to offer full-quality versions of my home movies on my website, and without breaking the bank. Cool!
