About a month back, Sun announced their Project Blackbox initiative. It’s a mobile data center product that’s essentially a shipping container full of water cooled low-power servers. And those containers are stackable (look at the picture here on Dave Leve’s blog). But the real story is not about the data center in a box, it’s about getting the data center from a wall-socket.
Byte and Switch has a story (thanks for sharing, Niranjan) about an online photo sharing site SmugMug that expects to save a $1m in storage costs by shifting its data onto Amazon’s recently launched S3 hosted storage service. Here is an extract from the article:
The exec says that data center savings aren’t just coming from hardware. “It’s in terms of buying the RAID boxes, the power, cooling, the data center space, and the manpower needed to maintain them,” he says.
The CEO admits there is an element of risk when you rely on hosted storage, particularly when it comes to availability. “It’s something that we obviously think about all the time,” he says, although he feels that Amazon is in an unusual position. “Amazon eats their own dog food — when S3 has problems, Amazon has problems.”
All said and done, Amazon’s S3 has created a huge marketplace for storage, backup tools, online backup vendors and other niche products. It’s quite likely that Amazon EC2 is going to do the same for the hosting market. Edwin Ong’s CastTV offers an example of what might happen. Here is what Edwin has to say:
We are right now in the midst of building out CastTV’s own data center infrastructure - and search is obviously a computationally intensive application (think about the 100,000s of servers run by Google). The ability for us to get hundreds of servers up and running on Amazon infrastructure in a matter of hours (?) with EC2 is a mind-bogglingly attractive alternative.
There is no denying that EC2 isn’t perfect yet. There is a risk of down-time and it isn’t a panacea for all applications. It’s good for the category of computational grids and static-filing systems (like web-servers), but it’s not useful to anyone who wants to deploy application-level servers such as J2EE. In spite of these limitations, EC2 and S3 have moved the process of data center virtualization forward.
Remake of an old movie!
Go back to the 80s. At that time the semiconductor companies were vertically integrated and they did both design and manufacture of integrated circuits. Then in late 80s, the value chain got modularized into specialist fabless companies and merchant foundries (like TMSC). Both sides benefited. As the wikipedia article suggests, the fabless company could “concentrate its research and development resources on the end market without being required to invest resources in staying current in semiconductor technology”. Merchant foundries were able to find work from the pool of fabless companies and by careful scheduling, pricing and contracting remain at full utilization. Doesn’t this sound very much like the world that we are about to enter in the data center and enterprise applications space.
Given that this movie has played out before, we know what to expect. As the tyranny of data center comes to an end, most startups will no longer need to invest in building data centers. Their capital intensity will fall. This lower threshold will result in more startups. There will be more innovation than before. Inevitably, out of all this, new application categories will emerge and new markets will get created.
Adaptable infrastructure
On the other side, the pressure to revamp legacy infrastructure will intensify dramatically driven by the lower cost structure of external specialist data center utilities. There will be a huge push towards adaptable infrastructure. There will be no sacred cows in this restructuring and anything and everything will be on the table. Data center processes will change (ITIL will get a further fillip), virtualization technologies will become more pervasive, management software will see a boom, and remote infrastructure management will take off. In the end, we will be closer to, what Nick Carr calls, frugal computing.
The India connection
India is ripe for startups that create new consumer markets or develop the SMB customer segment. After all, the broadband connectivity is in place, PC penetration is reaching the tipping point and the small businesses are investing in growth. These positive demand side conditions are completed by the readier availability of venture capital. When you add wall-socket data center to this favorable mix, it should setoff some interesting action. Let me know what you think.
Sharad,
Nice article on Amazon’s Web Services. Here are two more articles about how Amazon’s EC2 and S3 are managing to catch the imaginations of folks:
http://www.fcw.com/article96617-10-30-06-Print?
http://blogs.zdnet.com/BTL/?p=3966#comments
Interesting to note that the above two articles are about folks from very different segments that are considering Amazon.
In my mind, nearline storage will remain within data centers for quite a while. However “far storage” (i.e. storage that is not directly in the IO path of an application) is likely to rapidly move to the hosted model. And in my mind, the Amazons, Googles and Yahoos of the world will maintain the edge over traditional storage vendors. Primarily because of two reasons:
1) Their tremendous reach.
2) Their immense expertise in transacting business via the internet. Other companies will take time to acquire that experience and are likely to follow.
And yes, I also agree with you that hosted services like Amazon’s (”datacenter on tap”) talent from across the globe could prove to be a heady concoction for innovation and startups.
-sroy