Looking Back 2010: Netflix's NY Times Moment

This is my third post in the Looking Back 2010 series where I highlight important cloud related events in this year. After OpenStack and Salesforce.com’s Heroku acquisition news, the third important news from my point of view is Netflix’s embrace of AWS for all their IT needs. This is very important for the validation of not just AWS but also public clouds, in general. In short, this is the NY Times moment for AWS but at a larger scale. More importantly, this may end up convincing enterprises about the value of public clouds.

First some background

Netflix, the most popular video rental service, moved most of their infrastructure from their internal data center to Amazon Web Services, the poster boy/girl of public cloud services. Just a year ago, Netflix had none of their customer traffic going to AWS but, today, most of their customer traffic is served by software built on top of AWS including Netflix.com website and the software for supporting Netflix on many devices like iPhone, iPad, Apple TV, XBox, Wii, etc.. Some of their software built on top of AWS includes:

Search Engines
Recommendation Systems
AJB Testing infrastructure
Streaming Servers
Encoding software
Data stores
Caching architectures

etc.

As their business expanded into streaming and as they added support for modern day video devices, there was a need to expand their IT infrastructure and re-architect their existing platform. They were faced with a crucial decision about whether they should spend capital and build their own datacenters or outsource it to a third party. With all the advantages of cloud computing and, especially its cost effectiveness, outsourcing it to AWS made lot of sense to them. The key thinking for them is to use their engineers to help with their core business than wasting their resources on non-core issues like deploying and maintaining their own infrastructure. More importantly, the nature of their business needed a more elastic infrastructure that could scale up and down based on peaks that depend on various factors. When they put all this together, they clearly realized that public clouds are the way to go.

The reason Netflix chose AWS over others was their massive scale. Not only AWS is massively scalable, they offer innovative cloud based services that are more mature compared to other public cloud providers. However, keep in mind that public clouds are not a miracle pill to cure all of the ITs problems. In fact, public clouds based on commodity servers bring in their own problems while they solve some of the existing IT problems. Netflix faced many issues as they went about architecting their platform on AWS. They have a blog post highlighting the challenges they faced and I strongly recommend you to read the post.

Why it is significant?

In my opinion, this is a NY Times moment at a grander scale for both AWS and public clouds, in general. Remember those early days of the cloud when we will happily point out how NY Times tapped into Amazon to convert their archives into PDF by swiping the credit card and paying ridiculously small amount? This is one such story for public clouds but at a grander scale. But it goes far beyond what NY Times story told us. As I pointed out in my post Why Public Clouds Will Eventually Win The Game, this teaches something very important to the enterprises. It is about cutting down the wasteful expenditure. The smart enterprises of the future will keep this as one of their top priorities. Public clouds help achieve this and Netflix showed us the way.

In my opinion, the way scaling back helps in the cloud economics is going to be an important factor which enterprise planners should take into account when they look at their cloud options. Yes, most of the enterprises don’t have their peaks like the consumer facing sites such as Netflix or Amazon retail. But, this factor will nevertheless be significant in the long run.

Netflix clearly showed enterprises why it is smart to push most of their workloads to public clouds and focus their resources on their core business expertise. In future, when the enterprise use of public clouds becomes a norm than an exception, we will turn back and see this Netflix’s move to AWS as a significant moment.

One more thing

Even as we get ecstatic about Netflix and public clouds, it is important for us to realize that enterprise is a different ball game from Netflix. Not only they are more conservative when it comes to their IT, their worries are legitimate. It is important for the public cloud providers to be more transparent about their security and satisfy enterprise concerns.