Public vs Private Cloud Brouhaha: My Take

Image via Wikipedia

The blogosphere and Twitterverse is full of debates about whether private clouds are actually clouds or not. I have had my own share of debates on the Twitter and I thought I will bring it over to Cloud Ave for further enlightenment. For a change, I am going to follow the Redmonk style and go with a Q&A approach.

1) Do you want to make any disclosures?

Yes. I used to be an unabashed proponent of “only public clouds” theme. I have advocated this in this blog, in comments on other blogs and Twitter. I used to argue in favor of the same thing I am going to oppose in this post.

2) Then, why are you behaving like Democrats and flip flopping again and again? Why can’t you stand firm on your principles like Republicans?

My approach to life follows Scientific method of understanding. I start with a model, see if it fits with data (from real life). If it fits, I continue to stand by it. If not, I will tweak the model based on what I saw in the data and see if the updated model fits well with the data. I keep on iterating this forever. Sometimes, based on the data, I may even completely throw away my model. Unlike economists, who expect the whole world to change in order to fit their model, I take the path of scientists and tweak my model to go hand in hand with what I see in this world. Probably, this flip flop of mine is a result of my understanding of how the real world approaches cloud computing.

3) BTW, what is this public cloud and private cloud thing? Can you explain the basics before you expect me to understand this post?

Public Clouds are infrastructure services offered by companies like Amazon, Rackspace, GoGrid, etc. and platform services like Microsoft Azure, Google App Engine, etc.. These cloud providers take care of deploying, managing, securing the infrastructure and companies can consume it on demand with a pay as you consumption option, much like the utility consumption. Private Clouds are deployments made inside the company’s firewall (on-premise datacenters).

4) Wait a minute, I know this difference, what is the deeper philosophical difference that has split the Cloud community into two camps?

The problem lies in the very definition of Cloud Computing according to these two camps. The “only public clouds” camp wants to bring in the financial component inside the definition of Cloud Computing. They want the definition to include utility pricing (therefore, bringing in an outside vendor into picture for offering the services) and on-demand use of resources. They claim that the single most defining element in the definition of Cloud Computing is the enormous cost savings in OpEx and the zero CapEx. The supporters of private clouds argue that it is not necessary to bring in the financial component and they see it as an evolution of already existing technologies including virtualization. They see it from a technology perspective and, also, as an operational model. A new way to deliver computing resources to either users inside the enterprises or to consumers like a public utility service provider. So, the problem lies on the very definition of Cloud Computing.

5) Do you mean to say that the term Cloud Computing is not conclusively defined yet even though companies are spending Millions in adapting the Clouds in their workflow?

Sigh. Sadly, my dear, it appears to be the case.

6) Why did you support the “only public cloud” camp earlier?

There are many reasons for it. The most important reason is that the idea of Cloud Computing was brought to masses by the public cloud companies like Amazon and Google. They helped the imagination of masses run wild with an attractive utility based pricing and an ability to scale infinitely (albeit, theoretically) to meet any demand. This surging forward of public clouds and the initial debunking of the idea by enterprises and vendors like IBM, made sure that I was hooked on to the public cloud idea (the only cloud that was present then). Nick Carr’s book and the hype surrounding it cemented my infatuation with the public clouds.

7) So, what changed afterwards?

For one, the enterprises started saying “yes, we can” to the idea of cloud computing. They saw the advantage of Cloud based technologies and wanted to adapt these technologies in their infrastructure. Seeing this trend, the reluctant ones like IBM and others jumped in. As we started digging in deeply about applying the idea of Clouds in a wider enterprise context, we understood the many needs and issues that were not obvious earlier. Everything from security to regulation to the need for better infrastructure to, even, the very need to have some fine grained control became important and needed further understanding. If you ask Christopher Hoff or Craig Balding or Dan Kaminsky (See Cloud Ave coverage of Dan Kaminsky’s presentation at CloudCamp Seattle), they all will talk about how much more we have to do to make Cloud more secure, especially, in the enterprise realm. The same people and few others like Ward Spangenberg can point out how public clouds fail to meet the regulatory requirements. If you talk to Greg Ness of Archimedius fame, he will give his piece of mind about the need to upgrade the existing internet infrastructure to meet the needs of cloud computing. If you talk to those psychiatrists who counsel CTOs and CIOs, they will tell you how afraid these enterprise folks are when it comes to putting their crucial business data in the hands of third parties. If you talk to many other people around the world, they will talk about the need to customize the infrastructure to meet their requirements and how the public clouds is too restrictive in this regard. There are many such issues that ensures that the idea of private clouds is relevant and, in fact, important.

8) Before we go further into the debate, what is your vision for the future of Cloud Computing?

I want (and hope that it happens) an open, federated cloud ecosystem. Ultimately, the market will decide the evolution of the cloud ecosystem.

9) Don’t you think public clouds are the future?

Yes, eventually all of us will move into public clouds in one form or another. The current hybrid model (both public and private) is just an intermediate step in the evolution process. While I believe in the eventual future of having all the computing resources on public clouds, I am only against the short-circuiting of the evolution process. Expecting everyone to jump straight into the existing public cloud infrastructure is totally naive. It will take a long time before the complete transition takes place. First and foremost, the technology behind the cloud is still at a nascent stage. It needs to evolve. The idea of security in the cloud needs some revamp and new approaches are needed to drastically improve the cloud security. It is important for the vendors to sit with every single government in the world and discuss the regulations. It is important to evolve a worldwide consensus on the regulatory requirements on data security, privacy, etc.. Sometimes, handling a single government itself is difficult. Imagine, bringing together all kinds of regimes in the world and f
orging an uniformity in the way they treat their citizens rights data. Then, there is this need for markets to mature by taking advantage of the customization needs and local requirements of customers from many different parts of the world. There are many other issues which I am omitting here for the want of brevity. I will just make it short. There is no way we are going to see large scale adoption of public clouds in the near future. Even if people adopt to public clouds along the lines of electricity, there will definitely be users who embrace private clouds much like how some homeowners and farms use non-conventional locally produced wind and solar energy systems to take care of their electricity needs. Let us just keep this in mind. The world is diverse and their needs are diverse. There is no way we can overlook this diversity and say that the entire world will fall in line to consume computing resources as it is offered by the public providers today.

10) I am still confused. Are you with us (public clouds) or against us (public+private clouds)?

All I am saying is that let us leave the binary approach and look at the shades of grey. Let us take into account the diverse needs of the world and see how their requirements can be fulfilled. If we do that, I am pretty sure we will end up with an open federated cloud ecosystem.

11) So, you are going to keep flip-flopping?

If it will make the technology more useful to the world, I am fine with being a flipflopper.

12) Come on, isn’t private clouds just virtualization with some management software slapped on top of it?

In my opinion, tying virtualization to the idea of cloud computing itself is myopic. Even though virtualization plays a predominant role in most of the clouds, it is not the only way. Like Google, it is possible to build a cloud using commodity hardware and dumping a mapreduce like system on top of it. There is nothing in the world to prevent me from putting together several thousand commodity machines inside a datacenter (or multiple datacenters), inside my firewall, and build a cloud using something like Hadoop. So, even though there are many advantages in resorting to the use of the virtualization technologies, the idea of cloud is not restricted to virtualization alone. Even if I use virtualization in my architecture, the argument that private clouds are just virtualization plus management software is very shortsighted. The elasticity, self healing, etc. are not just management features on top of virtualization. They are the ones that define the very idea of clouds when we see it from a purely technological point of view.

Here is a counter argument to those who claim that private clouds are just virtualization+management features. These people claim that the outsourcing component is very important to call something a cloud. Then, if a third party offers infrastructure as a service where they use virtualization but replace the management software with humans who turn the VMs on & off and, then, check the plumbing regularly to make sure “bad VMs” are taken off, data copied over and new VMs are spun off. Will these pundits accept this arrangement as cloud because the outsourcing component, as required by them, is there and the management software, as dismissed by them, is not there. I would be eager to hear from them about this “cloud”.

13) Is there any other reason why you fear the “only public clouds” idea?

I think the Cloud Computing market is not mature yet. If we push for the “only public clouds” idea now, the existing cloud players and few others like Microsoft who have enough financial muscle will capture the market. Then, we will end up with a handful of monopoly players even before many countries wake up to the idea of cloud computing. My experience, living through the desktop era, teaches me that it is not a good idea to allow monopolies in the marketplace. This is one of the reasons why I don’t want the public clouds to take off at this time. Let the technology mature and let more and more players start playing the cloud game. Once the market matures, it will be difficult for few monopoly players to emerge (See my post Handful of monopoly infrastructure players – A shortsighted idea).

14) I am done listening to your Obama kind of professorial speech. Can’t you just wind up this post?

Will do. Whether it is a private cloud or IT Management 2.0 or public cloud or xxxxxxx, it is just a terminology to denote a certain thing. Why fight over names? Why can’t we live and let live? Why can’t US coexist with Iran and Cuba? Why fight over stuff that means nothing in the big picture? This reminds me of the rhetoric of FSF fanatics against Open Source licenses. This also reminds me of the whines of certain open source zealots against GPL. Why do we have to take a binary approach and say “either you are with us or against us”? Why can’t GPL and other open source licenses coexist? Why should one’s existence means “death to the other one”? It is just plain silly to fight over such trivial things like names and terminologies.

Let me just say one word before I end this post. Enough!!