I have been critical of “me too” approaches to PaaS by current breed of PaaS providers and have been calling for some differentiation which will take innovation to the next level. Some people have been asking me how the next generation of PaaS will look like. I have framed a simple model to explain where the next iteration will take PaaS but haven’t published it so far because I still haven’t tightened some loose ends in the model. With Amazon releasing DynamoDB this week, I thought I will just push the idea into the wild and use the feedback to tweak my model. The result is today’s post and I am expecting a barrage of criticism to my claims here which will help me fine-tune my ideas.
A simple model for PaaS v1
Even though individual PaaS services/software may differ in how they are architected, most of the current iteration of PaaS players can be modeled, in a simplistic way, as shown in the diagram below. Keep in mind I am trying to portray a simple picture to make a point and please don’t pounce on the nuances that are missing in the following diagrams.
The current PaaS layer completely abstracts the complexities associated with the infrastructure underneath and offers a very simple interface for developers to deploy their applications. The PaaS layer encompasses runtime frameworks, messaging services, data stores, monitoring services, etc.. Even though different PaaS services/software allows the use of different types of data sources based on the application needs, the data component (for all practical purposes) is “integrated” in the PaaS layer itself. The action in today’s PaaS mostly lies in this integrated layer that abstracts away most of the complexities underneath from the developers. This lead to the current bunch of PaaS players scrambling to add support for many languages, many cloud providers, different messaging services, different data sources, etc.. However, in the next generation of PaaS I am hoping to see in the coming years, the action will be centered around the data with the current PaaS layer relegated to the background just like what happened, in a way, to the middleware in the traditional development model.
A simple model for PaaS v2.0
In the next iteration, the entire action will be centered around the power of data. Instead of data sources being one of the many components of today’s PaaS system, the next iteration of PaaS will be dealing with a more powerful “Data as a Service”, which will include data sources, data cleaning tools, data management tools and, more importantly, a very powerful analytics engine. It is this analytics engine that will sitting on top of the data components which will make data as a service vital component to the IT of the future. The applications we will have then will be much different from the ones we have today. These next-gen applications will not just be consuming/producing raw data like today’s applications. The next-gen applications will be consuming the insights gained from the data it produced, fine-tune itself and produce better data. These data, in turn, will be fed into the associated analytics engine which, in turn, will offer better insights to the application which will, again, fine-tune itself to produce even better data. This cycle will continue with better data being produced as more and more good data is collected. Essentially, this will lead applications to self organize itself to become more intelligent applications.
In order to build these next-gen applications, we need next-gen PaaS which has the “Data as a Service” component tightly integrated within. Yes, we are seeing some analytics components that are built on top of different data sources like MongoDB, etc.. Some may even argue that these can be plugged into the current version of PaaS to get what I am talking above. Even though it is possible, I don’t think it will be powerful enough to handle the needs of next-gen organizations. The data as a service component I am visualizing will get their data from many different applications and most of them will be hosted on the public clouds. It may include one or more cloud infrastructure providers underneath but the interface will completely abstract away all the complexities and it will be a big firehose of data with a super powerful analytics engine sitting on the top offering actionable insights. The net result will be much different from what we can get by just plugging analytics into today’s platforms.
With this in mind, I propose the following model for the next iteration of PaaS. I encourage you to add to it or puncture holes so that I can tweak it for the better. The model itself is not very different from the current version of PaaS but the significance lies in the presence of the all powerful “data as a service” component and the way it is interfaced with the other platform components (which includes the current day PaaS layer).
By then we will be consuming most of our compute needs from the public clouds (though it may not be from a handful of providers as the “scarcity based economics” thinkers anticipate but it is a different discussion for a different day and I don’t want to talk here) and the idea of private PaaS as we have today will be moot.
Where is Amazon coming into the picture?
Already there are many companies who have clearly understood the importance of owning the data and eventually provide data as a service offering. Salesforce and IBM are clearly on track towards this end. So far, Amazon was making half hearted attempts towards this goal. With no credible PaaS play in their toolkit, they have realized that they can stay credible in the future of cloud services only if they can lock in all the data of their customers inside their cloud. With DynamoDB, Amazon is getting serious about it and if they succeed in convincing more and more organizations to put their data on this service, they will be well positioned to compete hard in the next iteration of PaaS. I think they have a great opportunity to do it. In fact, Stephen O’ Grady of Redmonk highlights this fact in his blog post.
The most obvious advantage of DynamoDB versus its current market competition is the fact that itâs already in the cloud, managed and offering consolidated billing for AWS customers. Requiring minimal setup and configuration versus native tooling, a subset of the addressable market is likely to be of a similar mindset to this commenter on the DataStax blog.
In fact, I would also advise you to read the comment he is highlighting regarding how lack of hosted Cassandra might help in the DynamoDB adoption.
I am completely clueless on where Amazon is going but if Amazon has a plan for PaaS (which I am sure they have because PaaS is the future of Cloud Services), DynamoDB is the first step for them to enter the game in the next iteration of PaaS. If their agility in the cloud market is any indication, they may even iterate PaaS to the next generation much before other players. Of course, Salesforce and IBM are trying to do the same coming from the application side of the things. We will have to wait and see who gets the next version of PaaS first.
HaHa, I told ya. Only handful of cloud providers will be in the game
Right now, only a handful of companies are leading the push towards taking the data to their clouds and offer the data as a service component needed for next-gen PaaS. But the percentage of data they hold is minuscule compared to the data we have in this world. There is more time for innovation which will help data as a service market emerge differently. Never underestimate the diverse needs of this world. Even though the advancement of capitalism into different parts of the world removes this diversity to a certain extent, the needs of the world are still diverse enough to have multiple players in the fold. I know we are going to see a more federated approach to emerge in the data as a service market. If you ask me today how it will look, I don’t have a concrete model but I can assure you that it is going to be one of my focus areas of research this year.
PaaS is definitely going to evolve from today’s iteration to the next one which will help in the creation of self organizing intelligent applications. There is very little innovation going on today to take the game to the next level but we will soon be seeing the shift either by some of the current day players or a complete disruption by a new kid in the town. It is still too difficult to predict when this will happen or how it will happen but it is going to happen sooner than most of us are anticipating. If you are a vendor in this space, you have to be focussing on the next steps. As far as the organizations on the buyer side, it is important to keep an eye on where the market will be in the next few years while planning the strategy for today.
Update: In the era of flashy names, I thought I will call the next iteration of PaaS enabling self organizing intelligent apps as Intelligent Platforms.
Krish,
I agree with the direction for your vision for PaaS 2.0. Clearly, the whole value proposition is the availability of consumable, metered services that are available to the PaaS container and shared across all instances. Data management is one of these services as is Security-as-a-Service (SecaaS), Messaging-as-a-Service, Collaboration-as-a-Service, etc. the goal is to make it as simple as possible for developers to build out new applications using basic semantics and let the power of the PaaS provide the speed, scalability and elasticity.
-JPM
http://about.me/jpmorgenthal
Hi Krish,
Good post !
Yes PaaS is evolving to next gen intelligent applications and PaaS 2.0 has “Data as a Service” component tightly integrated within. Then I see ‘Linked Data as a Service’ http://bit.ly/tPNeHq coming into play as a follow on.
One question i.e. when you say only handful of cloud providers ( Amazon and ? ) will be in the game I assume you’re talking about public clouds but not hybrid or private clouds
– Steve
Steve, I am talking about public clouds only. But I see the public infrastructure cloud market to be more federated than a handful of players like Amazon, et.al.. I promote federated clouds as an antidote for natural monopoly that will emerge in the market. I am just trying to see how we can ensure the federated model while also ensuring the simplicity offered by abstraction in the stacks above (like PaaS).
Thx Gotcha !
Krish, good post. I agree with your overall position, although I’ve thought about it a bit differently. Data and the ability to work with data will undoubtedly become the most important part of PaaS 2.0, but I don’t think an analytics engine is the hub in that sort of system. Instead, I believe that *good* PaaS offerings will offer trivial support for advanced patterns and computation models. This will be the preferred approach, which can equip developers to trivially build powerful analytics engines that are highly specialized and tuned for their data sets. Essentially, we’ll see a transition from multi-threaded apps as being the thing that “cool developers” do to multi-agent systems (MAS – something like what is described here, but I see a slightly different outcome to help broker the transition from old paradigms – http://en.wikipedia.org/wiki/Multi-agent_system). The PaaS offerings will make building MASs a simple matter, allowing developers to focus on the logic executed by these MAS engines.
The reason this distinction is important is that MAS is a general purpose construct that even in non-data intensive scenarios can be a powerful execution backdrop, and is a bit of a DSL for the analytics problem.
Not to beat a dead horse, but this is why I think that a deep runtime model for PaaS is appropriate rather than the non-runtime “Look at how many languages I support” approach; a runtime model is required to get this level of richness in the execution model, and the level of instrumentation necessary require focus on depth by PaaS providers.
Just my 2 cents.
Thanks Sinclair. Your model about letting the developers build analytics engine is definitely a viable model. However, I think we need some really powerful analytics engines than the plug and play types that developers can use/build. Let us wait and see how the field evolves.
I like the innovative thinking here, and believe that data-as-a-service will become a more sophisticated component of PaaS. It wasn’t clear to me if your PaaS 2.0 definition was encompassing the future of public PaaS offerings, private PaaS offerings, or both. If the focus was primarily on public PaaS, I have a couple of concerns about broad scale adoption of certain use cases you mention:
1) To realize the vision of self-optimizing applications with advanced data analytics, you’re talking about a lot of data over time. If you are dealing with a federated model of public clouds as you mention, you’re talking about potentially very costly bandwidth to move that data – perhaps cost prohibited. If everything was under one cloud, you could minimize bandwidth consumption, but you’ve vision leans towards federated clouds, which BTW I totally agree with.
2) The underlying raw data (for example, to support customer analytics) could represent a very strategic asset to an enterprise such that they may not want to move it off-premises… either due to regulatory constraints, fear of loss or theft, vendor lock-in, or other concerns. Corporate data security and governance concerns can be big hurdles to clear.
For these reasons, I wouldn’t minimize the importance of private PaaS 2.0 offerings with powerful analytics, etc. In fact, we are seeing growing interest in more advanced private PaaS in large enterprise accounts – http://bit.ly/qNHClu. My 2 cents and best of luck with the fine-tuning.
DT
I agree about the bandwidth issue but I am extrapolating to believe that we will see some innovation on the data transport front.
However, I am not entirely limiting to public clouds (as such) but I believe that by then (which could be few years away), we will have a much better “regulatory situation” than what we have now to move most of the workloads to the public clouds.
You’re right. I expect we’ll have a better regulatory situation in the future as well. Policy-driven cloud governance will be a key to that success – http://bit.ly/mIYxdW.