1-2-3 easy as V-P-C

Warning: This is a long post. If you suffer from ADD (like I do) then maybe this one isn’t for you.

Blink and you’ll miss it. No, not a shooting star, but the release of yet another celestial feature set delivered by the mercurial team at AWS.

These days, it seems there is not a week go by without a seemingly obligatory Jeff Barr midnight tweet heralding the availability of another set of awe inspiring “add-ons” to the existing AWS platform. The latest? A quite unassuming little thing named VPC (Virtual Private Cloud) – touted as “a new approach to amazon EC2 networking” and which may, over the coming months, prove to be a pivotal moment in providing the level of flexibility needed to turn the plethora of enterprise network architect naysayers into believers and maybe even, gasp, evangelists? It really is that powerful in my very humble opinion.

Despite being notoriously, and sometimes frustratingly guarded on all things “roadmap”, the release cadence combined with sheer quality of product within the recent (and most likely future) AWS feature sets is a clear reminder to anyone who has lived the enterprise nightmare of long product cycles, complex deployments and huge upfront costs that we are, without question, entering a very different era and this one ride you really don’t want to miss.

Let’s start with a very brief history lesson. VPC isn’t totally new, it’s been around in one of those “extended beta” states for quite some time, however this incarnation is barely distinguishable from it’s ancestor – it essentially has four options (scenarios) where the previous incarnation had a single solitary offering. The intention of this post is by no means to explain the deep ins and outs of VPC (you can read all you ever need to read here) so let’s just say that the previous single offering now has a place amongst three other scenarios which, when combined, make up the new Four Horsemen Of The Cloudopalypse, AWS-style.

Luckily for me, our good friends at AWS (note: I am simply an enterprise customer with an interest in solving challenges through partnerships) invited me to the very early adopter program of the original release, during which, I built our first VPC connection in July 2009. My invite was issued primarily because I had spent quite a bit of time working with the folks at AWS to help them understand “our burning need for secure connectivity into your environment because we really want the flexibility of EC2 but our enterprise apps won’t work with it”.

Why?

Well….at risk of sounding yet again like the proverbial piece of unwanted, warped vinyl that has been kicked around a yard sale and then placed on your Grandmother’s gramophone, many of today’s enterprises, ourselves included, have applications that are inextricably tied to traditional enterprise technologies, including Microsoft’s Active Directory, and specifically, integrated authentication.

I could wax lyrical for the next 5000 words on this topic alone, but as I’ve covered the crux of this in previous posts (with respect to the challenge of opportunity etc.) and to summarize a very long story, unless our enterprise applications already supported, or could cost effectively be made to support federation (which is a lot harder than you’d think), we had a challenge that needed to be solved:

How could we take advantage of the flexibility in EC2 but maintain the logical configurations of our environment such that simple things like extending the “Corporate” AD environment could be achieved.

A-ha. There you are, VPC. Lend me your ear.

As I alluded to in a very recent post (which again is testament to the roadmap secrecy of AWS) the initial VPC construct was a good “first step” but lacked anything more than the ability to carve off a “private” piece of the AWS environment, stick your own IP range on it, create an IPSEC VPN tunnel between your on premise router and then nail up a BGP session between your environment and AWS to form the VPC. The proverbial fly in the ointment was the requirement for me to use my own edge infrastructure to grant access to VPC resources – strange, seeing how they were all EC2 instances (yes, I know, something about security, apparently) yet the ability to “connect” internet-facing EC2 instances to VPC instances just made plain old sense to me, and as such, became my hobby horse.

Giddy up.

In that very post, I threw down a few random thoughts around an earlier Jeff Barr twupdate; the announcement of the support for automagic import of vmware-based virtual machines into AWS. It seemed to me (and a host of others in the #clouderati) that there was a growing focus between AWS and vmware to provide enterprises a mechanism to move “seamlessly” to cloud, using just the native tools provided by both vendors. AWS and vmware? Yes, makes total sense I suppose – they are the clear leaders in their respective fields after all.

In reading through the deep technical explanations of the new VPC and understanding the nuances between the scenarios, my attention was firmly attracted to what has been coined “Scenario 3”. It was so tantalizingly close to what I had asked the AWS Head Shed for (vociferously and on many an occasion) that I could really begin, for possibly the first time, so see how this could be incredibly beneficial not just to scratch my personal itch, but to other large enterprises facing, or yet to face, the same challenges as me.

But with vmware alone ? Huh. I can’t quite…erm…well, what if I wanted to export VMs from other hypervisors, like, you know, XenServer or something?

Then, in a very rare moment of clarity, I remembered this post from my good friend, all round super bloke and Citrix’ CTO of CloudyDatacenteryThings, Simon Crosby. Hmmm. Wait a minute now. As the fog lifted, I remembered the work that Citrix had done with Amazon to “line things up”. This might be bigger than I thought.

Crosby’s always-engaging writing (though bizarrely coming through on my own internal head speakers in a South African / UK vocoded hotchpotch), the points he raised in respect of the Citrix / AWS relationship is going (hopefully), twinned with what could and should now be absolutely possible in the new VPC (especially Scenario 3) very rapidly took me to a whole new world of what this may mean and in fact, leads me to the real point of this post:

The door to our hybrid cloud may be opening a little further every day.

Now, before Hoff (no, not that one, this one) has a coronary brought on by an overly animated arm waving session where he calls me out for crimes against cloud nomenclature, or before anybody else reading this blog throws down their fists in a fit of irrecoverable meh stating “hybrid heresy” as my crime, remember that I said “our” and by that it means “in our context”. The context, for those who missed it, is of the “crapplications” that can’t work today in any other hybrid way.

I don’t believe that we are much different that many other large corporations / enterprises. I say that without generalizing and with a modicum of authority because I talk to them. A lot. I work alongside them on Research Boards and Customer Councils. I listen and I learn.

If they don’t have the problem we have today, they will tomorrow.

In my recent experiences, the majority who don’t have it today are not really getting down and dirty into public (or hybrid) cloud. They’re thinking about thinking about getting around to thinking about it. That’s a fact. The ones who do, are the ones who are digging, to various depths, and have similar challenges to us.

Irrespective of all the rolling of eyes and flailing of arms in the Public Cloud Hoi-Polloi, I am here to tell you that the enterprise won’t and can’t magically change overnight no matter how tight we close our eyes wish really hard for the unicorns to come and help. There are no elves and no shoemaker in this story. I am absolutely convinced that our context is absolutely not unique, so we must all look for ways to work within the constructs of what is available for any denomination.

To quote Hoff again…”use the right tool for the right job at the right time at the right cost”. It’s cloud poetry. Match opportunity to solution. Job done.

So, with all this in mind, I’m switching gears a little, from the sublime to the ridiculous and offering up a little thought teaser on what might be possible for the enterprise using a little of what’s available via AWS VPC Scenario 3 and just a little artistic license from Crosby’s post.

Before we begin. Caveat Emptor: The following thought process(es) makes these basic assumptions:

• One day (soon) the AWS / Citrix relationship and offerings have proven out and matured enough to become available as AMIs (or equivalents), specifically in the case of the Netscaler VPX and / or the Citrix Web Interface / XenApp “Appliances” (just to make it easier to deploy the base infrastructure).
• The architecture could be deployed and managed from any toolset (including CloudFormation) and could be scaled out as needed. It is not shown with all fault tolerant pieces / HA etc. Think of it like a private cloud but where you can literally move to variable vs. fixed cost (unlike private).
• The components, by definition, are all EC2 instances, EBS backed, so they can be stopped (and therefore reducing costs to just storage when inoperable).
• The current limitations of VPC (i.e. what it can and can’t do compared to EC2) are not taken into consideration.
• Some of this may contain inaccuracies. Please don’t go all purist on me, just let me know if you discover any such that I can fix in future revision.

It is in no way intended to be a “blueprint”, simply an example based on a little experience and a little dreaming. Feel free to scream, cuss, throw jagged-edged rocks at it and when you come up with something better, here’s kudos to you. Post it for discussion. That’s how we learn.

You will also notice that I have explicitly called out “AWS Bandwidth Charges” in the diagram. I will cover that more in a future post as I am trying to look for positives in this one.

(Solely for the purposes of consistency, I’ve somewhat shamefully “borrowed” the original VPC diagram look and feel and amended it, with this complete acknowledgement to whomsoever owns the copyright).

What’s going on in the above scenario? Well, it’s really pretty simple. I promise to follow this up with a video screencast to show step by step how this is set up, but at a high level, here are some of the generics:

• The VPC gets created in either US-East or EU-West Region, using the brilliant VPC Creation “wizard” tool.
• The “wizard” tool creates an “on the fly” configuration for your Cisco IOS or Juniper JUNOS device, located in the Corporate HQ. This configuration helps establish an IPSEC VPN tunnel with BGP peering to route traffic between the Corporate HQ and AWS (the VPC). The Corporate HQ network engineers / administrators must enable routing between their “home network” and the subnet(s) in VPC.
• The Public Subnet and Private Subnet are created to almost provide the equivalent functionality of a traditional “DMZ” / “Private” network.
• The ® is an inbuilt routing functionality that allows (by default, but fully configurable by route table) the Public Subnet to route to the Private Subnet and vice versa.
• The Internet Gateway permits traffic to flow to / from the Public Subnet to the Internet. Any instances in the Public Subnet must have an Elastic IP to be reached from the Internet. By default, for any destination except the initial VPC CIDR Block, the Public Subnet routes traffic to the Internet (via the Internet Gateway).
• The Private Subnet cannot be contacted directly from the Internet. By default, for any destination except the initial VPC CIDR Block, the Private Subnet routes traffic over the VPN tunnel (via the VPN Gateway).
• The AWS Security Groups construct is supported in this scenario. Security Groups limit inbound and outbound ports, protocols and destination addresses (if applied) and can also “span” subnets.
• The Security Group contexts are shown in color to illustrate a possible configuration. There are five Security Groups in this example. (I have not detailed the exact rule sets).

Happy? OK. Let’s get a bit more specific. In the “example” scenario:

• ACME Corp. has several client / server LoB applications that it would like to use the flexible nature of AWS to “serve” them from. The applications are not accessed outside of North America and are used between 8am-8pm Eastern Time.
• ACME Corp. uses Citrix XenApp to provide access to the applications.
• The user base is internal and external, but all have valid Active Directory accounts for ACME Corp. environment.
• Occasionally, some users from the ACME Corp. HQ “pre-populate” large files for some of the applications to use as reference information.

What might a solution look like?

• ACME Corp. deploys a Citrix Netscaler VPX appliance in the Public Subnet with a Security Group that allows inbound access from any Internet address on ports 80/443.
• The Netscaler will be issues a valid, chained SSL certificate and provide SSL offload functionality to enable and ensure secure connectivity.
• ACME Corp. offers the apps.acmecorp.com FQDN, using split DNS configuration so that internal user DNS lookups resolve to the “private” IP address in the Public Subnet and external user DNS lookups resolve to the Elastic IP address of the Netscaler.
• ACME Corp. deploys two Citrix Web Interface servers (prebuilt appliances) that the Netscaler device can load balance between.
• The Citrix Web Interface servers contact the Cirtix XenApp server to enumerate available applications. Authentication requests (via Forms Based Auth) are sent over the VPN tunnel to Active Directory domain controllers in the ACME Corp. HQ.
• The Citrix XenApp server in the Private Subnet provides access to the applications via application streaming technology. The application “bits” reside (along with user profiles) on the W2K8 file server. The applications are executed on the server, not streamed to any client.
• ACME Corp. uses Riverbed Steelhead WAN Optimization technology in their enterprise WAN environment. A Cloud Steelhead appliance is deployed in the VPC and the W2K8 File Server becomes a managed endpoint, allowing WAN optimization for file transfers between the ACME Corp. HQ (and vice versa) and saving on AWS bandwidth charges (hooray).
• ACME Corp. uses EBS snapshots to provide point-in-time recovery points for running instances.
• ACME Corp. stops the instances at 8.30pm each evening. The instances (and hence their application environment) are restarted at 7.30am each morning.
• All databases reside in the ACME Corp. HQ. The applications connect to them across the VPN tunnel.
• The initial configuration is deployed via a CloudFormation template.
• The running configuration is monitored by ACME Corp. existing enterprise toolsets.
• The running configuration is managed by the available AWS toolset.

If you’re still alive, congratulations. If you’re actually following this, even bigger congratulations. Pretty heavy going (and frankly a bit dull) in places, eh? Maybe, but the point is that I believe this is a potentially very lucrative space as enterprises look to do things just a little differently.

Some of the above hypotheses may be real today, some of it may be fiction, but even though you could be forgiven for thinking it sounds like a Citrix sales pitch at this point, it absolutely is not, it is just what I believe is (and will continue to be) a likely scenario given the enterprise usage of their products and the application use case challenges I pointed out in the earlier context definition.

Honestly, it wouldn’t take Charles Babbage to figure out hundreds upon hundreds of applicable use cases with different problem sets to solve, some inclusive of Scenario 3 and some not. Isn’t it great to have some options and especially those to move an enterprise to variable cost without breaking today’s functionality in certain apps? Of course there is a lot to figure out, but isn’t that most of the fun?

The rhetorical debate over true private cloud versus hybrid versus public will no doubt rumble on for some considerable time, but, as long as there are advances like we are seeing here – which are no means limited to AWS I must add – then new solutions, new ideas and new models will flourish.

Limiting your thinking to any one type of solution may inhibit your ability to really make a difference. If I can dream this stuff up on a plane in just a few short hours….imagine what could really be possible.

Question is, are you brave enough to challenge the status quo?