From wattersjames.com: http://wattersjames.com/2013/05/09/economics-of-application-virtulization-on-aws/ - Considering what's going on with containers, this is a very interesting discussion of the economics of app virtualization AWS.
Given current EC2 pricing, deploying your application tier directly to EC2 VMs is the wrong architecture for many use cases. Using a Linux Container isolated and application centric platform can provide up to 10x+ hard cost savings vs. a VM instance based approach.
If you are developing and running applications on AWS you can save by purchasing reserved instances. Here are the savings on a $/gb/year basis:
AWS EC2 Pricing
1 Year Reserved/Year/GB
3 Year Reserved/Year/GB
m1 Small (1.7gb)
Reserved instances cost almost 3x less per GB. While this is well known, using the largest memory XL instance saves almost as much–over 2.5x/GB–without pre-purchasing reserved instances. AWS charges significantly more per GB on the smaller sized instances over their larger VMs. (full worksheet here)
m1 Small (1.7gb)
Quad XL (68gigs)
Cluster Memory XL (244gb)
Launched in August of 2006 EC2 was built before the the Linux 2.6.24 kernel release in 2008 which first included cgroup namespace isolation. Perhaps in part because of its age EC2’s architecture is fixed to a VM model for application tier scaling. Their Elastic Beanstalk deployment model binds a single application instance to each VM, offering coarse grain 1.7GB, 3.75GB, 7.5GB step function options. This VM centric approach is dated compared to the efficient container based systems already used at Google.
It took four years after launching before EC2 supported custom kernels and first class LXC support in 2010.
Enter Cloud Foundry Warden
In 2012 Cloud Foundry undertook a platform service optimized implementation of cgroup isolation called Warden. With the ‘NG’ or ‘2.0’ updates to Cloud Foundry completed this month the Warden API is now the central resource management and isolation mechanism. Warden can also take advantage of aufs to speed the provisioning of containers, allowing for rapid movement of application instances between VMs.
Cloud Foundry is not VM centric despite its VMware heritage. The core system components exist to control the automated staging, deployment, networking, health monitoring and redeployment of applications across Warden containers. Virtual machines are divided into Warden containers with minimal overhead and strong resource partitions.
Cloud Foundry application sizing is fine grain, with 128MB steps, allowing high efficiency packing of the available memory into variably sized Warden partitions. Using this approach, large memory VMs are transformed into hundreds of highly economical application containers, tightly packed without resource contention. This modern architecture eliminates the AWS pricing burden on small application instances.
The Over 10x Payoff on AWS
Capacity wastage is seemingly endemic to the EC2 fixed sized VM model– a study of 250 companies using 250,000 instances found utilization rates of only 15%. The 200,000 medium/small instances would have a conservative monthly cost over $18M–leaving room for $11m in savings for memory utilization. Pushing memory utilization to 80% and backing it with the CXL instances the $18M monthly spend is reduced to $2.7M.
Centralizing capacity can also make reserved instance capacity planning simpler, as the study also found only 33% of instances were reserved when 97% should have been given their usage profile.
In addition to the progress on the ‘2.0’ code this month the Cloud Foundry team has also partnered with Dr. Nic Williams of Stark and Wayne to make deploying the system simpler on AWS. This reduced setup complexity is allowing broader adoption on AWS.
We are in active development of a Cloudfoundry.com hosted service for AWS. Pivotal will be running and developing this service at scale using many of the same architectural patterns outlined here and sharing our code and configurations with the community. It will be exciting to share production tested configurations for realizing these benefits with everyone.
The biggest benefit of next generation application centric platforms is really about developer empowerment and agility. As the Wired article observes of similar efforts at Twitter and Google:
“Borg and Mesos don’t just wring extra computing power out of a server cluster. They let companies like Google and Twitter treat a data center like a single machine.“
The immediate economic benefits of treating EC2 as a single machine however are compelling at any deployment over 250GB in scale, even before increased agility and ease of use. As the Cloud Foundry on AWS community grows it will be interesting to see how EC2 reacts. Their burden on small and medium instances is no doubt a rich cash cow within their portfolio and they may be unlikely to abandon it anytime soon.