OpenStack 101

In building a cloud computing framework that could unify both on-premise infrastructure and hosted third party services, the open source advocates behind OpenStack have devised the one technology likely to pose a competitive threat to the Amazon Web Services and Google juggernauts.

By Gartner's reckoning, AWS now has five times the Infrastructure-as-a-service compute capacity of its 14 biggest rivals put together. The individual efforts of Rackspace, Microsoft, IBM, HP, CSC or Telstra are unlikely to create a genuine competitive threat in the near-term.

OpenStack, by contrast, has all the ingredients of something that can. OpenStack is a common specification for server, compute and network resources that can be built inside or outside the firewall, alongside a suite of open source management tools for management of these resources.

OpenStack is 100 percent open-source, can be downloaded free and is licensed under the Apache Foundation software model. It enjoys the support of a consortium of more than 200 companies and over 900 code contributors.

It began in July 2010 as a joint initiative of Rackspace and the United States of America’s National Aeronautics and Space Administration (NASA). It was intended to make the deployment and management of pools of cloud computing resources and services running on low cost commodity x86 and ARM hardware easy and affordable for organisations of all sizes. 

Just three years in some of the more notable users of OpenStack include AT&T, CERN, Deutsche Telekom, HP, and IBM. Backers have broadened to include AMD, Brocade, Canonical, Citrix, Cisco, Dell, EMC, Ericsson, Intel, NEC, NetApp, Red Hat, SuSE, VMware and Yahoo.

And for those hedging their bets, OpenStack has been developed with an eye on the API's (application programming interfaces) of the Amazon EC2 and Amazon S3 cloud services, such that client applications written for Amazon Web Services can run on OpenStack with minimal porting effort.

Under the hood, the OpenStack ecosystem is made up of three core components: compute, networking and storage. All can be accessed via a dashboard - a browser-based graphical user interface (GUI) “console” - or via a command line interface (CLI). Let's take a look at each in detail.

Compute

The OpenStack compute component is called Nova, and its role is the control and orchestration of the fabric of the infrastructure-as-a-service system.

Nova manages and automates pools of compute resources across one or more systems, usually in a cluster configuration. And Nova is compatible with everything from bare metal servers through to open source hypervisors (XenServer, KVM and LXC), plus hypervisors from proprietary vendors such as Microsoft's Hyper-V and VMware's ESX.

Nova is designed to scale horizontally on low-cost commodity hardware, without requiring any proprietary software or hardware - but proprietary software can be slotted in if you have existing investments in place.

Storage (Swift & Cinder)

OpenStack offers both block and object storage services in the form of the Swift “object storage” system and Cinder “block storage”.

Swift is a distributed highly redundant object storage system. Objects and “files” are written to multiple disks and usually across multiple servers to ensure data redundancy and integrity. If one system fails, the data on the failed system will be available across two or more servers in the cluster.

Much like Apache Hadoop, OpenStack has data replication and distribution built into its DNA. The management of objects and files across a cluster is performed in such a way that low cost commodity servers can be used without resorting to the more expensive pattern of using redundant array of inexpensive disks (RAID) or high availability server hardware.

Where Nova was contributed to OpenStack by NASA, Swift originated from a Rackspace project called Cloud Files.

Cinder provides persistent block-level storage devices for use by OpenStack compute instances. When Cinder storage is presented to compute instances it looks, feels and behaves like traditional enterprise-grade block storage.

Cinder is the component of OpenStack that controls the creating, attaching and detaching of block storage “devices” to servers.

Block storage volumes are integrated into OpenStack and managed either via the web dashboard or the CLI to provide persistent block-level “volumes” such as boot disks for virtual machines.

The Cinder storage system can use platforms such as Ceph, CloudByte, Coraid, EMC’s VMAX and VNX, FlusterFS, IBM’s Storwize, SAN Volume Controller and XIV Storage System or GPFS, Linus LIO, NetApp, Nexenta, Scality, Solidfire, HP’s StoreVirtual and StoreServe 3Par products, or the native Linux volume manager (LVM).

Cinder's block storage is usually used for “local” volumes for operating systems like Linux and Microsoft Windows, as well as for database storage.

Cinder also includes full support for traditional block storage management features such as snapshots and volume resizing.

Networking (Neutron)

The OpenStack networking component is called Neutron (formerly Quantum). Neutron is the system that manages all aspects of the OpenStack networking infrastructure such as IP addresses, routing, and switching.

Neutron provides the full suite of networking capabilities a cloud computing platform requires, either in flat network topologies or segmented topologies such as VLAN’s to allow the separation of servers and traffic, traffic management, quality of service and security.

Neutron offers software defined networking (SDN) capabilities through systems like OpenFlow. Neutron also allows for additional network services such as firewalling, virtual private networks (VPN’s), intrusion detection systems (IDS), and load balancing to name a few, allowing for enterprise-grade secure multi-tenancy. It can manage everything from small networks through to data centre-wide or international multi-site deployments.

Highly redundant networking can be achieved using floating IP’s that allow traffic to be rerouted dynamically within the OpenStack switched or routed network fabric to cater for maintenance outages, performance degradation or hardware failure.

Beyond the “core”

Supporting the three core components of the OpenStack ecosystem are a number of other key toolsets that can be bolted on.

These include machine image management services (Glance), a web administration dashboard (Horizon), an identity management for authentication and authorisation (Keystone) service, a metering service (Ceilometer) and service orchestration system (Heat). There are dozens more.

The number of value-add components are growing monthly, the larger among them announced during a six month release cycle that heralds the introduction of a new version.

The eighth and most recent 'Havana' was after an unincorporated community in Oregon and offers over 400 feature blueprints. The next to come is named 'Icehouse' after a street in Hong Kong, the location of the last OpenStack Summit.

Read on for OpenStack's compatibility with existing 'cloud' technologies, plus our frank assessment of its growth in Australia to date...

Sideways compatibility

The 'open' nature of the OpenStack framework is perhaps its most compelling feature.

The framework has been put together in such a way that components of OpenStack can be deployed in existing technology environments. While not 100 percent complementary, IT shops certainly don’t have to deploy the entire OpenStack system to replace existing infrastructure, but can use it to augment and complement existing investments.

Consider, for example, Swift Stack. Swift Stack is a platform that offers web-based storage but not OpenStack's compute component. If your organisation had an immediate requirements for low-cost, easy to use, high performance storage, you could deploy OpenStack and decide to only make use of Swift as a cheap, online backup or archive.

Further, organisations that have built high-performance hardware infrastructure stacks for the hosting of  production workloads, and have licensed VMware's or Microsoft's hypervisor solutions for DEV, TEST, UAT and DEMO environments. Ultimately, OpenStack can be employed to provide that same functionality at a lower cost or even on old hardware that is out of warranty and support.

Once deployed, OpenStack will happily deploy system instances across multiple hypervisors, allowing you to integrate and manage existing disparate VMware or XenServer environments, as well as hypervisors running under OpenStack. It offers the ability to  manage multiple technology stacks from one “pane of glass”.

But it doesn't have to be seen as an alternative or a replacement to existing investments in VMware ESX, Hyper-V or XenServer.

Australian organisations have been aggressive adopters of virtualisation software, but few have followed through with the more rounded 'virtual data centre' solutions VMware, Microsoft and others have pushed out for the build of a more complete cloud computing platform.

If anything, OpenStack is just as much a threat to networking and storage providers - boasting SDN capabilities that - in the most rudimentary deployments - could make it harder to sell traditional hardware such as routers, switches, network attached storage (NAS) and enterprise storage area networks (SAN).

Australia's sunken investments in server virtualisation may have perversely slowed the push for software defined networking, which is taking far more of a hold in the United States.

We are yet to hear of many large corporate deployments in Australia - but we nonetheless are building the necessary ecosystem for that to happen. 

The National eResearch Collaboration Tools and Resources (NeCTAR) project has a large-scale deployment of OpenStack in production. NeCTAR's OpenStack build complements supercomputing facilities old and new, hosting research applications and fostering research collaboration. 

Using little more than a web browser, Australian researchers are now able to wield the power of the cloud from anywhere, using only their email address and password from their research institution to authenticate access to cloud computing across this national OpenStack deployment.

Once fully deployed, the NeCTAR research cloud will consist of eight nodes distributed around Australia as part of a federated cloud. A user will be able to specify the zone their virtual machines should be located in for best network performance.

The lead node will have an initial capacity of around 4000 cores. The completed cloud could have as many as 32,000 cores. It will allow Australian researchers to concentrate on their work rather than obtaining funding for the computing resources or being bothered with the provisioning and maintenance of hardware.

Almost as important, the NeCTAR research cloud is one of the few projects fostering OpenStack skills in Australia. Where many corporates are only just starting to give developers permission to experiment with public cloud services, OpenStack provides a faster path to deploying an experiment into production behind the firewall. 

Early users report that one barrier to deploying OpenStack is the technical skill set required to design, deploy and manage the platform. It is still not quite the point-and-click process more mature platforms like XenServer or VMware have been able to achieve. Further, many enterprise buyers are restrained by sunken financial commitments such as service agreements and purchasing contracts with existing commercial suppliers.

OpenStack has arrived later than its commercial counterparts, and will take time to gain traction. But it's only a matter of time before the less nimble Australian organisations give OpenStack a try.

Interested parties might want to check out the DevStack.org and TryStack.org sandbox projects.

Not to be out-done, this week iTnews plans to give readers their first taste of OpenStack with our own super-shrunk flavour being made available for download. Stay tuned.