Xen bug made AWS, Rackspace data vulnerable to leaking

The Xen hypervisor flaw that forced Amazon and Rackspace to reboot much of their cloud computing infrastructures may also have left customers of both providers vulnerable to data leakage.

Olivier Lambert of the Xen Orchestra Project said the flaw could be used to read the memory of other guest virtual machines or crash them.

"In short, if someone has access to a guest (any DomU) running in HVM (hardware assisted virtualisation), it could crash the host or read any part of the memory: meaning any other guests or the hypervisor itself," Lambert wrote.

Lambert compared the Xen bug to the recent Shellshock Bash command line interpreter security issue, and said the flaw in the virtualisation layer could allow an attacker without any rights or access to a guest to achieve complete hypervisor control.

Known as XSA-108, the bug has been given the common vulnerability and exposures number CVE-2014-7188. It has been in Xen since version 4.1 of the hypervisor that was released in May last year.

Update 03/10/2014: Olivier Lambert contacted iTnews to clarify that his earlier comment was in error, and that XSA-108 cannot by itself be used to compromise a Xen host without root access on the guest. He also said that while XSA-108 isn't comparable to Shellshock, the exploit combined with the Bash flaw can be used to compromise a Xen host.

Discovered by Linux distributor SuSE engineer Jan Beulich, the bug was embargoed until this morning to give users a chance to patch against the flaw before publication.

Rackspace chief executive Taylor Rhodes published an email that he sent to customers yesterday, apologising for the short notice ahead the reboots and the need for the to keep quiet about the bug until it was fixed.

"We want to be as transparent as possible with you, our customers, so you can join us in taking actions to secure your data. But we don’t want to advertise the vulnerability before it’s fixed — lest we, in effect, ring a dinner bell for the world’s cyber criminals," Rhodes said.

"This particular vulnerability could have allowed bad actors who followed a certain series of memory commands to read snippets of data belonging to other customers, or to crash the host server."

It was particulary difficult to deal with the Xen security issue as the patching had to be done on short notice and over the weekend, Rhodes said.

Around a quarter of Rackspace's 200,000 customers were affected by the maintenance. Rhodes said the company "dropped a few balls" while undertaking it, with some reboots taking much longer than they should have and notifications not being clear.

Amazon Web Services evangelist Jeff Barr said the two-stage Xen community disclosure process that provides select organisations early access to security issues, with full publication of these at a later stage, meant his company "couldn't be as expansive as we'd have liked" on why it had to quickly patch and reboot EC2.

Barr suggested that customers who want a more fault-tolerant AWS service run instances in two or more availaibility zones and also "pay attention to your inbox and to the alerts in the AWS management console". 

XSA-108 affects only Intel x86 architecture computers and is described as "improper model-specific register range used for the [Intel] x2 advanced programmable interrupt controller emulation".

Normally, only 256 of the registers are specified for use by the x2 interrupt controller access model. However, Xen code that emulated read and write access to the registers mistakely spanned four times as many or 1024 of these.

Writes to the additional registers alone would not be dangerous, but reading from them could possibly access memory beyond the single page reserved for interrupt controller emulation.

As a result, the advisory warned "a buggy or malicious, full hardware-assisted virtualised guest can crash the host or read data relating to other guests or the hypervisor itself."

Using only para-virtualised guests avoids the vulnerability, but Windows virtual machines required hardware-assisted or HVM support.