Why PayPal chose OpenStack

PayPal's IT team has taken control of its technology release cycle by shifting key components of its IT infrastructure onto OpenStack.

Scott Carlson at VMWorld.

OpenStack is an IT infrastructure framework made up of several components contributed under an open source license by multiple technology providers.

Users of the OpenStack framework are able to pick and choose compute, storage and networking components of what would make up a 'private cloud' stack, as well as open source tools around user identity management and system imaging.

For PayPal, the decision to use components of OpenStack was based around speed to market. It allows the payments provider to untether its release cycle from those of vendor partners.

"PayPal has not historically been known for its fast reactions," PayPal senior engineer Scott Carlson conceded to attendees at the VMworld conference in San Francisco this week. "It has taken us six to nine months sometimes to react to our competitors."

Carlson said PayPal was continually challenged by nimble payments start-ups, and has responded with a series of measures to make its own processed more agile and innovative, including the adoption of open source cloud platform OpenStack.

OpenStack helps support a '15 minute rule' PayPal has set for its developers in an effort to stay ahead of the game.

"From the minute developers finish final QA in our dev space, they have 15 minutes to be live in front of customers on the PayPal website," Carlson said. "That means they put their innovation on the VM, they put the operating system on it, they put the app on it, they put on a load balancer, they fix the firewalls and its up."

As a Silicon Valley company with a wealth of in-house engineering resources, Carlson said  that PayPal doesn't have time to wait for a vendor to update a product before it can improve performance for customers.

"We have been involved in a number of scenarios in the past where you need just one little fix, but its not available until version next. We can't wait for that. We wanted to have access to the code so that we could go and fix it ourselves," he said.

The scalability of the OpenStack platform also offered a crucial advantage for when PayPal is forced to cope with sudden spikes in traffic.

"When [parent company] eBay has a free listing week, they don't necessarily pre-warn us at PayPal, but by the end of that week all of the payments suddenly flood in," said Carlson. "So our volume scales up in one day and goes away the next day."

Stripping out VMware hypervisors?

Carlson also made a point of arguing that, contrary to what has previously been reported, PayPal's adoption of OpenStack did not necessarily mean that it replaced VMware's ESX hypervisors.

"Some people will say that if you go to OpenStack, you have to get rid of ESX. But the answer is no, you don't, because they are not the same thing," he said.

"First of all, ESX is a hypervisor and OpenStack is a set of APIs to manage things. Nova, which is the compute layer in OpenStack, does not equal vSphere, vCenter or ESX. Nova is more like the vCloud Director.

"So when you compare ESX with OpenStack you are comparing the wrong things."

Paris Cowan travelled to VMworld as a guest of VMware.