Social networks stick with MySQL

Several of the world's largest users of the MySQL database have committed to updating their systems to new releases — but each has left questions marks over whether they will support the Oracle-controlled platform into the future.

Twitter confirmed at the MySQL Connect conference on Saturday that it intends to upgrade its 1000-plus servers from its modified version of MySQL 5.1 to the current version MySQL 5.6 over the next six months.

Twitter and LinkedIn count among several web-scale MySQL users that have waited on the sidelines for close to a year while Facebook attempts to shift its significantly larger deployment to MySQL 5.6 first, a project that has clocked up 12 months so far.

Facebook is widely seen as the largest and most complex MySQL deployment, and is often the source of new innovations fed back into the community platform.

Harrison Fisk, manager of the data performance team at Facebook told the MySQL Connect conference that the company was "50 to 60 percent the way through" an upgrade from its heavily modified version of MySQL 5.1 to 5.6. 

The upgrade is complicated by the scale of the aforementioned modifications Facebook has made to the software to meet its challenges.

Online payments platform Paypal, meanwhile, intends to wait out the upgrade to 5.6 until general availability of the next version (5.7).

Future database of choice?

Each of these web-scale customers, however, continued to put Oracle on notice that their support was not guaranteed into the future.

"The move to 5.6 has been a long effort," said Mark Callaghan, a software engineer at Facebook.  "It has required a lot of investment.  We've made it better, Oracle has made it better."

But there was "no guarantee" MySQL will be the database of choice at Facebook into the future, he said. "We're always evaluating and comparing to the alternatives."

Calvin Sun, senior engineering manager at Twitter said MySQL's 20 year legacy had forged it as a "stable and reliable" data store.

"But all of us - Twitter, Facebook — we are all looking at alternatives," he said.

Sun revealed during his presentation that Twitter designed its application with flexibility in mind.  Twitter manages sharding at the data services layer (Gizzard), for example, so that it isn't tied down to a specific back-end. 

"We currently use MySQL, one day we might use others," he said.

To fork or not to fork

Over the past 12 months, MySQL users concerned over Oracle's control of the project have been offered a new alternative, MariaDB.

MariaDB was developed by a team led by MySQL founder Michael 'Monty' Widenius, who protested against Oracle's acquisition of Sun and control of the software ecosystem he'd helped found.

Under Widenius' leadership, MySQL became the database most often used in the LAMP web application stack (Linux OS, Apache web server, MySQL database and Perl/Python/PHP application layer).

He had no scruples about selling his technology to Sun Microsystems for US$1 billion in 2008, but waged war when Oracle purchased Sun in 2009 after Oracle had earlier purchased the InnoDB database engine.

For those companies willing to switch, MariaDB is promised to be a "drop-in replacement" for MySQL, and has already picked up some big scalps — most notably Google, an important contributor to the MySQL effort, as well as Wikimedia (Wikipedia). 

Google's switch may have been motivated by a lawsuit filed by Oracle over alleged use of Java patents in Google's Android operating system.

MariaDB has also been bundled into Linux distributions — Fedora and openSUSE initially, followed by Red Hat last month -—at the expense of MySQL. 

Today iTnews asked Tomas Ulin, vice president of MySQL engineering at Oracle, whether these moves were political or practical. 

"Some of the movement around what the Linux distributions have done is a political move," he said.  "Some of them sense perceived openness from [the] MariaDB team and that is perhaps what they value."

Twitter's Calvin Sun agrees. "Its mostly a political issue, much less a practical one," he said. "I have friends on both sides. There are talented people on both sides."

The main differences between each platform on a technology level, he said, was MariaDB's expertise in database replication and optimisation versus Oracle's expertise in performance, particularly with relation to the InnoDB database engine.

Today Oracle announced that the MySQL 5.7.2 DMR (development milestone release) clocks InnoDB at 95 percent faster than 5.6, which makes it 172 percent faster than 5.5.

Mozilla, the open source community that builds the popular Firefox browser amongst other web software, used MariaDB 5.5 for a short period of time, but is now in the process of upgrading to MySQL 5.6 to take advantage of new features around subquery optimisation and performance schema. 

"We look for what's best, politics aside," said Sheeri Cabral, database architect at Mozilla."Maria 10 isn't [General Availability] yet, right now 5.6 features aren't even in it. We can't wait for them to catch up right now." 

Wish list

While impressed by these speeds, performance wasn't number one on the wishlist of any of the large web-scale MySQL users when asked what they required from future versions.

More commonly they sought support for NoSQL database extensions and the ability to use storage engines other than the Oracle-owned InnoDB — both areas in which MariaDB has made gains.

"InnoDB is not optimised for all workloads," Callaghan said. "It would be great to make it easier for innovative storage engines to participate in MySQL, to keep MySQL relevant at web scale in the future."

Sun told iTnews that while InnoDB's performance improvements were important, the next database architecture is likely to be chosen on the basis of cost reduction rather than speeds and feeds.

"Today we use SSD for low latency, and that hardware requirement is very expensive. One potential alternative in the future would be the combination of a NoSQL database combined with cheap disk on a large scale," he said.

"But I don't see another solution [to using MySQL] in the short or medium term."

Who will pay for MySQL?

Oracle currently invests directly in both improving the community version of MySQL and the paid enterprise version, which adds support and additional management features such as audit logging and authentication, high availability features, the Workbench GUI and faster backup tools.

Sun Microsystems had only committed to its current licensing schema until 2015 and Oracle has not given any formal indication of what happens after that.

Oracle has used keynotes at MySQL Connect for two years running encouraging users of the community version to upgrade to the fee-paying enterprise versions. 

Oracle chief corporate architect Edward Screven said the MySQL development team is twice the size as it was before the Sun acquisition. "We have spent more money. We have hired a lot more people. We've tripled the QA team. We've made it much faster. And we've made it, more importantly, better quality by creating a lot of test cases and fixing a lot of bugs."

Many of the largest MySQL users — Twitter included — do not currently pay Oracle for an enterprise licence. Twitter, like Facebook, prefers to build their own extensions and customisations off the community version. 

Oracle's Ulin told iTnews the "commitment to MySQL from Oracle is what it is" and that no change is strategy has been discussed.

"We are continuing to invest in MySQL, and we investing more and more because we believe in the technology," he said. "I don't see why Oracle would want to do anything different."

But asked whether license fee revenue is matching the cost of these engineering efforts, Ulin admitted that "not even I know if we're profitable.

"I can only assume Oracle thinks it is profitable to invest in," he said. "My personal view is it makes sense to continue on the path we have taken."

Read on for the mind-blowing numbers Facebook and Twitter reported on their database sizes…