Docker opens code behind native laptop apps

Docker has open-sourced three of the architectural components behind its native apps for Mac and Windows laptop users.

The apps, which were launched into beta in late March, are designed to make it far easier for developers to build, assemble and ship code in Linux containers from their laptops.

Docker is a way to package and ship code in “containers” that can then be easily moved between clouds or virtual machines.

In a keynote at open source conference OSCON, Docker founder Solomon Hykes said the Docker for Mac beta alone saw 30,000 sign-ups from developers “in the first 24 hours”.

“Then we started sending invites and the response has been extremely positive,” Hykes said.

Healthdirect’s DevOps solution architect Scott Coulton recently identified himself as a Docker for Mac beta user. Healthdirect is an Australian government-funded provider of health apps.

“Some of us were lucky enough to be on the Mac beta,” Coulton said in a Docker webinar.

“For the rest of my team that don’t have it … they’re all jealous that I’m actually testing it. When that product comes out [of beta it’ll be] awesome. It solves a lot of problems.”

Hykes said the native apps – particularly for Mac - came as a response to “dozens and dozens” of complaints from developers.

“The problem we’ve encountered in the last year is that using Docker on Mac is really hard,” he said.

“That’s been a recurring problem for us and we’ve wanted to address it for some time.”

But Hykes said making Docker run seamlessly and natively on Mac “takes a lot of hard engineering work … under the hood”.

“Of course, Docker is all about running Linux containers and Mac does not ship with Linux so we had to bring our own Linux, but then of course we need to virtualise it so we need to bring our hypervisor,” Hykes said.

“That hypervisor needs to somehow communicate over the network so we need to bring our own VPN/networking stack, and all of that needs to communicate with external programs, with Docker CLI, [so] we need some sort of data interface for that.”

The company had to build these three components – the hypervisor, VPN and data service interface – to make the native apps possible.

It had some help on this front from Unikernel Systems – which Docker bought earlier this year – and which Hykes describes as a small company of “really badass operating system and hypervisor hackers”.

Those components – HyperKit, DataKit and VPNKit – have now been open-sourced, with Hykes pushing them to Github live on stage at OSCON.

He said the desire to open source was “because we think others could use them in their own products.”

He singled out DataKit as the “most exciting” and the one he believed had the most potential.

Docker's open source learnings

Docker already runs “about 50 open source repositories, large and small”, according to Hykes.

“About 2000 people contributed to those repositories and my favourite metric is that on Github across these repos 18,000 issues have been opened.

“That’s 18,000 people saying either, ‘I don’t like this’ or ‘Please add that’ or ‘Hey, how can I help?’"

He said Docker presently receives about 1200 patches a month across the repositories from various open source contributors.

“Personally I would say that’s one of the most difficult parts of the week [is] to try and make sense of a patch and put yourself in the head of a contributor and just accept the contribution – that’s a lot of work,” Hykes said.

Hykes said one of Docker’s learnings from the open source space was to be careful which of these patches were accepted, despite individual pressure.

“Whenever you’re being asked to add something or make a significant design decision and you’re not sure [or] really in doubt you should say no because once you say yes you can never take that back [out],” he said.

“A lot of times you’ll rationalise it – ‘I’ll just add this feature in and then later I’ll remove it’. But you really won’t, and it’s going to stay there forever and haunt you. So if in doubt just say no.

“Really that’s a rule of software, it’s not a rule of open source, but the thing with open source is you’re under a lot of pressure to make those decisions at a very rapid pace.

“For us it’s about 1200 times a month, and there are pitchforks because people send patches and they are really upset sometimes when you don’t merge them. But my advice is to resist that pressure.”

He did however also note that open source was “really great for challengers”.

Despite its popularity – billions of downloads, and use by enterprise organisations such as Telstra, Seek, Healthdirect and Everydayhero in Australia – Docker still remains a relatively lean operation. It had just 20 staff in January 2014 and was up to 120 by mid last year.

“If you’re a small team or if you’re punching above your weight in any way, open source is really great because it levels the playing field,” Hykes said.

“Basically it gives you access to an enormous pool of expertise at a really large scale, so that you can focus on solving whatever problem you’re trying to solve, and pretend you have this enormous R&D department to back you up.

“That really allows small teams whether it’s a start-up or a hobby project to really compete with much larger organisations which is really a good thing for everybody.”