Yahoo7 takes Docker into high-volume production use

Yahoo7 is embarking on its first major production use of Docker containers as part of a project to rebuild its portfolio of websites.

Yahoo7's Paul Russell.

The media company is replacing an “old, legacy, internal Yahoo [PHP] framework” used to deploy its current websites with a new node.js framework.

Where its previous sites ran as a “stock standard LAMP-type implementation” on virtual machines, the new sites are architected with mostly reusable components housed in Docker containers and hosted on an internal Yahoo cloud.

Yahoo7’s CTO Paul Russell declined to talk about the cloud’s architecture, apart from saying it was “built up from a range of open source technologies and is centred on Docker".

The overhaul of Yahoo7’s media properties marks a major expansion for Docker in the company’s environment.

“It’s our first use of Docker in anger,” Russell said.

“We’ve been dabbling with it for a while. We’ve had a few small pieces running in production using it but this is the first real high volume application of that environment for us.”

He declined to size the Docker environment but said the company had fluctuating capacity requirements as a news outlet.

“We do get some fairly sharp rises as a breaking news website,” he said.

“Because [the new sites] are in containers it’s really quite quick for us to go and spin up additional [containers] so we can increase capacity should we require it.

“[That said] we do run with quite a significant amount of headroom just because you can’t plan some of these breaking news events.”

The first site to be ported onto the new stack and Docker environment is Be, an entertainment-oriented site and “one of our smaller ones”, a purposeful decision as it beds down the new architecture.

Yahoo7 will now pursue a reasonably aggressive transition for some of its other websites, including news, sport and its Plus7 TV catch-up site, though the latter will require some additional work.

Once the transition is finished, the company’s portfolio of sites will be largely made up of the same reusable components – with some small customisations to differentiate the properties.

Russell believes the standardisation will afford Yahoo7’s developers more time to create new features, since those features now only need to be deployed once.

“From a development point of view we’ve made [our environment] scalable in a way the previous stack wasn’t,” he said.

“With the previous stack each site was built bespoke for that particular property. They didn’t reuse very much [code] at all, whereas the way we’ve approached this new build is that we’re building everything as reusable.

“News, sport etc. are all relatively similar websites in terms of their underlying requirements, and we can quite easily add a small layer of customisation so those new sites don’t all look the same, yet they share the vast majority of the code underneath.”

Once Yahoo7 is “over the hump” of re-platforming its websites, attention will turn to a deep pipeline of feature requests.

One for which some work has begun is to improve the recommender system used to keep online video viewers watching.

“One of the areas that’s really important to us is the recommendations that come up if you watch a video,” Russell said.

“What we had previously in that area was something along the lines of here’s another video or set of videos that are in the same genre.

“We’re now using a lot of the data that we collect … to start powering those recommendations and make them more relevant to people.

“We’re going to continue to iterate on that and use more machine learning type algorithms to make those even better.”