Service NSW moves down Kubernetes path

Service NSW is refactoring its digital applications into microservices that run in containers to make them easier to update and manage, and to solve problems in the underlying infrastructure hosting.

Digital products director Rahul Dutta said the NSW government-owned agency is a year into the process of breaking up monolithic applications into microservices that are deployed in containers that are then orchestrated using Kubernetes.

Dutta said that Service NSW had created a number of digital services over the years, such as the digital driver’s licence that goes into full production availability in August, as well as Fuel Check and Feedback Assist.

The NSW government uses a digital marketplace called OneGov as a central access point for these and other digital applications.

“OneGov manages hundreds of digital applications that are deployed on thousands of virtual machines,” Dutta said.

“The way they were deployed was within the two [NSW] government data centres on OpenStack private cloud.

“Over a period of time, it was getting very difficult to not only manage, but also to patch, secure and operate these virtual machines. We're patching those nonstop 24x7.

“It literally felt like a hamster wheel, and it still does.”

A virtual machine could be home to one or more digital applications.

“Many applications shared the same web servers - virtual machines - making it very hard to deploy code to production without downtime, and [without the changes] affecting each other,” Dutta said.

A small change within a single application could also cause problems in other parts of the application.

“The monolithic nature of our applications meant it was almost impossible to push out a single feature without having to test the entire application,” Dutta said.

“A lot of revision testing was required every time a feature was released.

“It was also extremely hard to deploy to production without compromising the availability of services. You'd always have to bring down a service while you were doing a deployment, and … sometimes that’s not a very feasible thing to do when you have 24x7 availability requirements.”

In addition, if one part of an application was in high-demand, the entire application had to be scaled.

“The VM-based approach meant it was not easy to scale services without cloning an entire VM and using up lots of expensive resources,” Dutta said.

“Auto scaling was to be honest, very, very difficult. And if one part of an application or a feature in an application was in high demand, we pretty much had to scale the entire application up to meet that demand, and not just that feature.”

Service NSW encountered the scaling challenge with Fuel Check, its real-time fuel price check comparison app and service.

“One of the most popular features is people want to look up the fuel prices near where they live, so the call to Fuel Check that looks up ‘cheap, easy prices near me’ probably gets 1000 times more hits than any other feature,” Dutta said.

“In a monolithic application, we would have to scale the entire application up.

“Using microservices, all you do is basically scale up the microservice that's called 'find cheap fuel price near me' and that could run in one thousand pods [groups of containers on a single host] and look after millions of people.”

Dutta said that Service NSW “secret sauce” in solving these problems has been a microservices-based architecture and Kubernetes.

“For the last 12 months or so, what we looked at was microservices and Kubernetes being the silver lining, which provided us some sort of optimism around how we could solve some of these problems,” he said.

“So what we did was we deployed multiple large Kubernetes clusters, and we also started refactoring our applications into microservices.”

The clusters run on top of banks of virtual machines.

Dutta said that, so far, the move to microservices and containerisation was working out.

“With microservices, we could develop different features of our applications as independent microservices,” he said.

“Hence, different teams could work on these features and they could be deployed independently, which means for example, if you take digital driver’s licence as an application, if you wanted to check a licence and we wanted to just concentrate on that feature, we could roll that feature out very easily, independently of any other features of the digital licence.”

Dutta said that Service NSW now operated as a “multi-speed organisation with independent autonomous teams” of developers.

“Now, teams are not getting blocked by each other. They're working on features in parallel, they're deploying in parallel, everyone's moving quickly, and work flows smoothly,” he said.

Service NSW has been able to automate more test functions, since it is deploying much smaller pieces of code.

It can also scale those microservices independently of others that make up a single digital application.

Infrastructure wins

Dutta said that Kubernetes had helped Service NSW contain VM sprawl in its environment.

“By moving applications to Kubernetes, we were actually able to cut down the number of virtual machines,” he said.

Applications are also now isolated from one another on different pods. ”They’re not treading on each other's toes,” he said.

Containerising its applications also opens Service NSW up to more options around future workload portability.

“We wanted to make sure that we could run our workloads across multiple data centres, but also have the option to port the services to the cloud easily,” Dutta said.

“That was a really difficult challenge with big virtual machines running lots of applications, because it requires careful orchestration and it's very hard to do.  

“One of the great advantages of Kubernetes is you can have Kubernetes clusters running within your data centre or in Amazon or in Google - wherever you want - and if you wanted to port a microservice or a pod from one cluster to another cluster, it's really easy.

“So it allows you to move applications around and move workloads around very easy.”