The University of Technology Sydney has migrated its on-premises data warehouse to the cloud as data science becomes increasingly important to the university both academically and operationally.
The university recently shifted its legacy platform to AWS Redshift, which launched locally in February 2013, in a drive for lower costs, more capabilities and better performance.
It had previously used an on-premise Oracle relational database along with WhereScape RED and the IBM Cognos Suite for reporting.
Its past use use of data warehousing had primarily focused on storing historical data for administrative purposes.
But the university is becomingly increasingly aware of the usefulness of data and analytics, UTS web and applications manager David O'Connor told iTnews, and two years ago decided to rethink its entire approach to data warehousing.
Out of this process, the decision was made to implement a new platform that could support UTS’ needs into the next decade.
“While the data warehouse is primarily positioned as a consolidated central source for operational reporting and analytics, we do see a future in which the lines increasingly blur into research and teaching activities,” O'Connor said.
The key priorities for UTS in assessing various platforms were scalability, cost and performance. It opted against upgrading its legacy on-premises infrastructure due to its physical constraints.
“The BI team running a large query could cause problems for other development teams. So they would have to compromise and either limit the complexity of their queries, or run more complicated queries outside office hours, and all of this just stifles innovation and creativity,” O'Connor said.
“On the cloud platform, those constraints simply don’t exist, and that speeds up the rate of development and allows us to innovate without fear of failure causing problems for others."
IBM Cognos remains on premises, but UTS' plan is to move the rest of its business intelligence ecosystem to AWS in the coming months.
Jumping in with new technology
Given Redshift was relatively new at the time the university decided to sign up to the platform, UTS found itself faced with challenges like a lack of external consultants and vendors able to assist with the migration.
According to O'Connor, the power of staff enthusiasm as well as unique attributes of the higher education environment helped pull off the move.
“At the time we made the decision, Redshift was very new to the Australian market. What made the difference in the migration was having key staff who had a real passion and desire to make it work, and who had the skills to make it work. The move was not capital intensive, but it was skills intensive,” O'Connor said.
“We worked with Amazon a lot, and they’ve been excellent, but there weren’t many other vendors or contractors we could lean on.
“Because it was a new product, the partners didn’t have any experience with it either. So we were lucky in that we had been able to build an excellent team who did have some prior Redshift experience but who could also figure out new approaches for themselves."
The project team - led by BI and data warehouse manager Susan Gibson - also took advantage of the academic technology experts right at their doorstep.
“There is a lot of rich technology and business expertise for us to tap into. Being a university of technology gives us access to colleagues who are leaders in fields such as data science, human centered design, and innovation," O'Connor said.
The challenges of a new platform were compounded by the fact that Redshift was the first time UTS had deployed a core IT system in a public cloud environment.
“There were many internal governance and architectural questions that needed to be addressed,” O'Connor said.
“We had to build the road, so to speak, and work out how to approach things like security in this type of cloud setup.
“It was quite a bold decision that we feel has really paid off. This is the platform which will allow UTS to innovate with our data in the coming years.”
The platform returns reports for users in seconds rather than 40 minutes, according to O'Connor, and will allow the IT team to shift its focus more to predictive analytics.
“There is a huge amount of potential for UTS to leverage our data in new ways to help our business in areas as diverse as student retention, adaptive learning, digital marketing, academic integrity, space utilisation and building automation – but the list is really endless and limited only by the imagination," he said.
“As our business processes across the university increasing become digitalised, the potential for leveraging data to understand what is happening and to adapt accordingly just grows and grows. The good news is that our new cloud platform can support all of this and more.”
