iTnews

ASIC spins up protected AWS data lake

By Justin Hendry on Oct 26, 2020 11:58AM
ASIC spins up protected AWS data lake

In Australian first.

The Australian Securities and Investments Commission has stood up a protected-level certified data lake in Amazon Web Services to put its data on the financial services industry to better use.

The data lake, which is understood to be the first AWS data lake in the country to achieve a protected security classification, went live last month as part of a major program of work to uplift the financial regulator’s data and analytics capabilities.

It has been built by Sydney-based cloud and security service provider Cloudten, which was acquired by cyber security company CyberCX last week, over the past 10 months under a $1.2 million contract.

Cloudten worked with ASIC to design and deploy the solution on the agency’s protected-level AWS environment, which it set up early last year, using an eight-person team of data architects, engineers, analysts and data scientists. 

An ASIC spokesperson told iTnews the data lake started life as a proof-of-concept (PoC) in February, which was “used to refine the requirements and design of the final solution, which went live in September”.

With data an increasingly integral component of the agency’s work, the regulator plans to use the platform to store, consume and manage its “massive” range of financial datasets in order to gain greater insight.

The lake is capable of ingesting, processing and storing structured, semi-structured and unstructured data types from internal ASIC systems such as databases and SharePoint, as well as external feeds via SFTP [SSH file transfer protocol], API calls and stream data.

“Our data lake’s modern architecture allows us to store and process structured and unstructured data at scale to assist in meeting our data and analytics requirements,” the spokesperson said.

Fraud detection, general market awareness and predictive modelling across superannuation, mortgages and business lending are some of the immediate areas that are expected to benefit from the platform.

The platform has also provided the agency’s analysts with “new tools and environments to support their work”, with a series of dashboards and a natural language processing solution also delivered as part of the go-live. 

Speaking at AWS’s public sector summit last week, Cloudten managing director Richard Tomkinson touted the project as the “first protected-grade data lake in AWS in Australia”, without specifically referring to ASIC.

“We recently worked with a large federal government agency to design and deploy the first protected grade data lake in AWS in Australia. This solution was built inside an established customer AWS tenancy that had previously been deployed,” he said.

However, according to the Commonwealth procurement website AusTender, Cloudten’s data lake contract with ASIC is its largest with a government agency to date. The contract’s term also aligns with a published case study by Cloudten.

Tomkinson said the platform employed a number of pre-certified native AWS services such as Glue, Athena and Kinesis Data Streams, as well as third-party products like Databricks’ Unified Analytics Platform.

“Data is cleansed and transformed through a series of S3 buckets before being presented for visualisation [in] business intelligence tools such as Amazon QuickSight, Tableau or Qlik,” he said.

“Databricks is responsible for the bulk of the advanced analysis processing and there are close integrations with AWS services such as Amazon Glue and Athena. 

“A significant proportion of the workflow automation and orchestration processes within the lake environment run on serverless AWS services such as Elastic Container Services (ECS) and Elastic File System (EFS). 

“From a data governance perspective we have worked with a number of commercial products including Collibra, Talend and Alex Solutions.”

Got a news tip for our journalists? Share it with us anonymously here.
Copyright © iTnews.com.au . All rights reserved.
Tags:
amazon web servicesasicawscloudcloudtendatadata lakefederal governmentgovernmentitprotected

Partner Content

Top 5 Benefits of Managed IT Services
Promoted Content Top 5 Benefits of Managed IT Services
"We're seeing some good policy put in place, but that's the exception"
Partner Content "We're seeing some good policy put in place, but that's the exception"
Avoiding CAPEX by making on-premise IT more cloud-like
Promoted Content Avoiding CAPEX by making on-premise IT more cloud-like
Security "mindset shift" needed to protect organisations
Promoted Content Security "mindset shift" needed to protect organisations

Sponsored Whitepapers

Planning before the breach: You can’t protect what you can’t see
Planning before the breach: You can’t protect what you can’t see
Beyond FTP: Securing and Managing File Transfers
Beyond FTP: Securing and Managing File Transfers
NextGen Security Operations: A Roadmap for the Future
NextGen Security Operations: A Roadmap for the Future
Video: Watch Juniper talk about its Aston Martin partnership
Video: Watch Juniper talk about its Aston Martin partnership
Don’t pay the ransom: A three-step guide to ransomware protection
Don’t pay the ransom: A three-step guide to ransomware protection

Events

  • 11th Annual Fraud Prevention Summit 2022
  • iTnews Benchmark Awards 2022 - Finalist Showcase
  • IoT Impact Conference
  • Cyber Security for Government Summit
By Justin Hendry
Oct 26 2020
11:58AM
0 Comments

Related Articles

  • ASIC starts using its protected AWS data lake at scale
  • Govt certifies first four 'strategic' cloud providers
  • Defence reveals plans for secret cloud environment
  • DTA renews gov-wide AWS deal for $174 million
Share on Twitter Share on Facebook Share on LinkedIn Share on Whatsapp Email A Friend

Most Read Articles

NBN Co sizes up six-figure customer exodus a year to fixed wireless

NBN Co sizes up six-figure customer exodus a year to fixed wireless

NBN Co to cut 160 applications under $200m IT simplification

NBN Co to cut 160 applications under $200m IT simplification

NBN Co's 250Mbps and gigabit growth is finally clear

NBN Co's 250Mbps and gigabit growth is finally clear

What to expect from the incoming Labor government

What to expect from the incoming Labor government

Digital Nation

COVER STORY: A Year in the Metaverse
COVER STORY: A Year in the Metaverse
Why do DeFi and DAOs matter to business?
Why do DeFi and DAOs matter to business?
COVER STORY: Data and IoT set digital agriculture on a sustainable future
COVER STORY: Data and IoT set digital agriculture on a sustainable future
CTO Juergen Mueller offers a glimpse into SAP's metaverse play
CTO Juergen Mueller offers a glimpse into SAP's metaverse play
Lendlease launches its own metaverse in Milan
Lendlease launches its own metaverse in Milan
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in any form without prior authorisation.
Your use of this website constitutes acceptance of nextmedia's Privacy Policy and Terms & Conditions.