All-flash storage slowly making its mark on Aussie enterprise

Property information and analytics firm CoreLogic RP Data uses flash for its “platinum” storage tier, which is mostly reserved for I/O intensive, ETL-based workloads.

CoreLogic RP Data – ‘platinum’ tier now all-flash

The company, which is perhaps best known as a repository for property data and analytics, runs a tiered storage architecture consisting of flash (platinum), a fabric-attached storage cluster (gold, silver) and disk (bronze), and is presently a NetApp shop in the top three tiers.

Its workloads are typically “bursty”, Head of ICT Adrian Jansz said.

“Demand for reports can burst from 30 to 40 an hour to a couple of hundred,” Jansz said.

“Customers have an expectation that they will get a report instantaneously when they click for a report. They don’t want to be waiting minutes for it, they want it immediately.”

The flash tier is used to power the database processing of ETL requests, as well as to handle vast quantities of property image data.

“We store hundreds of millions of photos of different types of property,” Jansz said. “We have to ensure they are captured, thumbnailed and pushed into the appropriate location.”

Less process-intensive and time-critical workloads are targeted at the gold – or even silver – tier.

“We use the gold tier predominately for performance work that doesn’t really justify sitting on the really fast, more traditionally expensive flash disk,” Jansz said.

“This includes data processing from A to B or internal analytics processes that don’t need a result in seconds – where a response in minutes is OK."

CoreLogic’s all-flash array has 12TB of raw space. The company uses dedupe but doesn't see high dedupe ratios due to the types of data stored, according to Jansz.

However, it keeps about 20 percent of space available to meet unexpected growth.

Future decisions about whether flash is applicable in lower storage tiers will depend largely on the cost coming down, and the types of projects pursued by the business.

One option is to run most storage as platinum, gold or silver. Current bronze workloads could either be migrated to silver, or pushed to an AWS S3 bucket instead.

Jansz said CoreLogic is currently running proof-of-concepts on the feasibility of shuffling the lower tier storage to AWS – effectively moving to a hybrid on-premise/cloud storage model.

“The answers or outcomes will drive the next steps and discussions around where to next and where we can save some costs on the bronze side,” he said.