All-flash storage slowly making its mark on Aussie enterprise

All-flash arrays are still a fair way short of mass adoption, but there is fresh evidence that Australian enterprises are increasing their use of the technology. 

When iTnews last explored the all-flash data centre space a year ago, a few things were clear.

Firstly, “all-flash” was a misnomer. The economics made it unlikely that anyone would use flash for all their storage requirements, except perhaps if they had no legacy disk to sweat.

Second, claims that the cost of flash is falling to disk levels were based on the assumption that customers run dedupe at a much higher ratio than most are willing to consider.

And third, a year ago it was very difficult to find customers that had deployed all-flash arrays, much less all-flash data centres.

One year on, not much has changed on the first front. Flash is still being targeted at the top storage tier, brought in for specific projects or to run high-performance workloads.

Cost also remains an issue, though there is evidence of greater acceptability of dedupe, which will allow customers to bring the cost per usable gigabyte of flash down (even if the economics of flash replacing disk remain largely optimistic).

Where there has been the most movement in the past year is around adoption. Few if any customers were talking about all-flash a year ago; that has changed.

This year we talk to Deakin University, CoreLogic RP Data, Silver Chef and Connective Broker Services about the role they see flash playing in their storage architectures.

Tackling consolidation

One trend that hit the flash storage market with renewed vigour in the past year is consolidation.

NetApp bought SolidFire in December last year (adding to its existing fabric-attached storage and EF-Series platforms) and EMC is now joining forces with Dell (after buying XtremIO in 2012, ScaleIO in 2013 and DSSD in 2014).

“I think the acquisitions of SolidFire and EMC-Dell made have made it very clear that flash is the new platform and the new media going forward,” Pure Storage president David Hatfield told iTnews.

“We would anticipate more acquisitions and consolidation. It’s going to be difficult in our view for other flash-based companies as emerging companies to get out [into the market].”

While Australian customers remain cautious, Hatfield is optimistic that all-flash’s time is coming, believing the market for hybrid arrays that contain both flash and disk storage is dwindling.

“I think the economics have made it such that there’s no reason to do hybrid,” he said.

“It was a clever way to package flash when it was too expensive. A lot of the incumbents [in the storage market] built hybrid arrays, and emerging companies that built hybrid models spent all of their IP optimising cache-hit ratios between flash and slow disk.

“You don’t have to make that trade off anymore.”

SolidFire CEO and founder Dave Wright is also buoyant about the potential for all-flash, blogging earlier this year that “the beginning of the end of the all-flash array market” could soon be upon us.

“For customers, the conversation is switching rapidly from ‘what workloads do I put on flash?’ to ‘What data do I leave on disk?’” Wright claimed.

“The default behaviour is switching rapidly to a flash-first mentality. And that means that all-flash arrays have shifted from a niche technology to the default choice – from a small slice of the primary storage market to virtually the entire primary storage market.

“This shift in customer thinking and buying behaviour marks the true beginning of the end of all-flash arrays as a market in and of themselves.”

Not everyone is so bullish about the technology, but there are signs of greater acceptance in the einterprise. iTnews talked to IT leaders in four organisations about their experiences with all-flash storage. Read on for more on how:

• Deakin University is testing all-flash for a possible shift from hybrid
• Silver Chef is using all-flash for tier one storage
• CoreLogic RP Data shifted its ‘platinum’ tier to all-flash
• Connective Broker Services upgraded to an all-flash SAN.

Deakin University – from hybrid to all-flash?

Deakin University’s next storage refresh isn’t until 2017 but systems team leader Ryan Parker-Hill has already been playing with an all-flash array.

Such arrays were nascent – almost non-existent – when the university last refreshed its storage environment in 2013.

Parker-Hill recalls looking at an early all-flash box. As is still the case, the cost was predicated on achieving a high deduplication ratio on the data it was to store, and Parker-Hill wasn’t able to replicate the box’s claimed performance benchmarks internally.

“Based on that, there wasn’t much point going down that route,” he said. “At the time, hybrid arrays were just the best bang-for-buck.”

In total, Parker-Hill put four arrays through their paces for the 2013 refresh.

From that, the university ended up buying six Nimble Storage CS460G-X2 arrays for production, and some Nimble CS210 arrays for a test environment. All are a hybrid of flash and spinning disk.

Though its next refresh is still sometime away, Nimble has already made it interesting, launching its first all-flash arrays into the Australian market in February this year.

The existing relationship afforded Parker-Hill early access to the new all-flash array technology.

“We kicked it around, and it’s very impressive,” he said. “I can see a place for all-flash arrays in our environment if they’re cost effective. I’d be silly not to consider them. But for now we’ll just wait and see what happens [in the lead-up to the next refresh].”

One potential advantage Parker-Hill sees in the Nimble all-flash array is its ability to replicate to the university’s current Nimble hybrid arrays, “lowering the total cost of ownership of introducing all-flash into our environment”.

“It's the same management software and interface, and data can be moved between the hybrid arrays and the all-flash array in a group,” Parker-Hill said.

“If we were introducing another vendor into our environment the cost of entry would be much higher.”

Though he sees the potential for all-flash, Parker-Hill doesn’t necessarily believe it’s ready to become the sole storage medium in the enterprise.

“The vendors say the pricing of all-flash has come down a long way, but there’s still a bit to go before they are on-par with the price of disk,” he said.

“I don’t think we’ll be moving away from spinning disk completely quite yet.”

Silver Chef – using all-flash for tier one storage

ASX-listed hospitality equipment rental firm Silver Chef brought its first all-flash array online over the 2015-16 New Year period.

The NetApp AFF8020 is mainly used to speed up the backend performance of a contract management system that is critical to the day-to-day operation of the business.

“The SQL database underpinning the contract management system isn’t all that big, but the way queries are structured between it and the frontend means there are a lot of very small reads,” Silver Chef’s infrastructure and operations team leader Ryan Cleary said.

“We wanted to move that onto a faster disk, something with a bit more I/O grunt, but without having to go down the path of buying a big new tier-one storage system.”

The flash array is also used to quickly add small attachments such as PDF addendums to existing contracts.

Speed is crucial because interactions with customers about their contracts typically occur over the phone.

“Our people will be editing contracts as they speak to a customer,” Cleary said. “That’s a time-sensitive process because we don’t want to waste customers’ time waiting to query the details of a contract or to save attachments.”

Silver Chef used a planned weekend-long outage window to shift the SQL database from spinning disk to flash.

“The company doesn’t trade 24x7 so we basically had a whole weekend to do it,” Cleary said.

“But there’ll be a lot of people out there with massive databases that won’t be able to get an outage window like that.”

Five months in and the flash array is performing as expected.

Silver Chef’s array is 12TB of raw space but the effective capacity is “closer to 30TB” thanks to its use of data dedupe, which translates into an approximately 3:1 reduction ratio.

Cleary said that he had seen as much as 6:1 reduction ratios on some LUNs, but he said databases were more likely to be around 2:1, and even that could be “generous”.

“We’re getting varying results with dedupe,” Cleary said. “By and large, though, it makes up for the extra cost per gigabyte for flash storage.”

However, it’s still unclear how far the value proposition for flash stacks up in lower storage tiers that currently rely on spinning disk.

“In brownfield environments, flash is great for a cheap tier-one alternative,” he said. “But if I was in a greenfields environment I can see a reason to go with flash outright.

“For example, we run a few internal web servers. I can’t see a great reason to put them on flash, but if we were starting from scratch I think we could make a case for just having an all-flash array for everything.”

CoreLogic RP Data – ‘platinum’ tier now all-flash

Property information and analytics firm CoreLogic RP Data uses flash for its “platinum” storage tier, which is mostly reserved for I/O intensive, ETL-based workloads.

The company, which is perhaps best known as a repository for property data and analytics, runs a tiered storage architecture consisting of flash (platinum), a fabric-attached storage cluster (gold, silver) and disk (bronze), and is presently a NetApp shop in the top three tiers.

Its workloads are typically “bursty”, Head of ICT Adrian Jansz said.

“Demand for reports can burst from 30 to 40 an hour to a couple of hundred,” Jansz said.

“Customers have an expectation that they will get a report instantaneously when they click for a report. They don’t want to be waiting minutes for it, they want it immediately.”

The flash tier is used to power the database processing of ETL requests, as well as to handle vast quantities of property image data.

“We store hundreds of millions of photos of different types of property,” Jansz said. “We have to ensure they are captured, thumbnailed and pushed into the appropriate location.”

Less process-intensive and time-critical workloads are targeted at the gold – or even silver – tier.

“We use the gold tier predominately for performance work that doesn’t really justify sitting on the really fast, more traditionally expensive flash disk,” Jansz said.

“This includes data processing from A to B or internal analytics processes that don’t need a result in seconds – where a response in minutes is OK."

CoreLogic’s all-flash array has 12TB of raw space. The company uses dedupe but doesn't see high dedupe ratios due to the types of data stored, according to Jansz.

However, it keeps about 20 percent of space available to meet unexpected growth.

Future decisions about whether flash is applicable in lower storage tiers will depend largely on the cost coming down, and the types of projects pursued by the business.

One option is to run most storage as platinum, gold or silver. Current bronze workloads could either be migrated to silver, or pushed to an AWS S3 bucket instead.

Jansz said CoreLogic is currently running proof-of-concepts on the feasibility of shuffling the lower tier storage to AWS – effectively moving to a hybrid on-premise/cloud storage model.

“The answers or outcomes will drive the next steps and discussions around where to next and where we can save some costs on the bronze side,” he said.

Connective Broker Services’ all-flash SAN upgrade

Imagine walking into an established company and within weeks pushing the case for an all-flash storage area network that represented nearly 20 percent of the previous year's entire IT budget.

That’s exactly what Jonathan Meadows did when, in late 2015, he joined mortgage aggregator Connective Broker Services as general manager of technology.

Meadows started with “grand plans” for Connective, but quickly found out that the IT infrastructure had been “quite neglected”, he said.

The infrastructure supports the software development and operations teams behind Connective’s core Mercury software platform, which in turn supports thousands of mortgage brokers across Australia.

However, years of ad-hoc IT development had left the company with a half-executed virtualisation strategy, based on Microsoft Windows Server 2012 and Hyper-V technology but running on ‘Frankenstein’ servers that had been progressively upgraded over time.

Data was spread between local storage and a central NAS device.

The environment “did its job and had supported Connective's growth to a point in time, but it definitely wouldn't have supported the company's vision for the next five years,” Meadows told iTnews.

It was slowing down the entire business, including a key monthly email distribution to more than 300,000 recipients.

So Meadows recommended, and the company purchased, a Pure Storage FlashArray//m 4.75TB flash RAM-based storage area network – an investment that represented nearly one-fifth of the company’s entire technology budget.

The challenge

The first challenge, of course, was telling his new employer of the need to invest in a new storage subsystem. Meadows realised that any productive discussion about future technology direction would require quantitative evidence.

“When you're working with people who have built this from the ground up and they've put in their hearts and souls, you've got to be quite careful about telling them that their app isn't running as brilliantly as it could,” he said.

“If you're going into a conversation like that, you need to have statistics to back that up rather than a gut feel.”

So he set about collecting data to confirm what he had suspected: data latency of well over the desired maximum of 8ms for the network and storage environment.

Veeam’s virtualisation management tools soon revealed an average latency of 26ms. That was causing memory issues, memory overload issues, CPU contention and large disk queues on many of the company's virtual machine hosts.

Having the stats is one thing, but convincing the CEO of the need to spend nearly one-fifth of the company’s entire technology budget is quite another.

“Fortunately, I've got quite an understanding CEO and he's interested in technology,” Meadows said.

“I got staff together and I said that if we wanted Mercury to become the leading aggregator software in the next year and wanted to invest in the growth of the business, we needed to invest in flash and not in traditional disk.”

The pitch worked, and the Pure Storage flash array was purchased. However, during the new system’s testing phase, the previous environment fell over during a regular email run.

Meadows realised it was time to go live on the flash storage. An afternoon-long migration transferred most of its VMs to the new environment and by the next morning, Meadows was receiving feedback that the system was running much faster.

With average latency now less than 1ms, the bottlenecks of the old platform have disappeared and regular processes, such as snapshots taken automatically every 10 minutes for backup, have “put us in a fantastic position”, he said.

Other built-in features, such as a remote monitoring service, have picked up on potential issues well before they eventuated.

The flash platform also “instantly ticked our requirements for database encryption and data encryption at rest, and filled in our disaster-recovery plan,” Meadows said.

“It has given the business a lot of transparency into what IT actually does. We're still doing an overhaul of the environment but this has filled a big gap and enabled me to move on to concentrating on other things.”

With David Braue