Customers describe "chaos" at E3 data centre move

Customers of E3 Networks have described "chaotic" scenes over the weekend as the company attempted to migrate customer equipment from its racks in Brisbane's Wharf Street datacentre to a recently constructed alternative at Milton.

The data centre switch, which E3 chief Jon Eaves had previously told iTnews was due to a lack of space at the Wharf Street facility, appears to be the result of a falling out between E3 and Wharf Street's new owners - NextDC.

Customers reported that parts of the new E3 data centre in Milton was unfinished upon their arrival, with side panels from cabinets used to cover a large hole in the floor.

Carriage services provider Sonet publicly pulled its racks out of E3 Networks this morning in protest.

Sonet chief Curtis Bayne said in a Webhostingtalk post that the data centre move caused connectivity issues that led to lengthy outages of his network.

"At 2pm yesterday afternoon, I decided that I couldn't endure the downtime any longer, especially considering that it was becoming increasingly obvious that nobody was going to help us," Bayne wrote.

"We decided to rip equipment out of the facility. We no longer even have a switch in the facility."

Bayne has since told iTnews that within two hours of making Sonet's exit public, E3 staff pulled the plug on Sonet's other routers which has been housed in an E3 rack in Pipe Network's nearby DC2 datacentre.

"His [Eaves'] response was to pull the entire network out," Bayne told iTnews.

Today iTnews contacted Eaves for comment, and he directed all comment to Peter Blunt, at one time E3's legal counsel but now a third-party contractor that had been engaged to assist with the move logistics. Blunt said he was unable to comment on the specific allegations. Eaves has again been contacted with an opportunity to comment.

Further reports made on Whirlpool described the new centre as "a scene of absolute chaos" yesterday.

"There was a good hundred Dell servers sitting on the ground, as well as racks, floor tiles and a lot of customers," one user wrote. "The place looked like a construction site at best."

Customers in contact with iTnews have also alleged:

• The data centre was running off a single microwave backhaul link, causing latency issues and redundancy concerns.
• That assigned move schedules for equipment in Wharf Street were not adhered to, resulting in equipment being switched off or decommissioned early or without customer technicians being present.

One technician – who spoke on condition of anonymity – told iTnews that customers helped each other reconnect power to racks in Milton after being unable to attract the attention of swamped E3 staff.

"We had to roll the rack out of position, get the [power] patches out of the floor, and then lift the rack up with gear in it because the power cords were too short," he said.

But Blunt contradicted these reports, telling iTnews that the move "went incredibly well", that E3 was "very happy" and that all customers were "fully live" with no power issues and few connectivity issues.

"Individual customers may be having trouble [with connectivity], but that's usually down to problems with the way their equipment was configured," Blunt said.

He confirmed that there was a large hole in the data centre floor - reported by several customers to have been covered using side panels from cabinets.

"A section of the data centre [floor] at the back, that was not accessed by anybody, was missing because five pallets of floor tiles went missing on the way to Brisbane and we've not been able to replace them yet," Blunt said.

"[But] this is not in a client area."

The falling out

Customers reported dissatisfaction with the amount of notice given before the move - with some reporting that they only found out about the migration a little over a week ago.

E3 had commissioned the Milton data centre just six weeks ago and Eaves had previously said it was never meant to be the company's primary centre.

E3 was forced to expand the Milton centre after negotiations broke down with the new owners of the Wharf Street building.

The Wharf Street building, which originally housed E3's Pegasus data centre, was purchased by the Bevan Slattery-backed NextDC in August last year as the site for NextDC's Brisbane data centre, due to open in March 2011.

Slattery claimed that NextDC gave existing tenants in Wharf Street – including E3 Networks – several months notice to vacate the building before it was gutted.

"It is not appropriate to comment on specific tenants, however all tenants were given notice to vacate almost six months ago due to major refurbishment of 20 Wharf St," Slattery told iTnews.

"In fact, NextDC staff had to move from the building in December for the same reason as evidenced by our recent ASX release notifying the change in our registered address from 20 Wharf St to 88 Eagle St Brisbane."

E3 spokesman Peter Blunt disputed that E3 had been given six months notice to vacate but confirmed that it was "not substantially less".  

Blunt maintained that E3 had an "agreement" with NextDC to stay on in the Wharf Street facility until a newly-refurbished floor at the site was available.

"We were in quite advanced negotiations with NextDC," Blunt told iTnews.

"[NextDC said they] would keep our floor online for several months longer on level six [of Wharf Street] until they completed the fitout of another level. We'd negotiated space [on another level] and the move wasn't due to take place for several months.

"At the eleventh hour we realised that negotiations were becoming very protracted."

Blunt declined to put a date on when negotiations broke down. But he did say it was after E3 set up the Milton data centre some six weeks ago.

Price appeared to play a major role in the alleged breakdown.

"They [NextDC] weren't being unreasonable but [the price] wasn't in the range our customers [would be] prepared to pay," Blunt said.

Do you know more? Email rcrozier (at) itnews (dot) com (dot) au.