WebCentral email still down at 49 hour mark

An exhausted Melbourne IT spokesperson told iTnews at 7:00am (Thursday April 30) that the email system was "back online" and "all OK" after WebCentral technicians worked through a second night to address the problem.

As was the case Wednesday morning, around half of the queued emails WebCentral clients had been waiting on trickled through between midnight and the early hours of this morning.

But by 8:00am, customers of the Melbourne IT-owned web hosting company reported the system down again.

"We can send outgoing email, but when we try and receive incoming email we are just getting error messages again," said Theo Poulos, a WebCentral customer and employee at Sydney-based manufacturing software developer Pytha.

The error message reads 'Outlook is unable to connect to your incoming POP3 server. Please contact your ISP.'

Poulos says his company has learned through the experience just how much it relies on email to support its customers.

"I've got three [support] guys here doing jack s**t because the email is down," he said. "And I have clients ringing and abusing us because their customers are losing thousands of dollars on the production line. I'll end up cancelling my subscription to WebCentral."

Many customers have expressed disappointment over the lack of communication during the crisis.

"Outraged at lack of explanation or contact from Webcentral," said one user, Greg.

"I will change tomorrow - just really upset with the fact that they don't have the decency to communicate with their customers," said another customer, Gabi.

"I am really angry about WebCentral's apparent lack of concern for its customers. No communication on what is going on. Poor form," said another, Donna.

The problem according to Melbourne IT is that WebCentral does not have the capacity to email its customers (incoming POP e-mail down), nor enough staff on hand or systems in place to call them.

"The volume of calls is so high that the phone system is not able to handle them," the Melbourne IT spokesperson told iTnews.

At present the company is relying on regular technical updates on its web site to communicate its status - another of which is due shortly.

Messages posted thus far seem to indicate a major data corruption or storage error.

An update posted at 5pm on Wednesday, 29 April 2009, claimed that ongoing issues impact only 50 per cent of customers "with five of the ten message stores still impacted."

The company said four message stores were 'operational and available', one had 'completed recovery mode', and four had 'completed initial data integrity check' and is 'in recovery mode'. A final message store is being restored from back-up after it failed a data integrity check, the company said.

Several posters on forums at iTnews speculated a SAN hardware failure as the cause but this has not been confirmed by Melbourne IT.

ITnews will continue to seek answers from Melbourne IT.