Microsoft Azure customers around the world were unable to connect to a range of services on the cloud computing platform yesterday due to a glitch in a performance update to its cloud storage service.
Azure storage, virtual machines, websites, Active Directory and the management portal were among those affected by the partial service interrruption.
Additionally, Microsoft's own online businesses running on Azure, such as Office 365, Visual Studio Online, OneDrive for Business, Windows Store and Xbox Live, also went down.
Only the eastern and southern Australian regions along with Brazil South were unaffected, and Microsoft has since resolved the problem. It confirmed the outages but did not provide detail on the cause.
Microsoft's Application Insights Services were also impacted by Azure Storage Services yesterday, Microsoft said on the Visual Studio Online MSDN blog.
The Application Insight Services are now running normally, but there is a four-hour data gap between 0100 to 0500 UTC, Microsoft said.
Azure customers in the Northern and Western Europe region reported problems with virtual machines getting stuck in a continual start state, while others report connectivity issues.
A Norwegian Azure customer this morning said he continued to experience difficulties with virtual machines after 15 hours.
Dear @Azure, I still have 1 VM stuck in "starting" after 15 hrs. This is beyond embarrassing. It's incompetent #AzureDown
— Hans Olav Stjernholm (@HansOlavS) November 19, 2014
The service disruption is the second worldwide outage for Azure this year, following issues in August.
Update 21/11: Microsoft today revealed the outage was caused by a bug introduced during a configuration change in the Azure Storage Front End.
Despite having "extensively" tested the update for several weeks before its application, an issue arose after the update was applied, corporate vice president of the Microsoft Azure team Jason Zander wrote today in a blog post.
"During the rollout we discovered an issue that resulted in storage blob front ends going into an infinite loop, which had gone undetected during [testing the update]," he said.
"The net result was an inability for the front ends to take on further traffic, which in turn caused other services built on top to experience issues."
Microsoft started rolling back the update once the bug was discovered, but to do so it needed to restart the Azure Storage front ends, which downed the service for a number of users, Zander wrote.
