Apache publishes report about security breach

Apache Software Foundation has published a concise incident report following it being compromised by an SSH key last week.

The report by the Apache infrastructure team on its company blog claimed that it had ‘analysed the events that led to the breach, and continued to work on improving the security of our systems'.

A team statement said: “At no time were any Apache Software Foundation code repositories, downloads, or users put at risk by this intrusion. However, we believe that providing a detailed account of what happened will make the internet a better place, by allowing others to learn from our mistakes.”

In a section detailed ‘What didn't work?', it claimed that the use of SSH keys facilitated this attack and in hindsight, its implementation left a lot to be desired as it did not restrict SSH keys appropriately, and Apache was unaware of their misuse.

It also claimed that the rsync setup, which uses people.apache.org to manage the deployment of the websites, enabled the attackers to get their files onto the US mirror undetected. It was also unnecessarily vulnerable to an attack of this nature as there is the ability to run CGI scripts in any virtual host when most of its websites do not need this functionality.

Apache claimed that the lack of logs from the ApacheCon host prevented it from conclusively determining the full course of action taken by the attacker, as all but one log file were deleted by the attacker and logs were not kept off the machine.

However, it did claim that the use of ZFS snapshots enabled it to restore the EU production web server to a known-good state. Redundant services in two locations allowed it to run services from an alternate location while continuing to work on the affected servers and services.

The Apache infrastructure team claimed that it has made several changes ‘to help further secure our infrastructure from such issues in the future'. These include requiring all users with elevated privileges to use OPIE for sudo on certain machines. Apache already requires this in some places, but it will expand its use as necessary.

It will also recreate and use new SSH keys, one per host, for backups, while the VM that hosted the old apachecon.com site remains powered down, awaiting further detailed analysis.

Graham Cluley, senior technology consultant at Sophos, said: “What really impresses me, however, is how well Apache handled the potentially highly embarrassing incident - taking swift action and keeping their users informed via blog updates.

“Apache explains in their blog post that they ‘believe that providing a detailed account of what happened will make the internet a better place, by allowing others to learn from our mistakes'. So bravo to Apache for responding to the problem rapidly and with openness, proving it is possible to turn a potentially bad story into a positive experience.”

See original article on scmagazineus.com