XKEYSCORE global spy system detailed in new Snowden leaks

Top secret documents leaked by former US National Security Agency contractor Edward Snowden have cast further light on the secretive XKEYSCORE analysis system used to process information captured from taps on internet fibre-optic backbones and other communications channels.

Published by The Intercept, the documents date back to 2013 and detail a global mass surveillance system that indiscriminately takes in every bit of data it can for NSA analysts to process.

XKEYSCORE is a distributed system for querying tens of billions of records stored in its databases. It is used by spy agencies within the Five Eyes alliance - Australia, the US, NZ, Canada and Britain.

The system had around 150 field sites globally in 2008, some located in Australia and New Zealand where local and visiting US intelligence analysts also have access to the captured data.

Content is stored in XKEYSCORE for three to five days, but metadata such as call records and browsing history is kept much longer, for 30 to 45 days, the documents showed.

The extent of the full collection of internet users, businesses and other organisations' data with XKEYSCORE is substantial, and includes, among other things:

• Emails
• Chats
• Website traffic
• Pictures
• Documents
• Voice calls, including internet voice calls
• Webcam photos
• Search engine traffic
• Advertising analytics traffic
• Social media traffic
• Botnet traffic
• Logged keystrokes
• Computer network exploitation (CNE) targeting
• Intercepted username and password pairs
• File uploads to online services
• Skype sessions 

The NSA defended its use of systems such as XKEYSCORE to The Intercept, arguing they were ;necessary to protect the US, its populations and its allies against a wide range of serious threats.

XKEYSCORE is also used to steal login credentials, with system administrators in particular being targeted.

Security researcher Jonathan Brossard told The Intercept XKEYSCORE was very easy to use, with staff being trained in less than a day to use what he described as an automated hacking system.

The efficacy of XKEYSCORE to capture credentials, security product telemetry and other data appears to have been reduced with the increased use of encryption in later years, the documents noted.

Confirming Snowden's earlier assertions, there appears to be few if any means beyond warning messages asking analysts to comply with US and UK human rights and privacy regulations to limit what can be spied upon with XKEYSCORE.

NSA's ability to conduct mass surveillance appeared to be in jeopardy after a US court ruled that the agency was never legally authorised to bulk collect Americans' phone records, alongside the lapse of a sunset clause in the Patriot Act that enabled the spying.

However, the court ruling was overturned by a judge of the Foreign Intelligence Surveillance Court (FISC) who ruled that the new Freedom Act signed into law by US president Barack Obama meant the NSA could continue with its bulk metadata collection for a six-month transition period.