The mass retention of data is illegal, the European Union's highest court said today, dealing a blow to Britain's newly passed surveillance law and signalling that security concerns do not justify excessive privacy infringements.
The Court of Justice of the European Union (ECJ) said its ruling was based on the view that holding traffic and location data en masse allowed "very precise conclusions to be drawn concerning the private lives of the persons whose data has been retained".
Such interference with people's privacy could only be justified by the objective of fighting serious crime and access to data should be subject to prior review by a court or independent body except in urgent cases, it said.
The ruling is likely to upset governments seeking to deal with the threat of attacks such as those in Paris and Brussels and, on Monday, in Berlin.
Those attacks have reinforced calls from governments for security agencies to be given greater powers to protect citizens, while privacy advocates - who welcomed the ruling - say mass retention of data is ineffective in the fight against such crimes.
The perpetual debate over privacy versus security took on an extra dimension after Edward Snowden leaked details of mass spying by US and British agents in 2013.
The ECJ said governments could demand targeted data retention subject to strict safeguards such as limiting it to a particular geographic location, but the data must be stored within the EU given the risk of unlawful access.
The court was responding to challenges against data retention laws in Britain and Sweden on the grounds that they were no longer valid after the ECJ struck down an EU-wide data retention law in 2014.
A spokesman for Britain's interior ministry said it was disappointed with the judgment and would be considering its potential implications in the case, launched before Britain voted in June to quit the European Union.
"Given the importance of communications data to preventing and detecting crime, we will ensure plans are in place so that the police and other public authorities can continue to acquire such data in a way that is consistent with EU law and our obligation to protect the public,” he said.
When it leaves the bloc, Britain will not be subject to ECJ rulings, but with a two-year exit negotiation process due to begin by the end of March - and a potential transitional phase afterwards - it could be under its auspices for some time.
A number of British politicians - including Brexit minister David Davis - filed the legal challenge against the Data Retention and Investigatory Powers Act (DRIPA) 2014, part of which was suspended by a British court.
Britain subsequently passed the Investigatory Powers Act (IPA) - the so-called Snooper's Charter - which, while not directly affected by the ruling, could nevertheless be found to be incompatible with the conditions laid down by the court, lawyers said.
A number of provisions in the IPA, such as the purposes for which data may be retained which go beyond fighting serious crime, are unlikely to pass the court's muster, laywers said.
The requirement for targeted retention of communications data could be "a significant headache", said Linklaters partner Richard Cumbley, since security services often only know who is a suspect after the event.