Warning: Why your Internet might fail on May 5

 

Network operators urged to check routers, firewalls.

Network managers are being urged to run a series of checks on their routers and firewalls to ensure their users will still be able to connect to internet sites in the wake of a major change to the internet's domain name system next week.

On May 5, the world's top domain authorities (led by ICANN, the US Government and Verisign) will complete the first phase of the roll-out of DNSSEC (Domain Name System Security Extensions) across the 13 root servers that direct user requests to the relevant websites on the internet.

The DNSSEC upgrade adds a digital signature to the response from every DNS (Domain Name Server) request to give an internet user an extra level of assurance that the domain name is translated to the correct Internet location (such as a website, or email destination).

DNSSEC was developed in an attempt to thwart 'man in the middle' attacks, in which hackers intercept a request and respond with a message that fools the user system into going to a false location.

But the new protocol - much welcomed by the industry - could have an unfortunate side effect for unprepared network managers, according to Bruce Tonkin, chief strategy officer at Melbourne IT and a board director at ICANN.

A response to a standard DNS request tends to be in a single packet (UDP protocol) and tends to fall below 512 bytes in size.

In some older networking equipment, any larger request than this would be blocked by pre-configured factory settings, under the assumption that larger packets (and several of them) represent an anomaly of some kind.

As of May 5 at 17:00 UTC (which is actually pre-dawn on Thursday 6th on the East Coast of Australia), all DNSSEC signature-laden messages sent back to a user's DNS resolver will be four times the size - up to 2 KB.  And should packets of that size be rejected, the message would likely be sent in multiple packets via the TCP protocol.

(These signatures will be dummies at first to test the system, as of July 1, they will be the real deal.)

Tonkin fears that while DNSSEC has been on the agenda for some time, many IT and network managers have yet to test their older routers and firewalls to ensure they can handle the larger DNS responses.

"The bigger answer coming back from the DNS request might get blocked by some internet devices in the Corporate network," he said.

DNSSEC is in fact already rolled out across most of the world's 13 root server clusters, in an effort that began in December 2009.

But to date, Tonkin explained, it would only have resulted in a slight lag in the loading of a web page for those with outdated network equipment.

The beauty of DNS is that should a request made to one root server not receive a response, the DNS resolver on a user's machine simply makes the same request along the line of the 13 root servers until it gets a satisfactory response.

But on May 5, once all 13 root server clusters are live with the DNSSEC signatures, responses from all 13 root servers won't make it back inside the corporate LAN on some older systems.

Tonkin expects that the larger Internet Service Providers will have addressed the issue, so most home internet users will be unaffected.

"I'm not entirely sure all ISPs will be prepared, but I imagine the major ones are," he said. "ISPs tend to do DNS translation for you. But it is likely to have a big impact in the corporate environment, where you might run your own DNS server and infrastructure."

  • For more information  on the preparations of ISPs, telcos and network admins, check our update to this story.

In that sense Tonkin doesn't expect a "Y2K meltdown" of the internet May 5.

But he predicts a number of organisations will start experiencing internet access issues, and a number of network administrators will be left scratching their heads as to why.

To complicate the scenario further, network administrators and helpdesks "may not know what has gone wrong," he said.

The problem may take several days to surface and be inconsistent from one user's PC to the next.  A user at one machine that hasn't switched on his PC for two or three days will have no access to the internet. A user that left his machine on the night before will have some pages - and responses from DNS servers - cached on their machine, and will still have connectivity.

"It is usually much easier to address a problem when everything isn't working!" Tonkin said.

Tonkin recommended network managers run a series of simple online tests to ensure their network can handle the larger DNS responses:

- A reply-size test available at DNS-OARC:
https://www.dns-oarc.net/oarc/services/replysizetest

- Ripe Labs' 'Test your DNS Resolver'
http://labs.ripe.net/content/testing-your-resolver-dns-reply-size-issues


Warning: Why your Internet might fail on May 5
"I couldn't post on May 5 as my Internet failed. But good news is that it is back working today !"
By Digger11
 
 
 
Comments: 10
Jeremy
Apr 30, 2010 5:08 PM
A good article with additional information:

http://etherealmind.com/dnssec-and-why-the-internet-probably-wont-break-today/
martyvis
Apr 30, 2010 5:21 PM
I just used the test at http://labs.ripe.net/content/testing-your-resolver-dns-reply-size-issues against the Telstra DNS , 139.130.4.4 , used as the resolver for their Mobile data users (amongst other things). And guess what - "Your resolver was only able to get packets SMALLER than 512 bytes."

So do we expect breakage on Telstra Mobile unless something changes?
BrettWinterford
Apr 30, 2010 5:44 PM
@ martyvis - just ran the same test. Can't believe it. I am asking Telstra what is up with that.
Daniel15
Apr 30, 2010 9:27 PM
I got this via TPG's server:

Announced buffer size:
4096 bytes
Measured buffer size:
3839 bytes
EDNS enabled:
yes
DNSSEC enabled:
yes

Your resolver announced a buffer size bigger than the largest packet that it can receive.
Note: There will always be a difference between the announced and measured buffer size because of the algorithm used. However this difference should not exceed 300 bytes.
Mordd
May 1, 2010 7:14 AM
im getting same as above using iiNet, i need someone more technical minded than me to explain it though, im only half understanding it.
stebie
May 1, 2010 1:45 PM
"If this tool reveals that your resolver is announcing a bigger buffer size than it can handle, first check to see whether the difference between the announced buffer size and measured buffer size is small. Because of the way the algorithm works, there will always be a small difference between announced and measured buffer sizes. You don't need to worry if the difference is small (up to 300 bytes). However, if your announced buffer size is the default of 4096 bytes, and the measured buffer size is much smaller (say 1400 bytes), then it is a cause for concern. You should reconfigure your resolver to announce a buffer size which is equal to the measured buffer size."

Seeing the difference is 257 bytes, it's under the 300-byte 'limit' so you should be fine.

P.S. my home router, and Optus DNS (tested 211.31.138.11, 211.29.132.12, 198.142.0.51) all returned the same results - 4096 / 3839 / yes / yes.
jez
May 1, 2010 9:12 PM
@martyvis: It's unlikely to be a problem. The Telstra DNS server has EDNS disabled, which is why it can't handle large responses, so will switch to using TCP instead of UDP. There's no 512-byte limit when using TCP.

The problem comes when the server thinks it can accept large responses, but it sits behind a firewall that blocks them. Not an issue here.
martyvis
May 3, 2010 7:58 AM
Just got a twitter response from @cricketondns:- "In this case, it probably doesn't matter. Google Public DNS isn't setting the DO bit, so they won't get DNSSEC RRs in responses." (I had asked him specfically about Google's public 8.8.8.8 DNS failing the large packet test. The DO bit is "DNSSEC OK", which would be set as an option by the client. So unless you are asking for DNSSEC this issue shouldn't be a problem. If you are going to be asking for DNSSEC signed records, then more than likely you will either have your own properly setup DNS resolver or you will fallback due to fact that DNSSEC is not ubiquitous enough to start mandating it.

(Cricket Liu is Vice President of Architecture at Infoblox, coauthor of O'Reilly Media's DNS and BIND. So pretty much one of the DNS gurus)
jez
May 3, 2010 2:27 PM
@martyvis: Yes, Cricket is right, Google DNS resolvers should be OK, in much the same way that Telstra's DNS server should be OK. According to the test results, neither support EDNS, so case closed.

But the "fallback" case you're hoping for is not universal. The most common DNS caching resolver software "BIND" sets the "DO" bit on all of its queries, regardless of whether the original requester has it set. In fact, the RFC 3225 requires this behaviour on resolver software that supports DNSSEC even if it isn't configured for DNSSEC. So while your computer might not set the "DO" bit, the resolver you're using might. In itself, this isn't a problem, but if a DNSSEC-aware resolver is behind a firewall that can't handle the larger DNSSEC packets, you will get problems.
Digger11
May 6, 2010 9:03 AM
I couldn't post on May 5 as my Internet failed. But good news is that it is back working today !
Comments have been disabled for this article.
 
 
Top Stories
Review: Microsoft Surface Pro
A year is a long time in the computer hardware business.
 
NBN Co could miss revised June fibre targets
Analysis: Cutting it fine in the race to the line.
 
Review: Sydney's Opal smartcard
It's no Oyster card.
 
 
Sign up to receive iTnews email bulletins
   FOLLOW US...

Latest VideosSee all videos »

iTnews Academy: Microsoft Windows Server 2012 - Hyper-V
iTnews Academy: Microsoft Windows Server 2012 - Hyper-V
Interview: Australia's 'cloud-last' policy is dangerous.
Interview: Australia's 'cloud-last' policy is dangerous.
Interview: Vivek Kundra on Australia's 'cloud last' policy
Bankwest builds continuous delivery capability
Bankwest builds continuous delivery capability
To automatically deploy test/dev sandboxes by mid-year.
Veterans' Affairs sets sights on modernisation
Veterans' Affairs sets sights on modernisation
Data safe with Human Services, CIO says.
Citi Australia drops platform customisations
Citi Australia drops platform customisations
Technology chief shifts focus from building to leveraging systems.
VicRoads restructures IT team
VicRoads restructures IT team
Department moves to align with industry benchmarks.
Zurich Australia extends IT team offshore
Zurich Australia extends IT team offshore
Malaysian staff served from Australian data centres.
Leigh Berrell - Utilities CIO of the Year
Leigh Berrell - Utilities CIO of the Year
Yarra Valley Water CIO Leigh Berrell accepts his Benchmark Award for Utilities CIO of the Year.
Wayne McMahon - Retail CIO of the Year
Wayne McMahon - Retail CIO of the Year
Domino's Pizza CIO Wayne McMahon accepts his Benchmark Award for Retail CIO of the Year.
Inside Perpetual's ongoing IT transformation
Inside Perpetual's ongoing IT transformation
CIO Jenny Levy discusses how outsourcing will help the firm "simplify, refocus and grow".
Managing Complexity - Defence's Daniel McCabe
Managing Complexity - Defence's Daniel McCabe
Daniel McCabe, Assistant Secretary of Australia's Department of Defence, provides the audience at the iTnews Data Centre Strategy Summit with a deep dive into the organisation's data centre consolidation program.
How Facebook designed the data centre from scratch - Marco Magarelli
How Facebook designed the data centre from scratch - Marco Magarelli
The full keynote by Facebook data centre architect Marco Magarelli at the Australian Data Centre Strategy Summit. Magarelli details the design considerations behind the social network's Prineville, Oregon; North Carolina and Luleå, Sweden data centres.
Modernising Legacy Data Centres - Telstra's Jon Curry
Modernising Legacy Data Centres - Telstra's Jon Curry
Telstra general manager of managed data centres Jon Curry guides the audience at the iTnews Australian Data Centre Summit through the build of the telco's Clayton, Victoria data centre.
NSW Government launches NABERS data centre rating tools
NSW Government launches NABERS data centre rating tools
Matthew Clark from the NSW Department of Environment guides facilties managers through the details of the new NABERS data centre energy rating tool at the Australian Data Centre Strategy Summit.
NABERS launch panel: Australian Data Centre Strategy Summit
NABERS launch panel: Australian Data Centre Strategy Summit
Matthew Clark (NSW Dept of Environment), Greg Boorer (Canberra Data Centres), Glenn Allan (National Australia Bank), Mike Andrea (Strategic Directions) and Bob Sharon (Green Global Consulting) discuss the impact of the NABERS data centre rating.
Judges notes: Fortescue Metals [The Benchmark Awards]
Judges notes: Fortescue Metals [The Benchmark Awards]
iTnews' panel of judges discuss Fortescue Metals 'New World of Work" project, one of three shortlisted finalists for the Industrials category of the CIO Benchmark Awards.
Judges notes: Retail [The Benchmark Awards]
Judges notes: Retail [The Benchmark Awards]
iTnews' panel of judges discuss the shortlisted finalists for the Retail category of the CIO Benchmark Awards.
Judges notes: Pacific Aluminium [The Benchmark Awards]
Judges notes: Pacific Aluminium [The Benchmark Awards]
iTnews' panel of judges discuss Pacific Aluminium's lightning fast service desk refresh, one of three shortlisted finalists for the Industrials category of the CIO Benchmark Awards.
Judges notes: Domino's Pizza [The Benchmark Awards]
Judges notes: Domino's Pizza [The Benchmark Awards]
iTnews' panel of judges discuss Domino's Pizza's shift to hosted services, one of three shortlisted finalists for the Retail category of the CIO Benchmark Awards.
Judges notes: McDonald's Australia [The Benchmark Awards]
Judges notes: McDonald's Australia [The Benchmark Awards]
iTnews' panel of judges discuss McDonald's Australia's new self-service portal for employees, one of three shortlisted finalists for the Retail category of the CIO Benchmark Awards.
Latest Comments
Polls
Will you quit any cloud services in light of PRISM?

   |   View results
Yes
  61%
 
No
  39%
TOTAL VOTES: 72

Vote