Warning: Why your Internet might fail on May 5

Apr 30, 2010 12:16 PM
Tags: dnssec | internet | fail | y2k | melbourne it

Network operators urged to check routers, firewalls.

Network managers are being urged to run a series of checks on their routers and firewalls to ensure their users will still be able to connect to internet sites in the wake of a major change to the internet's domain name system next week.

On May 5, the world's top domain authorities (led by ICANN, the US Government and Verisign) will complete the first phase of the roll-out of DNSSEC (Domain Name System Security Extensions) across the 13 root servers that direct user requests to the relevant websites on the internet.

The DNSSEC upgrade adds a digital signature to the response from every DNS (Domain Name Server) request to give an internet user an extra level of assurance that the domain name is translated to the correct Internet location (such as a website, or email destination).

DNSSEC was developed in an attempt to thwart 'man in the middle' attacks, in which hackers intercept a request and respond with a message that fools the user system into going to a false location.

But the new protocol - much welcomed by the industry - could have an unfortunate side effect for unprepared network managers, according to Bruce Tonkin, chief strategy officer at Melbourne IT and a board director at ICANN.

A response to a standard DNS request tends to be in a single packet (UDP protocol) and tends to fall below 512 bytes in size.

In some older networking equipment, any larger request than this would be blocked by pre-configured factory settings, under the assumption that larger packets (and several of them) represent an anomaly of some kind.

As of May 5 at 17:00 UTC (which is actually pre-dawn on Thursday 6th on the East Coast of Australia), all DNSSEC signature-laden messages sent back to a user's DNS resolver will be four times the size - up to 2 KB.  And should packets of that size be rejected, the message would likely be sent in multiple packets via the TCP protocol.

(These signatures will be dummies at first to test the system, as of July 1, they will be the real deal.)

Tonkin fears that while DNSSEC has been on the agenda for some time, many IT and network managers have yet to test their older routers and firewalls to ensure they can handle the larger DNS responses.

"The bigger answer coming back from the DNS request might get blocked by some internet devices in the Corporate network," he said.

DNSSEC is in fact already rolled out across most of the world's 13 root server clusters, in an effort that began in December 2009.

But to date, Tonkin explained, it would only have resulted in a slight lag in the loading of a web page for those with outdated network equipment.

The beauty of DNS is that should a request made to one root server not receive a response, the DNS resolver on a user's machine simply makes the same request along the line of the 13 root servers until it gets a satisfactory response.

But on May 5, once all 13 root server clusters are live with the DNSSEC signatures, responses from all 13 root servers won't make it back inside the corporate LAN on some older systems.

Tonkin expects that the larger Internet Service Providers will have addressed the issue, so most home internet users will be unaffected.

"I'm not entirely sure all ISPs will be prepared, but I imagine the major ones are," he said. "ISPs tend to do DNS translation for you. But it is likely to have a big impact in the corporate environment, where you might run your own DNS server and infrastructure."

  • For more information  on the preparations of ISPs, telcos and network admins, check our update to this story.

In that sense Tonkin doesn't expect a "Y2K meltdown" of the internet May 5.

But he predicts a number of organisations will start experiencing internet access issues, and a number of network administrators will be left scratching their heads as to why.

To complicate the scenario further, network administrators and helpdesks "may not know what has gone wrong," he said.

The problem may take several days to surface and be inconsistent from one user's PC to the next.  A user at one machine that hasn't switched on his PC for two or three days will have no access to the internet. A user that left his machine on the night before will have some pages - and responses from DNS servers - cached on their machine, and will still have connectivity.

"It is usually much easier to address a problem when everything isn't working!" Tonkin said.

Tonkin recommended network managers run a series of simple online tests to ensure their network can handle the larger DNS responses:

- A reply-size test available at DNS-OARC:
https://www.dns-oarc.net/oarc/services/replysizetest

- Ripe Labs' 'Test your DNS Resolver'
http://labs.ripe.net/content/testing-your-resolver-dns-reply-size-issues


  • Email a Friend
  • Print Page
Warning: Why your Internet might fail on May 5
"I couldn't post on May 5 as my Internet failed. But good news is that it is back working today !"
 
 
Comments: 10
Thoughts on this article? Add a comment below.
Jeremy
Apr 30, 2010 5:08 PM
A good article with additional information:

http://etherealmind.com/dnssec-and-why-the-internet-probably-wont-break-today/
martyvis
Apr 30, 2010 5:21 PM
I just used the test at http://labs.ripe.net/content/testing-your-resolver-dns-reply-size-issues against the Telstra DNS , 139.130.4.4 , used as the resolver for their Mobile data users (amongst other things). And guess what - "Your resolver was only able to get packets SMALLER than 512 bytes."

So do we expect breakage on Telstra Mobile unless something changes?
BrettWinterford
Apr 30, 2010 5:44 PM
@ martyvis - just ran the same test. Can't believe it. I am asking Telstra what is up with that.
Daniel15
Apr 30, 2010 9:27 PM
I got this via TPG's server:

Announced buffer size:
4096 bytes
Measured buffer size:
3839 bytes
EDNS enabled:
yes
DNSSEC enabled:
yes

Your resolver announced a buffer size bigger than the largest packet that it can receive.
Note: There will always be a difference between the announced and measured buffer size because of the algorithm used. However this difference should not exceed 300 bytes.
Mordd
May 1, 2010 7:14 AM
im getting same as above using iiNet, i need someone more technical minded than me to explain it though, im only half understanding it.
stebie
May 1, 2010 1:45 PM
"If this tool reveals that your resolver is announcing a bigger buffer size than it can handle, first check to see whether the difference between the announced buffer size and measured buffer size is small. Because of the way the algorithm works, there will always be a small difference between announced and measured buffer sizes. You don't need to worry if the difference is small (up to 300 bytes). However, if your announced buffer size is the default of 4096 bytes, and the measured buffer size is much smaller (say 1400 bytes), then it is a cause for concern. You should reconfigure your resolver to announce a buffer size which is equal to the measured buffer size."

Seeing the difference is 257 bytes, it's under the 300-byte 'limit' so you should be fine.

P.S. my home router, and Optus DNS (tested 211.31.138.11, 211.29.132.12, 198.142.0.51) all returned the same results - 4096 / 3839 / yes / yes.
jez
May 1, 2010 9:12 PM
@martyvis: It's unlikely to be a problem. The Telstra DNS server has EDNS disabled, which is why it can't handle large responses, so will switch to using TCP instead of UDP. There's no 512-byte limit when using TCP.

The problem comes when the server thinks it can accept large responses, but it sits behind a firewall that blocks them. Not an issue here.
martyvis
May 3, 2010 7:58 AM
Just got a twitter response from @cricketondns:- "In this case, it probably doesn't matter. Google Public DNS isn't setting the DO bit, so they won't get DNSSEC RRs in responses." (I had asked him specfically about Google's public 8.8.8.8 DNS failing the large packet test. The DO bit is "DNSSEC OK", which would be set as an option by the client. So unless you are asking for DNSSEC this issue shouldn't be a problem. If you are going to be asking for DNSSEC signed records, then more than likely you will either have your own properly setup DNS resolver or you will fallback due to fact that DNSSEC is not ubiquitous enough to start mandating it.

(Cricket Liu is Vice President of Architecture at Infoblox, coauthor of O'Reilly Media's DNS and BIND. So pretty much one of the DNS gurus)
jez
May 3, 2010 2:27 PM
@martyvis: Yes, Cricket is right, Google DNS resolvers should be OK, in much the same way that Telstra's DNS server should be OK. According to the test results, neither support EDNS, so case closed.

But the "fallback" case you're hoping for is not universal. The most common DNS caching resolver software "BIND" sets the "DO" bit on all of its queries, regardless of whether the original requester has it set. In fact, the RFC 3225 requires this behaviour on resolver software that supports DNSSEC even if it isn't configured for DNSSEC. So while your computer might not set the "DO" bit, the resolver you're using might. In itself, this isn't a problem, but if a DNSSEC-aware resolver is behind a firewall that can't handle the larger DNSSEC packets, you will get problems.
Digger11
May 6, 2010 9:03 AM
I couldn't post on May 5 as my Internet failed. But good news is that it is back working today !
Comment:
Want to participate in the discussion?
Or log in now to comment
 
 
 
Top Stories
Labor wins broadband election
Independents back Gillard minority Government.
 
Analysis: Net gambling next on ISP filter hitlist?
Stakeholders brand online gambling laws ineffective.
 
Industry ‘relieved’ over NBN
Gillard welcomed but changes needed.
 

Latest VideosSee all videos »

Latest Comments
"The headline to this article is misleading! Labor did not win this election. They bought it ..."
by epimetheus Sep 9, 2010 4:42 PM
 
"Is it not a pity that most people cannot see beyond the hype? This NBN [no broadband - never] ..."
by epimetheus Sep 9, 2010 4:29 PM
 
"You can already enable it from within Australia -- needs a Firefox extension called Google ..."
by threnchpot Sep 9, 2010 4:08 PM
 
"@Francis: "Conboy is not the brightest minister for communications that we have had to suffer." ..."
by anonymous Sep 9, 2010 3:39 PM
 
"rsgooch wrote: I will throw my two cents in here even though it may well get thrown back at ..."
by deteego Sep 9, 2010 3:28 PM
Polls
Do you think the Labor minority Government will survive a full term?

   |   View results
Yes
  41%
 
No
  59%
TOTAL VOTES: 73

Vote