How to make Route53 up to 10x faster in one single step

Many startups use Amazon Route53 today — it’s affordable and got a great set of features. One side that many developers avoid discussing though is Route53’s performance, and believe me (or believe this site) it doesn’t perform well!

Luckily, we found a way to make it way, way faster. If most of your users reside in the same geographical area (eg. Europe) it will give you a 10x boost!

Preamble

We live in the world of fast pace and low latency. Things in Internet are getting faster and faster — practically every major website uses a global content delivery network to display your image or video as quickly as possible. Most DNS providers use AnyCast technology meaning that each nameserver has, in fact, multiple instances, and user will be routed to the closest one. Well, that’s the idea in general but it seems to be not the case with Amazon’s Route53! :)

Route53: the problem

If you ever created a zone in Route53, you are pretty familiar with the following tab:

This is how your zone typically looks in Route53

Amazon assigns four weird looking nameservers to every zone it creates. These nameservers are supposed to be AnyCast so if you are asking “what’s the IP address of google.com” from Australia — you normally expect to receive a reply from within Australia itself or from somewhere nearby. Let’s see if that’s true?

Let’s do some traceroutes, and let’s do them from Amazon EC2 instance in Sydney to avoid “but it’s probably a peering problem” arguments:

traceroute to ns-75.awsdns-09.com (205.251.192.75), 30 hops max, 60 byte packets
6 vl2928.sw-1–1.as45671.net.au (221.121.137.68) 1.101 ms 0.470 ms 3.722 ms
7 ve1021.br-1–2.as45671.net.au (203.26.198.60) 3.802 ms 3.352 ms 3.523 ms
8 203.26.197.9 (203.26.197.9) 3.381 ms 3.378 ms 3.371 ms
9 xe-0–0–0.gw101.sy1.ap.equinix.com (27.111.240.142) 3.350 ms 3.351 ms 3.348 ms
10 202.68.67.41 (202.68.67.41) 3.717 ms 3.874 ms 3.973 ms
11 xe-0–0–3.r20.sydnau02.au.bb.gin.ntt.net (202.68.64.214) 140.201 ms 140.205 ms 140.185 ms
12 ae-8.r22.lsanca07.us.bb.gin.ntt.net (129.250.3.164) 146.452 ms 140.086 ms 140.085 ms
13 ae-7.r22.dllstx09.us.bb.gin.ntt.net (129.250.7.68) 175.559 ms 182.553 ms 175.542 ms
14 ae-3.r23.dllstx09.us.bb.gin.ntt.net (129.250.5.21) 169.283 ms 171.902 ms 177.460 ms
15 ae-5.r21.atlnga05.us.bb.gin.ntt.net (129.250.4.117) 207.058 ms 198.091 ms

As we can see, Atlanta was chosen in this case. Distance between Sydney and Atlanta is roughly 15,000 km. If you are not good with networking and traceroutes, you could do this:

$ time dig www.google.com @ns-75.awsdns-09.com

And you will get nearly the same latency — 220ms. So, doesn’t matter how cool is your CDN or how close your API is, your users will spend extra 220ms resolving your domain name. It’s a common trend today to set DNS TTL pretty low, so you can expect users to do this query again and again!

The second nameserver is in Atlanta as well, while third and fourth are in Australia (hooray!)

You can use this one-liner in the terminal to test all 4 at once:

$ while read -r host; do ping -c 1 $host; done <<< $’ns-75.awsdns-09.com\nns-588.awsdns-09.net\nns-1689.awsdns-19.co.uk\nns-1105.awsdns-10.org’ |egrep “rtt|statistics”
 — — ns-75.awsdns-09.com ping statistics — -
rtt min/avg/max/mdev = 202.203/202.203/202.203/0.000 ms
— — ns-588.awsdns-09.net ping statistics — -
rtt min/avg/max/mdev = 219.252/219.252/219.252/0.000 ms
— — ns-1689.awsdns-19.co.uk ping statistics — -
rtt min/avg/max/mdev = 13.949/13.949/13.949/0.000 ms
— — ns-1105.awsdns-10.org ping statistics — -
rtt min/avg/max/mdev = 0.902/0.902/0.902/0.000 ms

Let’s test from another few points in Australia (different providers) and compare.

Vultr Sydney:

 — — ns-75.awsdns-09.com ping statistics — -
rtt min/avg/max/mdev = 209.292/209.292/209.292/0.000 ms
— — ns-588.awsdns-09.net ping statistics — -
rtt min/avg/max/mdev = 227.579/227.579/227.579/0.000 ms
— — ns-1689.awsdns-19.co.uk ping statistics — -
rtt min/avg/max/mdev = 12.426/12.426/12.426/0.000 ms
— — ns-1105.awsdns-10.org ping statistics — -
rtt min/avg/max/mdev = 0.845/0.845/0.845/0.000 ms

My office connection in Melbourne:


— — ns-75.awsdns-09.com ping statistics — -
round-trip min/avg/max/stddev = 219.359/219.359/219.359/0.000 ms
— — ns-588.awsdns-09.net ping statistics — -
round-trip min/avg/max/stddev = 203.459/203.459/203.459/0.000 ms
— — ns-1689.awsdns-19.co.uk ping statistics — -
round-trip min/avg/max/stddev = 10.071/10.071/10.071/0.000 ms
— — ns-1105.awsdns-10.org ping statistics — -
round-trip min/avg/max/stddev = 14.901/14.901/14.901/0.000 ms

Telstra Melbourne:

ns-75.awsdns-09.com 193ms
ns-588.awsdns-09.net 157ms
ns-1689.awsdns-19.co.uk 1ms
ns-1105.awsdns-10.org 11ms

AARNet Sydney:

ns-75.awsdns-09.com 184ms
ns-588.awsdns-09.net 316ms (routed to Marseille!)
ns-1689.awsdns-19.co.uk 11ms
ns-1105.awsdns-10.org 1ms

InterNode Melbourne:

ns-75.awsdns-09.com 220ms
ns-588.awsdns-09.net 8ms
ns-1689.awsdns-19.co.uk 1ms
ns-1105.awsdns-10.org 12ms

AAPT Sydney:

ns-75.awsdns-09.com 216ms
ns-588.awsdns-09.net 1ms
ns-1689.awsdns-19.co.uk 12ms
ns-1105.awsdns-10.org 1ms

Optus Sydney:

ns-75.awsdns-09.com 250ms
ns-588.awsdns-09.net 196ms
ns-1689.awsdns-19.co.uk 12ms
ns-1105.awsdns-10.org 1ms

We can see a pretty clear pattern now — somehow first two nameservers are always far-far away, while the latter two are somewhere nearby.

Therefore, removing .com and .net nameservers from both your domain record and Route53 zone will give you a very tangible performance boost — from ~100+ms average latency down to just ~10ms.

Conclusion

As shown above, Route53 always assigns two ‘sucker’ nameservers and two performant (network-wise) nameservers. By getting rid of two of them (and you don’t actually need four anyway — Cloudflare, world’s best DNS provider, uses 2) you can increase your DNS performance dramatically.

Asia and Australia should use .org and .co.uk nameservers. Europe should use .com and .net nameservers. All nameservers work reasonably ok in the US (would be surprised if they didn’t). Therefore, you can’t use this hack to boost performance in Asia and Europe at the same time.

.org is the best performant nameserver on average, but I don’t suppose you are going to leave just one nameserver in the zone? :)


About us

PoweredLocal is a Melbourne-based wi-fi innovation startup. We host PHP, Ruby and Node.js microservices behind our newly baked API gateway.

About author

Written by Denis Mysenko, Chief Technical Officer at PoweredLocal

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.