Anycast Networking: One IP, Many Servers

When you query a DNS server at 8.8.8.8, your request is answered by the Google server physically closest to you — not a single machine in California. This is possible because of anycast, a routing technique that lets the same IP address exist simultaneously on thousands of servers around the world. Anycast is one of the most important and least-understood technologies in internet infrastructure.

The Four Routing Methods

To understand anycast, it helps to contrast it with the other routing methods:

How Anycast Works

Anycast leverages the way the Border Gateway Protocol (BGP) naturally selects routes. BGP is the routing protocol that exchanges reachability information between autonomous systems (the large networks that make up the internet).

Here is the core idea:

1. An organization deploys identical servers in multiple locations (e.g., New York, London, Singapore, São Paulo).
2. Each server is configured with the same IP address — for example, 192.0.2.1.
3. Each location's router announces this IP address via BGP to the surrounding internet.
4. BGP routers across the internet receive multiple paths to 192.0.2.1 — one from New York, one from London, one from Singapore.
5. Each BGP router selects the best path according to BGP's route selection rules, which include AS path length, local preference, and geographic proximity.
6. Traffic from a user in Tokyo follows the path to Singapore. Traffic from a user in Berlin follows the path to London. Neither user knows or cares that the same IP is served from multiple locations.

BGP Route Selection in Anycast

BGP does not explicitly know about physical distance. Instead, it selects routes based on policy attributes:

• AS path length — Fewer hops between autonomous systems is preferred
• MED (Multi-Exit Discriminator) — Influences entry point when multiple paths exist into the same AS
• Local preference — An ISP can prefer one path over another for policy reasons
• IGP metric — The internal routing cost within the ISP's network to reach the BGP next-hop

In practice, these attributes naturally cluster traffic to the geographically nearest server — because fewer AS hops typically correlates with physical proximity.

What Happens When a Server Goes Down

This is where anycast delivers a major operational advantage. If the Singapore server fails:

1. The Singapore router stops announcing the anycast prefix.
2. Within 30-90 seconds, BGP reconverges and routers that were pointing to Singapore select the next-best path — perhaps to Sydney or Tokyo.
3. Traffic automatically shifts to the surviving nodes.
4. No DNS changes, no failover scripts, no manual intervention required.

This automatic failover happens at the routing layer — completely transparent to users and applications.

Use Case 1: DNS Root Servers

The internet's DNS root servers are the best-known example of anycast. There are 13 root server addresses (labeled A through M, such as a.root-servers.net through m.root-servers.net). To the outside world, each is a single IP address.

In reality, each root server address is served by dozens or hundreds of physical machines distributed globally. As of 2024:

The F root server alone has over 275 anycast instances. Globally, the 13 root addresses are served from more than 1,600 physical locations. Using dig or any DNS tool, all queries to 198.41.0.4 (A root) are answered, but the actual server responding depends on where you are.

# Query a root server directly
dig . NS @198.41.0.4

# See which anycast instance answered (look at the server's hostname)
dig +short hostname.bind chaos txt @198.41.0.4

This architecture means root servers can absorb massive distributed denial-of-service (DDoS) attacks by distributing the traffic across hundreds of nodes. The 2016 DDoS attack against Dyn (a DNS provider) highlighted how critical anycast is for resilience — anycast-based providers absorbed attacks that would have taken down unicast systems.

Use Case 2: Content Delivery Networks

CDNs like Cloudflare, Fastly, and Akamai use anycast for their edge networks. When Cloudflare announces that a customer's website is protected by Cloudflare, traffic flows through Cloudflare's anycast network:

1. Cloudflare announces a block of IP addresses (e.g., 104.16.0.0/12) from all of its 300+ data centers simultaneously.
2. A user in Mumbai is routed to Cloudflare's Mumbai PoP.
3. A user in Chicago is routed to Cloudflare's Chicago PoP.
4. Each PoP handles DDoS mitigation, TLS termination, caching, and load balancing before forwarding traffic to the origin server.

This gives Cloudflare the ability to: - Absorb massive DDoS attacks by spreading traffic across 300 locations - Serve cached content from the nearest PoP, reducing latency - Terminate TLS connections close to users, shaving hundreds of milliseconds off HTTPS handshake times

Cloudflare's 1.1.1.1 DNS

Cloudflare's public DNS resolver at 1.1.1.1 demonstrates how anycast enables fast DNS resolution. When you query 1.1.1.1, you are routed to the Cloudflare PoP closest to you — often just 1-5 milliseconds away. Without anycast, all queries would travel to a central server, potentially adding 50-200ms of latency.

Use Case 3: NTP Time Servers

The Network Time Protocol Pool (pool.ntp.org) uses anycast-like techniques combined with DNS rotation to distribute time synchronization load. Large cloud providers also use anycast for their internal NTP services.

AWS Time Sync Service, for example, provides 169.254.169.123 as a link-local anycast address available from every EC2 instance. Each AWS availability zone answers this address from local hardware, ensuring sub-millisecond time synchronization without any cross-region traffic.

Anycast vs. Unicast Load Balancing

It is worth clarifying how anycast differs from traditional load balancing:

The Session Stickiness Problem

Anycast has one important limitation: it is not suitable for stateful protocols that require a long-lived connection to the same server. If a BGP route changes mid-session (due to congestion, a router upgrade, or a policy change), traffic can suddenly shift to a different server that has no knowledge of the existing session state.

This is why anycast is commonly used for: - UDP-based protocols like DNS, where each query is independent - Short-lived TCP connections like HTTP/1.1 requests - Stateless services like DDoS scrubbing

It is generally avoided for: - Long-lived TCP connections with application state - Database connections - WebSocket connections

Modern CDNs work around this by using anycast only to get traffic to the nearest PoP, then maintaining a stable unicast connection from the PoP to the origin or using consistent hashing within the PoP for session affinity.

Implementing Anycast: The Technical Requirements

To deploy anycast, an organization typically needs:

1. Their own AS number — Required to announce prefixes via BGP. Obtained from an RIR.
2. IP address space — A block that can be announced globally. PI (Provider Independent) space is preferred over PA (Provider Aggregatable) space.
3. BGP sessions — Peering relationships at each PoP location. This requires physical presence at an IXP or colocated with a transit provider.
4. Consistent server configuration — All anycast nodes must respond identically to queries. This requires synchronized configuration management and identical software versions.
5. Health checking — Automation that withdraws the BGP announcement when a server is unhealthy, to prevent traffic from being sent to a failed node.

Organizations without their own AS can use "borrowed anycast" by working with a CDN or DNS provider that announces the IP addresses on their behalf.

Anycast and IPv6

Anycast works identically with IPv6. In fact, the IPv6 specification explicitly includes anycast as one of three address types (unicast, anycast, multicast). IPv6 anycast addresses are syntactically identical to unicast addresses — there is no special format that identifies an address as anycast. The distinction is purely in how the address is routed.

IPv6 loopback and link-local addresses are excluded from anycast use, but all global unicast addresses can be deployed as anycast. Major DNS providers and CDNs have IPv6 anycast networks that mirror their IPv4 infrastructure.

Understanding anycast helps explain why the internet is so resilient: critical services like DNS are not hosted on a handful of servers that could be taken offline, but distributed across hundreds or thousands of locations, with BGP automatically routing around failures.