IETF DNSOP Working Group S. Sato
Internet-Draft T. Matsuura
Expires: March 8, 2008 Y. Morishita
JPRS
September 5, 2007
BGP Anycast Node Requirements for Authoritative Name Servers
draft-sato-dnsop-anycast-node-requirements-02.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on March 8, 2008.
Copyright Notice
Copyright (C) The IETF Trust (2007).
Abstract
This memo describes the details of requirements and preconditions for
making "Global-Scope" IP anycast nodes for authoritative name
servers, with the conformance of the practices in RFC 2182, 2870,
3258 and 4786. This memo is intended to be used for the DNS
operators who are going to deploy IP Anycast nodes, especially for
the TLD operators.
Sato, et al. Expires March 8, 2008 [Page 1]
Internet-Draft Anycast Node Requirements September 2007
1. Introduction
IP anycast [1] is a technology to share one IP address for the
Internet services with multiple server nodes. It is now being
deployed for improving service reliability, scalability, and
stability. Especially, "BGP anycast" is now being deployed for
authoritative name servers, typically for root servers.
By applying the IP anycast technology to DNS, name server operators
can increase the number of authoritative name server nodes, and
distribute them in topologically and geographically diverse
locations, without violating the DNS protocol limitations [2] [3].
If IP anycast is appropreately operated for DNS server nodes, it
improves the robustness against Denial-of-Service (DoS) Attacks and
Distributed Denial-of-Service (DDoS) Attacks and improves the
reliability of DNS service. And it improves the DNS total response
by decreasing RTT from the clients, and distributes authoritative
name servers' load, too.
However, IP anycast needs more careful operations for achieving the
original goals and improvements. In the IP anycast technology, IP
address does not specify the individual (real) end-point for the
Internet communications any longer. Which means that the real
communication peer can not be specified by the destination IP address
only. It means some new risks for the DNS service can be found, due
to the increase of physical servers. For example, difficulty of
monitoring the availability of the service, and an extra effort for
syncing the date of the all DNS server nodes.
The original goals of IP anycast are to improve the robustness and
the reliability of the services. By introducing IP anycast, the
reliability of the entire system must not decrease. Especially, DNS
is one of an important infrastructures of the Internet, so,
introducing IP anycast in critical DNS authoritative servers should
be done carefully.
RFC 3258 [4] describes a set of practices to apply IP anycast
technology for authoritative name servers. And "Operation of Anycast
Services" RFC 4786 [5] describes a series of recommendations and
considerations for distribution of services using IP anycast.
On the other hand, operators of authoritative name servers can also
refer to RFC 2182 [6] and 2870 [7] for general guidances on
appropriate practices for authoritative name servers.
This memo describes the details of requirements and preconditions for
making "Global-Scope" IP anycast nodes for authoritative name
servers, with the conformance of the practices in RFC 2182, 2870,
Sato, et al. Expires March 8, 2008 [Page 2]
Internet-Draft Anycast Node Requirements September 2007
3258 and 4786.
And this memo also describes our findings and experiences for making
IP anycast nodes for ourself. Authors hope that it is useful for DNS
operators who will walk on the same way in the future, especially for
TLD operators.
Authors focus on "BGP anycast" for making "Global-scope" IP anycast
nodes. It is the currently most popular technology for making IP
anycast nodes and it is already used for making some root servers and
some TLD DNS servers. Authors think a part of the basic point of
view can be also applied to "Local-scope" IP anycast.
2. BGP Anycast and DNS Service
BGP anycast is a part of IP anycasting technology. It uses a shared
IP address block and a shared AS number for BGP anycast nodes, and
their nodes are placed in the Internet. Reachability of each nodes
are served by BGP routing protocol [8].
Each anycast nodes propagate the routing information of the shared IP
address block and AS number by BGP. Each BGP routers in the Internet
choose "nearest" node by BGP's best route selection algorithm. That
is, the accesses to the shared IP address block will be distributed
to the each anycast nodes, depending on the clients' locations and
network topologies.
BGP anycast can control each anycast nodes by configuring as "local
node" or "global node" using BGP's routing framework. Using the
"NO_EXPORT" [9] or "NOPEER" [10] community, the "local node"
operators can limit distributing the routing information of anycast
node only for the directly peering sites. Therefore, the "local
node" can localize the access to anycast node from directly peering
sites. On the other hand, the "global node" operators apply the
normal BGP anycast for its node. In this memo, authors focus on
"global node" as main target.
When one of BGP anycast nodes goes down, routing informations will be
automatically recalculated. The datagrams to the anycast node are
automatically rerouted to other anycast nodes. Thus, BGP anycast can
provide redundancy for the Internet services.
Current BGP anycast is hard to apply for longer-lived transaction
service, because the instability of the dynamic routing protocol can
carry the packets to different anycast nodes. But most of the DNS
queries and answers are based on a single UDP packet, so, DNS is
considered to be one of the typical services which BGP anycast can be
Sato, et al. Expires March 8, 2008 [Page 3]
Internet-Draft Anycast Node Requirements September 2007
applied to.
In this memo, authors focus the critical DNS infrastructure,
especially root and TLD DNS servers. So, authors only focus a case
of "a single IP address for DNS service covered by a single exported
route". It means advertisement and withdrawal of a single covering
prefix can be tightly coupled to the availability of the single
associated service, as described "4.8.1. Multiple Covering Prefixes"
of Anycast BCP.
In this situation, DNS service providers need an exclusive IP address
block which is a provider independent CIDR block and an exclusive AS
number for making each anycast nodes.
3. Requirements and Preconditions for IP Anycast DNS Server Nodes
As described above, IP anycast is one of the effective ways for
improving the robustness and the reliability of the services. In the
recent critical authoritative name servers, especially for large
TLDs, ability to handle more data, more frequent data updating, and
higher reliability than before are required.
So, when BGP anycast technology is applied to the critical servers,
the operators have to be very careful about their systems. Authors
expect that the requirements and preconditions which described by
this memo would be useful.
In this section, this memo describes requirements and preconditions
for making BGP anycast nodes for authoritave name servers in the
following two points of view, the Internet access service, and data
center.
3.1. Choosing the Internet Access Service
For making BGP anycast nodes distributed in the wide area, it is
important to make network environment with geographical and network
topological diversity.
In case of making such network environment, each anycast nodes should
have Internet connectivity from different Internet access service
provider (hereafter, called ISP) for ensuring network diversity.
And in case of ensuring the BGP connectivity, the owner of the
authoritative name servers must consider the following preconditions
and requirements to choose the Internet access services.
Sato, et al. Expires March 8, 2008 [Page 4]
Internet-Draft Anycast Node Requirements September 2007
3.1.1. Reliability of the Backbone Network
When making a critical authoritative name server, higher reliability
for ISP's network itself is needed. For implementing this, it is
desirable for ISP itself to have owned and managed its backbone
network.
In general, an ISP which owns and manages the backbone network itself
is expected to have stronger responsibility for network stability.
Then it is expectable that the stability of a network is higher. Of
course, it is not absolute requirement, but it will surely be one of
the important elements.
3.1.2. Connectivity to the Global Internet
In case of critical authoritative name servers, such as the servers
for root and/or TLD zones, there are many accesses from not only its
country and its local area but also foreign area. Thus, connectivity
for them must be needed. In the same reason of Section 3.1.1, it is
desirable that an ISP which owns and manages the stable connectivity
to the global Interenet by itself.
3.1.3. Number of the Peerings
When ensuring highly reliable Internet connectivity, the number of
the peering is an important element for ensuring the diversity of
Internet routes, including multiple alternative paths. Moreover,
providing DNS service to multiple ISP networks efficiently, it is
desirable for the ISP to have many BGP peers with other ISPs to have
much stable connectivity.
3.1.4. Connectivity for Provider Independent CIDR Block and AS Number
When making a set of BGP anycast node for critical authritative name
servers, a provider independent CIDR block and an AS number must be
prepared in advance. The same CIDR block and the AS number must be
used for the DNS service at all anycast nodes. It is similar to make
a multihomed connectivity.
In this case, the ISP must support propagating CIDR block and AS
number for anycasting service to the Internet widely, and the ISP
must provide connectivity for them from the Internet. Concretely,
the ISP must provide "transit" service. As in the case of the
multihomed network, each anycast node should improve its network
reliability by its own local connectivities.
Sato, et al. Expires March 8, 2008 [Page 5]
Internet-Draft Anycast Node Requirements September 2007
3.1.5. Connectivity for Operations and Data Synchronization
As in RFC 3258, an Internet connectivity which is different from IP
anycasting must be needed for anycast node operations. And as in RFC
4786, the synchronization of data between anycast nodes involves
transactions between non-anycast addresses. This connectivity should
be separated from the connectivies for the DNS service. The assured
operations are required at any time, even when the DNS service is not
available. When using the same connectivity for both the operations
and the DNS service, QoS may work well.
3.1.6. Connectivity for IPv6
RFC 3513 [11] prohibited host-based anycast in IPv6. But RFC 4291
[12] removed this limitation, and RFC 3513 is generally obsoleted.
So, there are no protocol limitations for using IP anycasting for
IPv6 network by using the same technology as IPv4. But currently,
there are less experiences for the IPv6 anycast services. That is,
the anycast node owner should take extra care for the IPv6
connectivity.
3.2. Choosing the Location
RFC 2182 and 2870 can be refered for useful guidance on appropriate
practices for choosing the BGP anycast node locations of the critical
authoritative name servers, By referencing them, the owner of the
anycast nodes should consider the following preconditions and
requirements.
3.2.1. Providing Higher Security Level
To realize higher defense performance to physical destruction and/or
the intrusion from the outside, the location must provide higher
security level.
The IP anycast technology may reduce the impact of the failure at
single node, but it does not mean the security requirements for the
each node are relaxed. Malicious attacks against the physical
servers can bring more serious problems, such as data falsifications.
3.2.2. Providing Higher Tolerance Against Disasters
DNS service requires high continuity and stability, the location must
provide high tolerance against disasters, for example fire,
earthquake and other disasters.
The IP anycast technology may reduce the impact of the failure at a
Sato, et al. Expires March 8, 2008 [Page 6]
Internet-Draft Anycast Node Requirements September 2007
single node or location, but the each locations must have high
tolerance against disasters by its own.
3.2.3. Providing the Diversity of Locations
For ensuring tolerance and redundancy, the diversity of locations is
important. Concretely, even if a fatal disaster occurred at one
location, other locations should not be damaged by the same factor.
The continuity of the critical DNS service must be ensured.
To select the locations, the choice of the place can be found from
all around the world, because the DNS servers are accessed by the
resolvers in the whole Internet.
The basis for the selections may come from the measurements. The
source addresses of the DNS queries shows the distributions of the
clients. RIPE DNSMON can show the RTT from many regions. These
measurements are the good hints for the selections of the locations.
4. Cost and Operational Issue
In the technical point of view, BGP anycast nodes can be made in
numbers of locations. To make the high tolerance to the DDoS
attacks, the number of locations should be high. But it is not
realistic to prepare them more than necessity. In general, to
satisfy the preconditions and requirements which is previously
described, BGP anycast node needs high cost, including financial,
human and operational resources.
In the current condition, this cost is mandatory for making BGP
anycast node. Especially, to guarantee the quality of service, for
example SLA (Service Level Agreement), needs higher cost than normal
Internet connectivity. This is one of big burden for operating BGP
anycast node. The authors believe that this is one of in the future
issue for deploying IP anycast.
Furthermore, for administrating remote anycast nodes smoothly, many
human recources are needed, including local and remote technical
staffs and operators. When making BGP anycast node for critical DNS
service, the owner of authoritative name servers must consider about
this issue.
5. Measurement Issue
To evaluate the practical effect of IP anycast, for example, to
verify whether the selections of the IP anycast nodes are appropriate
Sato, et al. Expires March 8, 2008 [Page 7]
Internet-Draft Anycast Node Requirements September 2007
or not, objective measurement is very important. When making BGP
anycast mesh in the wide area, the measurement must also be carried
out in the wide area.
In such case, there is an effective guideline defined by ICANN,
called "CNNP test" [13]. This guideline is useful for making
critical DNS server nodes.
The service continuity is another important point for the DNS
service. The operators should ensure the continuity of DNS service
by measurement. RIPE NCC's DNSMON service [14] is one of typical
notable project.
However, the cost of the measurement is very high, and it is hard to
make and maintain many measurement points/probes.
6. Data Synchronization
In this section, this memo describes the effective monitoring of the
data synchronization among the anycast nodes.
The anycast nodes tend to be widely distributed around the world, and
the number of the servers increases as the anycast nodes increse.
Thus, the data synchronization among all DNS servers and the
monitoring of the synchronization come up as an important issue.
6.1. Synchronization Method
The network connectivities between the master DNS server which
provides zone data and the slave DNS servers in each anycast nodes
may not be so stable. These can be happen when the advantages of the
selection of the locations exceed the disadvantage of the network
connectivity selections. To support these cases, data transaction
should be designed to decrease the amount of the traffic. In
particular, zone transfers should be performed in incremental zone
transfer (IXFR) method to decrease the traffic and shorten the
transaction time.
However, from the operational point of view, not only IXFR but AXFR
method should be provided to ensure the data. AXFR should be
enforced in the case of finding the broken data consistency.
All data synchronization transactions must be performed by the
unicast addresses. And the the zone transfer source should be
duplicated in more than two diverse locations. It is useful for
providing the diversity to the network connectivity for the zone
transfers.
Sato, et al. Expires March 8, 2008 [Page 8]
Internet-Draft Anycast Node Requirements September 2007
6.2. Monitoring the Synchcronization
To check the synchronization of the all anycast nodes independently,
the monitoring of the SOA serial number should be performed to the
unicast addresses used for the zone transfers.
The checks must be performed frequently, according to the zone update
frequency. However, the continuous checks of the SOA serials of all
anycast nodes are effective.
To have a process only for querying and recording the SOA data, and
another process to check the synchroneity of the data will make a
effective monitoring system.
The monitoring system should be seperated from the DNS servers itself
to ensure the objective monitoring of the systems.
7. Our Findings through Making Overseas Anycast nodes
In this section, this memo describes our findings for making oversea
IP anycast nodes for our DNS server. At this point, these server
nodes are still for reserach and development, but when they were
made, we did some useful experiences and got some useful findings.
Authors hope that it is useful for DNS operators that they will walk
on the same way in the future, especially for TLD operators.
7.1. Importance of the Running Cost
Running cost is more dominant than initial cost. Human resources and
traveling expense are needed for troubleshooting and trouble
recovery. Especially for oversea sites, sometimes the linquistic
barriers apper as a high cost. For instance, a simple replacement
may reduce the total cost from finding and fixing the minimum failure
by using remote hands.
7.2. Difference of Business Practices
Differences of business parctices among the countries are imporatant
issue for practical server node operations. Some data center
requires damage insurance contract. But it is hard to have contract
with foreign customer.
7.3. Broken Hardware Replacement
In many cases, both the server and network hardwares which have a lot
of experiences in one country do not have any maintenance
availability in other countries. It is important for the gears to
Sato, et al. Expires March 8, 2008 [Page 9]
Internet-Draft Anycast Node Requirements September 2007
have ability for the maintenance in many countries, especially in the
location of the nodes. The gears of the global companies may be
better to choose.
7.4. Operation and Maintenance Tools
To ensure the operations and maintenances of the system, each anycast
nodes should prepare the means to access each gears as many as
possible.
Those are remote KVM switches, serial console switches, remote
management systems of the servers and others. The troubles cannot be
all predicted, thus the means of operations should be prepared
widely.
The remote connectivity without using the Internet, such as dial-up
network directly to the local network or the console switches of the
anycast node, is also suggested.
7.5. Time Synchronization
To make the operation, measurement and trouble shooting easily and
effectively, all the anycast nodes must be using the synchronized
clocks from the same source. In addition, time zones used for the
gears should be uniformed.
8. Security Considerations
IP anycast nodes mitigate DoS attack effect and constrain DDoS attack
in their local network. It is one of the most important goals of IP
anycast.
To keep higher security level of each DNS server nodes is one of most
important points of managing critical authrotative name servers. Of
couese, it is the same as non-anycasted DNS service, but in IP
anycasting environment, all of IP anycast nodes have the same IP
address for authoritative DNS service, so, it means it is much
important than single server because all of nodes should be applied
the same higher security policy.
In IP anycast environment, number of nodes increases compared with
non-anycasting environment. It means the number of the places where
the safety of the data must be guaranteed, also increases. And
practical secure data synchronization method between nodes must be
required, typically data encryption.
Sato, et al. Expires March 8, 2008 [Page 10]
Internet-Draft Anycast Node Requirements September 2007
9. IANA Considerations
This document requests no action from IANA.
10. Acknowledgements
Paul Vixie and Bill Manning reviewed a previous version of this
document. Joe Abley reviewed a previous version of this document and
provided detailed and useful comments.
Yoshiki Ishida, Tomoya Yoshida, George Michaelson and Peter Koch
provided some useful comments and suggestions.
The authors acknowledge many helpful suggestions from the members of
JPRS Research and Development Department and System administration
Department.
This memo is included in the results of the research activities
funded by National Institute of Information and Communications
Technology (NICT).
11. References
[1] Partridge, C., Mendez, T., and W. Milliken, "Host Anycasting
Service", RFC 1546, November 1993.
[2] Mockapetris, P., "DOMAIN NAMES - CONCEPTS AND FACILITIES",
RFC 1034, November 1987.
[3] Mockapetris, P., "DOMAIN NAMES - IMPLEMENTATION AND
SPECIFICATION", RFC 1035, November 1987.
[4] Hardie, T., "Distributing Authoritative Name Servers via Shared
Unicast Addresses", RFC 3258, April 2002.
[5] Abley, J. and K. Lindqvist, "Operation of Anycast Services",
RFC 4786, December 2006.
[6] Elz, R., Bradner, S., and M. Patton, "Selection and Operation
of Secondary DNS Servers", RFC 2182, July 1997.
[7] Bush, R., Karrenberg, D., Kosters, M., and R. Plzak, "Root Name
Server Operational Requirements", RFC 2870, June 2000.
[8] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)",
RFC 1771, March 1995.
Sato, et al. Expires March 8, 2008 [Page 11]
Internet-Draft Anycast Node Requirements September 2007
[9] Chen, E. and J. Stewart, "A Framework for Inter-Domain Route
Aggregation", RFC 2519, February 1999.
[10] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP)
Route Scope Control", RFC 3765, April 2004.
[11] Hinden, R. and S. Deering, "Internet Protocol Version 6 (IPv6)
Addressing Architecture", RFC 3513, April 2003.
[12] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, February 2006.
[13] "ICANN Unsponsored TLD Agreement: Appendix D (.info)",
May 2001.
[14] "RIPE-NCC/DNS Server Monitoring", <http://dnsmon.ripe.net/>.
Authors' Addresses
Shinta Sato
Japan Registry Services Co., Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: shinta@jprs.co.jp
URI: http://jprs.jp/
Takayasu Matsuura
Japan Registry Services Co., Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: matuura@jprs.co.jp
URI: http://jprs.jp/
Sato, et al. Expires March 8, 2008 [Page 12]
Internet-Draft Anycast Node Requirements September 2007
Yasuhiro Orange Morishita
Japan Registry Services Co., Ltd.
Chiyoda First Bldg. East 13F, 3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo 101-0065
Japan
Email: yasuhiro@jprs.co.jp
URI: http://jprs.jp/
Sato, et al. Expires March 8, 2008 [Page 13]
Internet-Draft Anycast Node Requirements September 2007
Full Copyright Statement
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Sato, et al. Expires March 8, 2008 [Page 14]