IPv6 Operations (v6ops) Working Group N. Buraglio
Internet Draft Energy Sciences Network
Intended status: Informational K. Frank
Expires: September 2023 P. Nero
Private
P. Volpato
E. Vasilenko
Huawei Technologies
March 3, 2023
IPv6 Site connection to many Carriers
draft-fbnvv-v6ops-site-multihoming-00
Abstract
Carrier resilience is a typical business requirement. IPv4
deployments have traditionally solved this challenge through private
internal site addressing in combination with separate NAT engines
attached to multiple redundant carriers. IPv6 brings support for
true end-to-end connectivity on the Internet, and hence NAT is the
least desirable option in such deployments. Native IPv6 solutions
for carrier resiliency, however, have drawbacks. This document
discusses all currently-available options to organize carrier
resiliency for a site, their strengths and weaknesses, and provides
a history of past IETF efforts approaching the issue. The views
presented here are the summary of discussions on the v6ops mailing
list and are not just the personal opinion of the authors.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 2023.
FBNVV Expires September 3, 2023 [Page 1]
Internet-Draft site multihoming March 2023
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
Table of Contents
1. Terminology and pre-requisite..................................2
2. Problem statement..............................................3
3. Problem history for the host-driven solution...................7
4. Solution requirements..........................................9
5. Available Solutions...........................................10
5.1. PI-based.................................................10
5.2. PA-based solution........................................13
5.3. ULA with NPT.............................................17
5.4. ULA with NAT66...........................................20
5.5. Shifting the problem to the centralized site.............22
6. Conclusion....................................................26
7. Security Considerations.......................................29
8. IANA Considerations...........................................29
9. References....................................................29
9.1. Normative References.....................................29
9.2. Informative References...................................31
10. Acknowledgments..............................................32
1. Terminology and pre-requisite
Terminology is inherited from [ND] and [SLAAC]. Additional terms:
RIR (Regional Internet Registry): an organization that manages
Internet numbering resources (such as IP addresses and autonomous
system (AS) numbers) within a geographical region of the world.
FBNVV Expires September 3, 2023 [Page 2]
Internet-Draft site multihoming March 2023
LIR (Local Internet Registry): an organization (usually a carrier
or an Enterprise/Academic) that receives its allocation of IP
addresses from the Regional Internet Registry (RIR), then assigns
parts of that allocation to its customers.
PA (provider-assigned or provider-aggregatable) address space: a
block of IP addresses assigned to the end customer by a carrier
or other owner that received it inside the bigger block from RIR
or LIR. The principal characteristic of the PA address block is
that it is aggregated in the bigger block (not announced
separately) in the Internet routing tables.
PI (provider-independent) address space: a block of IP addresses
assigned by RIR or LIR to the end customer with the possibility
for independent announcements in the Internet routing tables.
DHCP-PD: IPv6 Prefix delegation by DHCP from the carrier to the
client to be used for numbering through the client site (As
currently defined by [RFC8415] Section 6.3).
End-to-end connectivity: The possibility to initiate the connection
from any direction, including the case of complex protocols with
many logically related sessions.
ALG (Application-level gateway): Monitoring connections data to
discover and fix application-level referrals (embedded IP
addresses).
Subscriber-only services: The resource that is filtered on the
public Internet and available only for some portion of the
Internet, typically only for subscribers of the particular
carrier.
2. Problem statement
Always-on connectivity is a key requirement for the vast majority of
businesses. [IAB report] predicts that there "might be as many as 10
million multihomed sites by 2050". Unfortunately, several issues may
affect the connection of a business to its upstream service
provider. For example, the carrier's network, the network gateway,
or the first-mile infrastructure may experience issues. It is
especially true now after many recent examples of massive carrier
outages.
A redundant connection to the carrier is then the norm for business.
FBNVV Expires September 3, 2023 [Page 3]
Internet-Draft site multihoming March 2023
A simple topology that showcases this key requirement is shown in
figure 1. Note that the topology could be more complex as shown, for
example, in figure 3 [MHMP Enterprise].
+------+ _________
| | / \
+---| CPE1 |----/ Carrier \
2001:db8:0:1001::xx | | |\ \ 1 /
+------+ | +------+ \ \_________/
| | | \
| Host |----+ \
| | | \
+------+ | +------+ \_________
2001:db8:0:2001::yy | | | / \
+---| CPE2 |----/ Carrier \
| | \ 2 /
+------+ \_________/
Figure 1: The basic Carrier Resiliency topology
Without entering too much into details, resilience is generally
achieved by employing redundant elements. Two Customer Premises
Equipment (CPE) systems are usually employed. Very often, each CPE
connects the business to a different carrier. In some cases, a CPE
may even connect to two different carriers, to achieve a higher
level of protection against network failures.
In literature, a topology such as that shown in figure 1 based on
IPv6 connectivity is often referred to as Multi-Home, Multi-Prefix
(MHMP). The name implies that a network is multi-homed to different
carriers, receiving from them different network prefixes.
FBNVV Expires September 3, 2023 [Page 4]
Internet-Draft site multihoming March 2023
The problem of providing network protection with IPv6 was thoroughly
discussed in [Local Protection]. This document is an overview of the
most commonly used methods to facilitate the desired protection in
modern IPv6 networks, along with a discussion of the advantages and
disadvantages of each method.
In IPv4 environments, such a scenario is implemented through
independent NAT translation on every CPE to the carrier in
combination with private address space on-site. [Local Protection]
has a good list of benefits. The solution was initially adopted due
to the shortage of globally-reachable address space. Later, security
and carrier resiliency were identified as additional benefits. Due
to the prevalence of a solution based around address translation in
IPv4, demands are often voiced for such a mechanism in IPv6 as well,
running the risk of being chosen by network designers without
evaluating alternative options.
[Local Protection] has a list of IPv6 tools that replace all
functionality of the NAT solution except address conservation, which
is not a necessity in IPv6. Time has shown that all IPv6 tools have
been accepted as valid replacements except IPv6 "multi-homing and
renumbering". Discussions about this last problem are ongoing.
The approaches currently available to address local protection and,
more specifically, the multi-homing and renumbering issue are the
subject of this analysis and are listed hereafter.
There are 4 potential possible solutions:
- Static PI address space for the site, routed by multiple carriers,
- Dynamic PA addresses distribution and withdrawal from carriers,
- Static ULA address space for the site with NPT translation,
- Static ULA address space for the site with NAT66 translation.
The simplest solution is to have Provider Independent (PI) addresses
on-site with proper routing announcements by carriers on behalf of
the client. This method does not have a significant discussion
history because the solution was always stable and simple. The
solution has challenges that prevent it from becoming universal and
widespread. It will be discussed in section 5.1.
A widespread approach sees an [IPv6] host have many IPv6 addresses
on its link(s). These addresses may be Provider Aggregatable (PA)
from different carriers. In this typical scenario, the host has the
challenge of properly choosing the combination of 1) a source
address that would not be filtered by a particular resource on the
FBNVV Expires September 3, 2023 [Page 5]
Internet-Draft site multihoming March 2023
Internet, and 2) a next hop that is leading to the carrier who owns
the source address space, and which would not filter the packet.
This host-driven solution is primarily based on the interaction of
[ND] and [SASA]. Routers should supply all needed information, and
then the host should choose the proper combination of source address
and next hop for any destination address that the host would like to
communicate with.
Further challenges may make this scenario even more complex. In
particular, the site may have a multi-hop topology (many links and
routers). In such a case, the decision taken by the host on how to
deliver traffic to the carrier that has assigned the source address
needs to be complemented with source routing. This source-routing
problem is properly discussed in [MHMP Enterprise].
In the subset of scenarios where the path for return traffic is
required to be symmetric, such as when performing application-based
traffic steering, the issue is further complicated. It has become
common in enterprise networks to steer traffic from a host in the
internal network to exit via different Internet uplinks based on
transport and application-layer information, instead of only the
network characteristics of packets. The edge gateway can in this
case perform complex forwarding decisions based on a policy table
and deep-packet inspection, but routers on the Internet do not share
the same list of policies and are thus unable to guarantee the same
path for returning packets in a given flow. Similar mechanisms have
until today relied on IPv4 PA addressing combined with NAT44 to
guarantee flow symmetry and centralize the decision-making process
when selecting which carrier to deliver the traffic to.
An additional challenge is posed by DNS. If the host has separate
interfaces then it is possible to have a separate DNS resolver for
each interface.
[Provisioning domains] is needed when the single router has many
upstream carriers. [Provisioning domains] allows associating the
prefix used to select the source address with the proper DNS
resolver; otherwise, the packet might be delivered to the wrong
carrier (due to missing or irrelevant DNS information) and then
filtered out.
The host-driven solution is complex (involving [ND], [SASA], source
routing, and DNS), and hence it has a long history of improvements
and debates. It is extensively discussed in section 5.2.
FBNVV Expires September 3, 2023 [Page 6]
Internet-Draft site multihoming March 2023
Other available solutions see the use of [NPT] and NAT66 in
combination with Unique Local Addresses (ULA). The IETF consensus to
preserve end-to-end connectivity does not favor the general
acceptance of these two solutions, yet they are described in
sections 5.3 and 5.4 because they offer a degree of support for the
requirements listed in section 4.
3. Problem history for the host-driven solution
Client applications typically utilize getaddrinfo() to establish
communication and to perform source address selection. It was
initially assumed in [SASA] and implemented by getaddrinfo() that
the next hop is chosen before the source address.
Bind() permits overriding this order, but it is typically used only
in server-side applications.
This specific process is the reason why in the case of a network
fault, when a redundant CPE/link is promoted to the primary role,
some specific destinations may become unreachable, causing the
solutions listed hereafter to leave some unresolved scenarios.
The initial discussion on carrier resiliency occurred in the [Shim6]
working group, where it was proposed to separate the location and
identity properties of addresses. [Shim6] did not gain market
acceptance.
The next notable step was the addition of rule 5.5 to [SASA] to
prefer the selection of a source address covered by a prefix
advertised by the chosen next hop router. This allows the packet to
be on the path to the PA-owning carrier and avoid packet filtering
due to the application of [BCP38].
With reference to figure 1, if connectivity to Carrier 1 were lost
then the host would select a source address in the prefix belonging
to Carrier 2 to communicate with a certain destination outside of
its local segment, thus achieving resiliency.
On the other hand, this method is not a solution when only a
particular source address is permitted (i.e., not filtered) to
access a particular outside resource (e.g., "subscriber-only
services") or when any type of deterministic traffic distribution
policy is desired, because the random next hop choice would in turn
lead to a random choice of source address.
FBNVV Expires September 3, 2023 [Page 7]
Internet-Draft site multihoming March 2023
[MHMP] then discussed the problem thoroughly, making an attempt to
use only already available tools. It was suggested that the solution
must push related policies to the host as 1) "Routing Information
Options" of [Route Preferences] and 2) [Policy by DHCP] to modify
policies in the host's [SASA] selection algorithm. This solution has
not gained market acceptance due to complexity reasons. Moreover,
DHCPv6 is not universally implemented, being notably absent from
some of the most widely deployed client platforms.
Additionally, [HNCP] prescribed to deprecate delegated prefixes (by
setting their preferred lifetime to zero) when the router has
information about loss of reachability to the carrier that sourced
the prefix. This is particularly important when renumbering occurs
(the PA prefix may change after disconnection and re-connection to
the carrier).
The next solution was proposed in [Multi-Homing]. The document
incorrectly assumed (errata 7009 and 7010 are published) that the
source address is chosen first in the typical scenario when a client
initiates outbound communications. [Multi-Homing] section 3.2
proposed to prefer next hops from those routers that advertise the
prefix covering the already selected source address. Hence, [Multi-
Homing] unblocked the possibility for an application to use bind()
to select the source address first since section 3.2 contains vital
instructions on how to choose the next hop in that condition. It is
important to note that the more common scenario of choosing the next
hop before the source address is not solved by [Multi-Homing].
Further progress in the problem discussion was made in [MHMP
Enterprise], which discusses potentially complex on-site topologies
and the source routing that is needed in such a scenario; it may in
fact be considered a comprehensive guide covering the source routing
aspect of the complex site with carrier resiliency. Unfortunately,
at this time it is not yet possible to use it in practice due to the
lack of any market-accepted solutions to split and distribute PA
prefixes throughout the complex site. All other solutions (PI,
ULA+NAT, ULA+NPT) do not require source routing in principle. [MHMP
Enterprise] might become vital in the future if a solution were to
be adopted for splitting and propagating PA prefixes through the
complex site; however, such a solution is currently unavailable.
Restrictions to the list of applicable source addresses for a
specific next hop (rule 5 or 5.5 of [SASA]) may not have been
implemented in certain host operating systems. [Conditional PIO] can
in this case mitigate the problem through selective PIO
announcements to a particular host. This represents a valuable
FBNVV Expires September 3, 2023 [Page 8]
Internet-Draft site multihoming March 2023
transition mechanism until rule 5.5 of [SASA] will be universally
applied.
[DNS Options] allows the router to supply the addresses of many
different DNS resolvers, including those of completely unrelated
carriers. It is not possible, however, to provide information to
clients regarding which DNS resolver is related to which particular
prefix; such information might be crucial in scenarios where traffic
steering policies are required for successful communication
(including when accessing filtered resources such as "subscriber-
only services").
Finally, [Provisioning domains] permit virtualization of the router
on the link, representing one physical device as many logical ones
with fully separate sets of link parameters. This solution is
valuable in some scenarios to deliver more diverse information to
the host but does little to assist it with choosing the proper
combination of next hop and source address that is still restricted
by [SASA]. The challenge remains the same independently of how many
physical or logical routers are present on the link. Moreover,
virtualization of a single router on a link having two uplinks to
different carriers creates a problem, because the host could
randomly choose the wrong combination of source address and next hop
announced by one of the virtual routers.
Yet, [Provisioning domains] remains valuable in scenarios where
several different routers are behind a single router, as well as
when multiple physical routers are present on the same link (i.e.,
the problem of host choice already exists). This is because, in
contrast to [DNS Options], [Provisioning domains] retains the
information that associates prefixes with the DNS resolvers of their
respective carrier.
4. Solution requirements
1. Site resiliency to an arbitrary number of carriers.
2. End-to-end connectivity wherever possible.
3. Possibility for internal communication using any prefixes
distributed by local routers, irrespective of the status of the
connectivity to the carriers that distributed such prefixes.
4. Sub-second convergence for the prefix deprication on the site
after connectivity is lost to any particular carrier.
5. Support for sites with complex topologies, including multiple
internal on-site hops requiring many routers and links.
6. Access to resources from the carrier's "subscriber-only
services" that is permitted only when using the address space
distributed by the particular carrier.
FBNVV Expires September 3, 2023 [Page 9]
Internet-Draft site multihoming March 2023
There may exist a need for the host to check DNS resolvers from
all carriers before it can even discover the restricted
resource.
A given host may thus need to choose the correct source address
that would be accepted by the particular carrier.
7. Possibility for traffic steering between different paths
(including both internal to the site, and the Internet) based
on bandwidth, cost, load, latency, packet loss, hop count, etc.
(e.g. application traffic/path engineering) for both outbound
and inbound packets.
It is out of scope for this document to evaluate issues related to
perimeter security; every system should assume an insecure
environment, which is already the case since the host is
establishing frequent connections to the open Internet.
Different environments have diverse security policies, needs, and
obligations that may be shaped by internal policy, regulatory
compliance, or national security requirements, and as such will not
be discussed in detail in this document.
Privacy is furthermore assumed to be protected by [Temporary
addressing] and is also kept outside of the carrier resiliency
design consideration.
All solutions to the problem statement in this document would have
different cost advantages and disadvantages. The associated costs
may greatly vary for different geographies, market segments, and
organization sizes, but the choices should be ultimately driven by
any needs and requirements set upon the organization by their
respective governing bodies, with input from the appropriate subject
matter experts.
It is recognized that cost is often a determining factor in IT and
networking decisions, but it is also considered out of the scope of
this document.
5. Available Solutions
5.1. PI-based
While building or expanding an IPv6 addressing plan, it may become
necessary for an organization to procure or acquire additional
network resources; it is reasonably straightforward to obtain
additional Provider-Independent GUA address space from an RIR or
LIR. However, not all organizations have the in-house expertise,
desire, or capability to execute such a plan.
FBNVV Expires September 3, 2023 [Page 10]
Internet-Draft site multihoming March 2023
PI prefixes allow the creation of an address plan that would never
need renumbering, due to the non-dynamic and non-conditional nature
of the prefix allocation. Hence, the address plan may be stable
enough to be manually provisioned over all routers and links even in
a complex topology.
The long-term supportability and network convergence time of this
solution is excellent because there is no need for renumbering;
losing one of the Internet connectivities simply implies normal
routing updates with default routing status withdrawal by the
affected router.
It may, however, be a more involved process to get the PI prefix
routed by the carrier because such type of customer attachment is
typically charged more by the carrier due to the more complicated
nature of the connection and configuration (i.e. dynamic protocol
configuration, configuration of appropriate prefix filtering, and
equipment to support the necessary protocols). It generally requires
a manual procedure on the carrier's side, and hence it is not
possible to set this up without their cooperation.
For faster convergence, routers should announce a "default" on-site
and then additionally some more specific prefixes if needed.
Additional destination prefixes facilitate the distribution of
traffic between the carriers.
In cases where internal routing hops are present between the edge
devices, such announcements likely require other protocols in
addition to Neighbor Discovery.
As for the network interface of the internal host (the last hop),
such an announcement comes in the form of default router status in
[ND], and [Route Preferences] for the more specific routes.
The crucial requirement in the PI-based solution is that routers
have to continuously track connectivity to carriers to be able to
deprecate the "default" as well as any more specific announcements
when such connectivity is lost.
When multi-homing with PI space there are multiple entry points into
the local network for traffic destined to every single address in
the site, and this opens up discussion on how to load balance
incoming traffic to such a site. It is possible for example to split
the prefix into many smaller prefixes and announce them separately
to different carriers. This requires that the initial prefix be of
adequate size, in order to avoid ending up with announcements
FBNVV Expires September 3, 2023 [Page 11]
Internet-Draft site multihoming March 2023
smaller than the longest acceptable prefix length within the
default-free zone, which is conventionally /48. This advanced
incoming traffic engineering is possible only with PI address space.
Where traffic symmetry over the WAN is important, however, such as
in environments performing application-based traffic steering, PI
can fail to guarantee the necessary control over returning traffic.
Traffic engineering using routing advertisements provides benefits
for simple active/passive or active/active connectivity needs, but
it can also be insufficient; routing advertisements are not granular
or fast enough to make things work in scenarios where different
application traffic must be steered towards different uplinks based
on upper-layer information. As an example, an administrator might
need FTP traffic generated by an internal host to always exit
through the secondary Internet link, and for the return traffic to
also arrive back on that same link; allowing the downstream
component of FTP connections to occupy bandwidth on the primary link
might in fact be undesirable, and impact the performance of more
sensitive protocols running over the primary link. Even in the
extreme case where /128 prefixes for the internal host were
advertised, engineering different return paths based on upper-layer
information would not be feasible in a deterministic manner for the
same source address.
In addition, environments where the edge devices (commonly,
perimeter firewalls) are stateful might experience packet drops due
to TCP session flows being split across Internet links. This would
be especially true in the case where multiple stateful devices were
deployed, one for each Internet link; the devices would thus only
see a portion of the TCP sessions and never consider them fully
established, or miss enough segments that the next arriving sequence
number might be considered out of window from their standpoint. Both
conditions could cause the devices to drop segments. Note that these
symptoms would be exaggerated for TCP traffic but also possible with
other protocols such as UDP, if the stateful devices had a way to
track the sessions and allowed inbound UDP traffic only when a
matching tuple already existed in their session tables.
On the other hand, a crucial advantage of using a prefix which is
globally reachable through multiple Internet links is the seamless
failover of transport-layer sessions across the links without having
to re-establish them. Conventionally, in IPv4 environments, such a
failover would have been basically guaranteed to break existing
communication due to the ubiquity of PA addressing combined with
NAT44, and the consequent change in source address as seen by the
remote endpoint. This issue would also be felt when leveraging IPv6
FBNVV Expires September 3, 2023 [Page 12]
Internet-Draft site multihoming March 2023
multi-homing in a way that caused source address changes upon
failover, either due to NAT (see sections 5.3 and 5.4) or the prefix
of the existing connection becoming unusable (see section 5.2).
The biggest problem with PI addresses today remains the fact that
the widespread practice of using PI space would bloat the Internet
routing table by at least 10 times, which would greatly impact the
cost and scalability of all Internet routers. Admittedly, however,
this problem would not really be an immediate concern for any
company implementing a PI-based solution.
Advantages:
- Preserves end-to-end communication,
- Does not require any special functionality from the host,
- Straightforward design,
- Allows aggregation of multiple uplinks for increased throughput,
- Supports sites with complex topology,
- Seamless link failover without transport session re-establishment,
- Supports outbound traffic steering.
Disadvantages:
- Hard to implement especially for smaller entities, as it requires
knowledge of advanced routing protocols like BGP and availability
of advanced networking hardware (it is not as simple as plugging
two independent CPEs into the same switch as the client and having
it work),
- In case of provider failure a prefix may not be correctly
deprecated from global routing, and this could lead to complex
error scenarios and downtime,
- The need to pay and liaise with a RIR or LIR for the PI address
space,
- Needs carriers to accept PI prefix advertisements,
- Carriers typically charge significantly more for such type of
attachment; often SD-WAN contracts are required instead of
business DSL contracts,
- The return traffic path is not guaranteed to mirror the outbound
path,
- Bloats the Internet routing table.
5.2. PA-based solution
Let us assume that a site is connected to the Internet through many
carriers, with every carrier delegating a PA prefix (most likely
using DHCP-PD). We will also assume for this example that the local
FBNVV Expires September 3, 2023 [Page 13]
Internet-Draft site multihoming March 2023
CPE performs no proprietary functions and that hosts are provisioned
with an address from each PA allocation.
It is mandatory to implement requirement L-13 of [HNCP] section 11
to deprecate delegated prefixes (set preferred time to zero) if the
router has information about path loss to the carrier that sourced
the prefix. [HNCP] is the only standard track document that requests
to propagate the prefix unavailability information to the host.
[Node Requirements] does not make DHCP mandatory for address
assignment. Many popular client operating systems do not support
DHCP even for other configurations, despite the fact that they
"should", according to [Node Requirements]. Hence, for the general
case, we should assume that address assignment and configuration is
done with [SLAAC]. Note that the logic below would not change if
DHCP were to be used for address assignment instead.
As a result, we have a host with many IP addresses that needs to
choose the source address and the next hop before any communication
attempts it may wish to make.
Potentially, the application on the host may use bind() to choose
the source address first using any logic pre-programmed into the
application. This application-driven solution is the only current
way to access the "subscriber-only services" of the particular
carrier, and for steering traffic based on cost, latency, packet
loss, hop count, etc.
After the application selects the proper source address with bind(),
it needs to choose the next hop to serve it.
There are two options to accomplish this:
1) [Multi-Homing] section 3.2 has an augmentation to [ND] section
6.3.6 asking to prefer default routers that advertised the
particular prefix already used for the source address.
2) [Route Preferences] to install the external prefix into the
host's routing table.
It may be valuable to implement [Provisioning domains] to supply
along with DNS resolvers the relationship of such resolvers to each
prefix provided by the different carriers, as unrelated resolvers
may respond with unusable or missing information when queried for
"subscriber-only services". In any case, IP stacks of host OSes are,
in practice, not capable of accepting and using this additional
information - so it would not play any role in the decision-making
process.
FBNVV Expires September 3, 2023 [Page 14]
Internet-Draft site multihoming March 2023
The implementer should check that the functionality mentioned in
this paragraph is supported on routers and applications before
relying on it in designs. Unfortunately, [Provisioning domains] and
[Multi-Homing] have low acceptance on the market, and hence the
application-driven solution for carrier resiliency has a low
probability of having been implemented, and at best would be
inconsistent.
Let us now consider the more typical case in which the application
is simplified in its networking aspects. In this case, the
application would use getaddrinfo(), which is typically compliant
with [SASA] on all OSes. This solution is referred to as "host-
driven" within this section.
For historical reasons, getaddrinfo() selects the next hop first. By
default, the next hop is chosen in a random round-robin manner
between all available routers on the particular link (in accordance
with [ND] section 6.3.6).
Rule 5.5 of [SASA] section 5 will then prefer, in the list of
available source addresses, those inside prefixes that are already
advertised by the chosen next hop. Hence, the random next hop leads
in turn to a random source address choice among those available.
Such behavior may block the possibility of accessing the
"subscriber-only services" of a particular carrier (traffic would be
filtered due to having a source address belonging to a different
carrier) as well as prevent traffic steering by any sort of policy.
Moreover, even if [Provisioning domains] were to be implemented, the
recommendation for random next hop choice would prevent effective
use of it.
Simple Internet connectivity with carrier resiliency could be
achieved in this manner, as carrier resiliency would work on a basic
level albeit with unpredictable traffic distribution between the
carriers. All that is needed for this is to support rule 5.5 of
[SASA] section 5.
Importantly, in addition to supporting rule 5.5 of [SASA], the
router should also support the L-13 requirement of [HNCP] to
deprecate prefixes individually and not the default router status
itself. This is the only way for convergence to take place
effectively in the case of connectivity loss to the PA carrier.
FBNVV Expires September 3, 2023 [Page 15]
Internet-Draft site multihoming March 2023
[MHMP] discusses that it is possible to achieve traffic steering
supplying policies by two mechanisms:
1) "Routing Information Options" of [Route Preferences] to
influence next hop choice
2) [Policy by DHCP] to modify policies in the [SASA] source
address selection algorithm.
The latter method is considered impractical because of its
complexity and general lack of DHCPv6 support in many commonly
deployed operating systems.
One of the drawbacks of the PA-based solution is that it can fail to
meet the requirements for realtime traffic steering based on
measured link quality and upper-layer information. While PA-based
traffic steering using the methods described above can be sufficient
in some environments, it must be kept in mind that the decision of
which link to exit from is ultimately left to the host in the form
of source address selection. In environments performing application-
based traffic steering, it is instead crucial that the steering
decision be made by the edge device, due to it having a policy table
based on network, transport, and application-layer aspects of
packets as well as realtime link quality metrics. This policy
represents the intent of the administrator and, ultimately, the
business requirements of the organization which owns the network; it
may for various reasons not discussed in this document have been
deemed unfeasible to replicate and apply this policy table directly
at the host level inside the clients. In the PA-based solution, the
host has access to prefixes from all available Internet links and
can assign itself routable addresses from them, and is thus free to
ignore any policy configured on the edge device. The gateway is in
fact forced to forward the packet in a way that honors the source
address chosen by the host, lest the packet be dropped upstream due
to implementations of [BCP38]. Once the traffic leaves the network
through the correct uplink, however, the return path is guaranteed
to be symmetric due to the address selected by the host being routed
only to one of the Internet links; this could be considered to
satisfy requirement 7 of section 4 with regards to the inbound
portion of traffic steering.
Finally, in the case where a site has many links and routers
(complex topology) then source routing or other connection tracking
mechanism in the internal network is mandatory to deliver the
traffic to the carrier owning the source address of the packet. This
aspect is properly discussed in [MHMP Enterprise], albeit it is not
yet of any use in the PA-based scenario because the delegated PA
FBNVV Expires September 3, 2023 [Page 16]
Internet-Draft site multihoming March 2023
prefix should since the beginning be dynamically split into smaller
prefixes and propagated to all links throughout the site, and there
is currently no accepted and available method to do this. Such a
method should also assist with prefix deprecation in the case where
connectivity to the carrier is lost. This is not supported at the
moment because neither [HNCP] nor DHCP-PD have gained acceptance for
this purpose. There is thus a chance that if this problem is
addressed then [MHMP Enterprise] might become very important for
sites with complex topologies.
Advantages:
- No need to own and operate a registered, Provider-Independent
address space,
- Preserves end-to-end communication,
- In simple networks, it may be as easy as plugging all CPE routers
and client devices into the same L2 domain (i.e. Switch).
Disadvantages:
- Scalability issues, easy for simple networks, but exponentially
more difficult in complex networks,
- Prefixes may not get deprecated when the CPE itself fails, as
opposed to just the link,
- Not all issues are resolved yet, only the simplest scenario is
possible (simple topology and unpredictable traffic distribution),
- Carriers may frequently change the prefix (flash renumbering), and
this could disrupt communication,
- Sites with complex topologies are not well supported yet,
- Traffic steering by any policies (including the capability to
access "subscriber-only services") is not supported yet.
5.3. ULA with NPT
[ULA] allows for the creation and use of local, non-globally
reachable and not centrally assigned address space that is
sufficiently random to be treated like globally-unique addressing
within a given organization or environment. However, [ULA] has
notable limits concerning the number of sites that one prefix may
span, and well documented usability limitations when considering
address selection details across large, diverse network
environments.
Organizations requiring larger than a /48 prefix are often better
served applying for and receiving a global PI allocation that is
right-sized for their needs. There may exist creative tweaks for
FBNVV Expires September 3, 2023 [Page 17]
Internet-Draft site multihoming March 2023
expanding ULA beyond its normal size, but that is outside the scope
of this document.
There is also one significant detail of [ULA] address space that is
important to note in the presence of a dual-stacked environment, as
[ULA] is prioritized below GUA and IPv4 address space on the hosts
according to [SASA] section 2.1. Hence, in a dual-stack environment,
it is necessary to modify the [SASA] policy table to insert the /48
prefix with higher precedence, as recommended in section 10.6 of
[SASA]. Automation for such configuration is OS-specific, and in
some cases may not be possible.
It is not mandatory to have [ULA] for the solution described in this
section, registered GUA (PI addresses in particular) may be used
too, but this has a low chance of happening as PI address space is
much more widely deployed as a routed policy directly to carriers or
other upstream service providers.
Network Prefix translation [NPT] is the unique IPv6 technology that
enables a lightweight version of NAT with a 1:1 stateless
relationship between addresses on the "inside" and "outside". A
stateless algorithmic relationship permits to have asymmetric
routing and easy redundancy if multiple gateways are implemented
facing a single carrier.
Like any NAT it may create a challenge for protocols that embed IP
addresses at the application level. It may require the usage of an
external [STUN] server for address translation, or monitoring of the
session by an ALG.
Additionally, when crossing NAT environments, protocols such as
IPSec require or fallback to "NAT traversal" schemes, which
typically work by encapsulating the original session into UDP.
Application support for this with IPv6 is often poor because a main
talking point for the adoption of IPv6 is that "NAT traversal" is no
longer required and that this simplifies the application logic.
However, in practice, this is not always the case and legacy
applications may still require these techniques to operate.
Contrary to other forms of NAT, with NPT there is no need to
generate or retain translation logs because translation is stateless
and deterministic.
Two-thirds of carriers lease a permanent prefix to subscribers, and
such prefix would thus remain the same after the uplink is
disconnected and re-connected. For this reason, it is possible to
FBNVV Expires September 3, 2023 [Page 18]
Internet-Draft site multihoming March 2023
initiate an inbound session from the "outside" in a stable and
continuous manner; the NPT engine is able to apply the required
translation on all outbound and inbound packets regardless of
whether they are part of a new or existing connection. For other
carriers which do not lease prefixes permanently, some additional
efforts are needed to dynamically update DNS records, and use those
to establish inbound connections. The possibility of connection
initiation in any direction (when firewall rules allow it) is
considered valuable by some engineers, while others value the one-
way connectivity typical of NAT that is lost with NPT.
Similarly, other NAT-related problems are not present with NPT
because it does not require manipulation of the transport layer.
[NPT] is partially acting against the [IAB request] to preserve the
end-to-end transparency of the Internet which is important for the
Internet's future flexibility.
Together, [ULA] and [NPT] may effectively mimic the typical IPv4
carrier resilience practices: the organization might only have [ULA]
inside its network (no GUA), and every site could have many
redundant connections through separate [NPT] engines on every border
gateway, making use of the actual PA space provided by each carrier
on the external side of the translation to enable global
reachability towards Internet destinations.
There is, however, a principal difference from the typical IPv4 NAT
solution in that [NPT] needs an equally-sized prefix on the "inside"
and "outside". While it is typically possible to get a /56 or /60
from the fixed broadband carrier, it is significantly less common to
be delegated more than a /64 by the mobile carrier; hence, if the
carrier is mobile then only a simplified site with one internal /64
subnet is feasible.
For faster convergence, routers should announce a "default" on-site
and then additionally some more specific prefixes if needed.
Additional destination prefixes facilitate the distribution of
traffic between the carriers.
In cases where internal routing hops are present between the edge
devices, such announcements likely require other protocols in
addition to Neighbor Discovery.
As for the network interface of the internal host (the last hop),
such an announcement comes in the form of default router status in
[ND], and [Route Preferences] for the more specific routes. The last
FBNVV Expires September 3, 2023 [Page 19]
Internet-Draft site multihoming March 2023
one in particular is essential to be able to access "subscriber-only
services".
The crucial requirement remains that routers must continuously track
connectivity to carriers to be able to deprecate the "default" as
well as any more specific announcements when such connectivity is
lost.
Advantages:
- No need for official address space, the ULA prefix is pseudo-
randomly self-generated,
- Easy to implement, similar in practice to current IPv4 carrier
resiliency techniques,
- Potential for traffic distribution policy between different
carriers.
Disadvantages:
- It is challenging to automate ULA prioritization above IPv4 on
hosts,
- NPT breaks some applications with address referrals at the
application level, some additional solutions are needed (STUN,
ALG),
- Custom distribution policies are needed for access to filtered
resources ("subscriber-only services"),
- Session initiation from the outside is practical only in cases
where the carrier prefix is stable or DNS records are dynamically
updated,
- Currently limited to one subnet per site in mobile environments,
- May hinder overall IPv6 adoption as IPv6 with NPT loses the end-
to-end connectivity advantage,
- For applications, the drawbacks are similar to ULA with NAT66
(section 5.4).
5.4. ULA with NAT66
The observations regarding the use of ULA from the first paragraph
of the previous section apply to this section as well. It is
important be mindful of the requirement and effort necessary to
prioritize ULA above IPv4 in the [SASA] policy table of hosts.
It is not mandatory to have [ULA] for the solution described in this
section, alternatively, registered GUA (PI addresses in particular)
may be used, but this brings its own set of requirements and effort,
as PI address space is much more useful when routed directly to
FBNVV Expires September 3, 2023 [Page 20]
Internet-Draft site multihoming March 2023
carriers. However, not all carriers may support this, or they might
charge significantly more to do so.
The [IAB request] to avoid the use of NAT has been heavily based on
the long list of [NAT implications]. In the case of stateful NAT66,
the full list of problems caused by NAT is applicable: breaking of
the end-to-end model, impossibility of initiating sessions from the
outside, breaking of application-layer referrals to the addresses,
single point of failure, redundancy challenges (state replication),
and performance challenges (stateful processing).
Hence, it is easy to understand the IETF consensus for not having a
stateful NAT standard for IPv6. There is no RFC for NAT66. A
proprietary implementation may furthermore create interoperability
challenges.
Those details aside, NAT has operational advantages: it avoids
renumbering in case of PA address space change, and is historically
the most common solution for carrier resiliency, especially in small
to medium sized environments that may not have the resources,
availability, or expertise to leverage a PI based solution with
upstream carriers. Hence, it is supported by many products, both
commercial and open source; one example is [nftables NAT66].
Another issue with NAT is the lack of UPnPv6 standardization and
implementation. With IPv4, applications behind a NAT44 can
dynamically request to know their public IP as well as new port
forwardings, thanks to UPnP. With IPv6 this is not available, and
the end-user experience with NAT66 is thus likely to be worse than
with NAT44.
For faster convergence, routers should announce a "default" on-site
and then additionally some more specific prefixes if needed.
Additional destination prefixes facilitate the distribution of
traffic between the carriers.
In cases where internal routing hops are present between the edge
devices, such announcements likely require other protocols in
addition to Neighbor Discovery.
As for the network interface of the internal host (the last hop),
such an announcement comes in the form of default router status in
[ND], and [Route Preferences] for the more specific routes. The last
one in particular is essential to be able to access "subscriber-only
services".
FBNVV Expires September 3, 2023 [Page 21]
Internet-Draft site multihoming March 2023
The crucial requirement remains that routers must continuously track
connectivity to carriers to be able to deprecate the "default" as
well as any more specific announcements when such connectivity is
lost.
Advantages:
- No need for official address space, as the ULA prefix is pseudo-
randomly self-generated,
- Easy to implement,
- Equivalent in practice to current IPv4 carrier resiliency
techniques,
- NAT may be a normative requirement in itself (this is highly
questionable, but nonetheless brought forward in many
discussions),
- Support for sites with complex topologies,
- Potential for traffic distribution policy between different
carriers.
Disadvantages:
- It is challenging to automate ULA prioritization above IPv4 on
hosts,
- NAT breaks some applications with address referrals at the
application level,
- Custom distribution policies needed for access to filtered
resources ("subscriber-only services"),
- Some additional solutions are needed (STUN, ALG),
- Session initiation from the outside is blocked in practice (needs
complex configuration),
- NAT implies the requirement to keep logs for compliance and
troubleshooting,
- Expensive, due to the higher costs of stateful processing,
- May hinder overall IPv6 adoption,
- For applications, the drawbacks are similar to ULA with NPT
(section 5.3).
5.5. Shifting the problem to the centralized site
Another possible approach is shifting the Internet access resiliency
problem to a central site when the branch has a redundant, private
WAN connectivity provisioned through any or multiple available
methods (SDH/PDH/OTN links, MPLS VPNs, customer-managed overlays).
In this scenario, the backhaul towards the central site (which then
performs the ultimate handoff to the Internet) happens over
physically or virtually dedicated links, and the actual addressing
FBNVV Expires September 3, 2023 [Page 22]
Internet-Draft site multihoming March 2023
solution offered by the carrier serving the branch becomes
irrelevant; in the extreme case, in fact, the branch carrier does
not provide an IP service at all (as in the case of optical
transport or a layer-2 VPN) or the provided addressing is only used
to establish tunnels (in the case of overlays). It is not uncommon
for the branch edge device to not even have a default route
configured towards any of the carrier next hops, instead configuring
a default route through the overlay or having as next hop a loopback
interface address reachable through the overlay itself.
The reasoning behind this choice is based on several factors and
commonly involves one or more of the following assumptions:
1. Branch site Multi-Homing is mainly a matter of first mile
redundancy due to the increased difficulty of providing stable
connectivity to remote sites compared to a large central site.
Suppose the traffic can be made to traverse the first mile in an
optimal environment (because the entire path is under the network
administrator's control, at least at the network level). In that
case, the relatively high-quality Internet circuits found at the
central site can be managed using more traditional and resource-
intensive techniques (for example, by significantly increasing
capacity and carrier diversity, tuning routing advertisements, and
using ECMP).
2. Obtaining enterprise-class connectivity, where the customer has
the option to announce their own address space dynamically to the
carriers, can be complex and cannot thus be done for every site.
3. Managing a geographically-distributed Internet breakout may pose
greater operational, financial, and security-related challenges when
the proper orchestration tools are not employed.
It is typically much easier to arrange the resiliency over internal
WAN links. This is primarily because the addressing structure and
the path selection are under the control of a single entity
throughout the LAN and the private WAN. This can allow, for example,
to number the entire internetwork using a single type of addressing,
recreating the benefits of the solution in section 5.1 while
avoiding some of its disadvantages, namely:
- "In case of provider failure, a prefix may not be correctly
deprecated[...]": in the tunnel-based solution, all network devices
are under the control of the same entity experiencing the
FBNVV Expires September 3, 2023 [Page 23]
Internet-Draft site multihoming March 2023
hypothetical issue, simplifying troubleshooting and speeding up
resolution.
- "Need carriers to accept PI prefix advertisements": in the tunnel-
based solution, this needs to be done only once, at the central
site, instead of at every site implementing the solution.
- "Carriers typically charge significantly more for such type of
attachment[...]": see the previous observation.
- "Return traffic path not guaranteed to mirror the outbound path":
path symmetry in the more critical first mile can be guaranteed by
the network administrator through configuring the network devices at
the far end of the WAN to honor the original link choice of the
incoming sessions in a stateful manner, or by applying the same
deterministic forwarding algorithms on devices at both ends. Several
vendors provide this functionality out of the box in certain
products. Symmetry is thus guaranteed where it matters, i.e., where
traffic must traverse links having vastly different characteristics
and quality (it is not uncommon for remote sites to be served by a
primary DSL/fiber link and a secondary, much more limited cellular
link).
The tunnel-based solution described in this section may also be
implemented as a scheme in which the central site is not owned by
the organization at all and is instead part of a service offered by
a tunnel broker somewhere on the Internet. Such a choice can be
appealing due to factors such as outsourcing of operational burden
and the possibility of superior performance due to the broker having
a globally distributed and fine-tuned network of "hubs," to the
closest of which each site can then connect to.
Another advantage of having control over the path crossing the first
mile of the branch site lies in the possibility of applying error-
correcting algorithms to the traffic; several vendors offer this
functionality which, although proprietary, can be made to work by
placing compatible devices at the branch edge and the central site,
usually terminating the tunnels comprising the overlay. Such
techniques typically include forward error correction, compression,
packet duplication, and deduplication across multiple low-quality
links, with the goal to prevent or lessen packet loss across the
overlay. These techniques, however, cannot improve other metrics
such as latency.
FBNVV Expires September 3, 2023 [Page 24]
Internet-Draft site multihoming March 2023
The principal downside of the tunnel-based solution is not making
use of the "local Internet breakout": users at the branch site are
almost guaranteed to experience worse performance towards Internet
destinations compared to solutions listed in sections 5.1-5.4 due to
the traffic having to be backhauled to the central site first.
However, it should be noted that as long as the centralized site
uses a solution similar to 5.1 or 5.2, it'll also enable
communication that otherwise would have failed entirely or required
complex use-case-specific workarounds.
In fact, despite the alleviating factors discussed above, shifting
the problem to a different area of the network might not be
considered a technical solution at all because the central site
would face the same fundamental challenges, and it would ultimately
have the same options for multi-homing as discussed in sections 5.1-
5.4. As such, what is described in this section could be considered
a non-technical solution for a small site.
Enterprise WAN design in itself remains outside the scope of this
document.
Advantages:
- Shifting the problem to a different location may help solve it
more efficiently,
- Traffic steering is easy to implement, including traffic symmetry
requirements,
- A centralized Internet gateway simplifies perimeter security,
- Possibility of applying WAN optimization techniques to the first
section of the path toward the Internet.
Disadvantages:
- Looping the Internet traffic through the centralized site might
increase latency, and additional links on the traffic path may
contribute to jitter and packet loss,
- More bandwidth is needed for rented WAN links,
- Side effects related to tunneling, such as encapsulation
processing and overhead,
- Convergence time in case of underlay network failures may be
affected by the need to re-establish the tunnels and routing
neighborships of the overlay,
- The central site becomes a single point of failure for the
Internet access of the entire organization,
FBNVV Expires September 3, 2023 [Page 25]
Internet-Draft site multihoming March 2023
- Some or all of the disadvantages listed in sections 5.1-5.4 apply,
depending on the specific solution selected to solve the multi-
homing issue at the central site. These may include end-to-end
connectivity and traffic steering issues toward Internet
destinations.
6. Conclusion
Not all requirements can be satisfied by every solution:
+--+--------------------------------+----+-----+---------+---------+
| | Requirement | PI | PA | ULA+NPT | ULA+NAT |
+--+--------------------------------+----+-----+---------+---------+
| 1| Carriers Resiliency | + | + | + | + |
| 2| End-to-End Connectivity | + | + | +/- | - |
| 3| Internal Connectivity | + | + | +/- | +/- |
| 4| Convergence speed | + | +/- | + | + |
| 5| Complex Topology support | + | - | +/- | + |
| 6| Subscriber-only services | - | - | +/- | +/- |
| 7| Traffic Steering on Router | +/-| - | + | + |
| 7| Traffic Steering on Host OS | - | - | - | - |
| 7| Traffic Steering on Application| - | - | - | - |
+--+--------------------------------+----+-----+---------+---------+
The table above shows partial support for end-to-end connectivity
for the ULA+NPT solution because, while it does allow initiating
connectivity in any direction, employing address references at the
application layer requires extra steps, for example in the form of
an ALG or [STUN].
Internal connectivity is marked as partial support in the ULA
solutions due to the complexity involved in promoting the ULA
address space above IPv4 in the [SASA] policy table of hosts.
FBNVV Expires September 3, 2023 [Page 26]
Internet-Draft site multihoming March 2023
Convergence speed is partially satisfied by the PA-based solution
because [HNCP] or DHCP-PD have not been adopted by the market, and
they would be needed for prefix deprecation propagation over a
complex site.
Complex topology support is marked as partially satisfied by the
ULA+NPT solution because it is not possible in practice to get an
external prefix larger than /64 from mobile carriers.
Support for "subscriber-only services" is marked as partial in the
ULA solutions because it needs a routing announcement as a "Routing
Information Option" of [Route Preferences], which is not widely
supported.
Traffic steering on routers is marked as partially supported for the
PI-based solution because of the high complexity involved in
organizing the steering of incoming traffic. NAT/NPT-based solutions
connect ingress traffic steering to egress which makes them simpler
in this regard.
Some of the functionality reflected in the table above may be
improved in the future, but a roadmap (active IETF draft) is not
available at the time of writing.
Theoretically, from a purely technical point of view, the solutions
in section 5 are ordered by the number of requirements satisfied,
from most to least: PI is the best, PA-based is more complex, NPT
breaks the end-to-end Internet model, and so on. In the real world,
though, the company could have non-technical requirements that
override the technical ones. For example, an organization might find
that the tunnel broker solution described in 5.5 fits their use-case
best, even though it doesn't really solve the issue so much as
outsource it to a different organization (the tunnel-based solution
in 5.5 cannot be properly evaluated in the requirements table due to
it technically not being a solution, and anything added to the table
would merely reflect the choice of multi-homing techniques at the
central site). The table above is, in this sense, not complete - it
should be enriched with non-technical requirements as perceived by
the network owner.
For many network owners, the main deciding factor may be the desire
to have end-to-end connectivity, as it is the most notable advantage
of deploying IPv6 compared to IPv4. This may be especially important
when managing resources that need to be exposed to the Internet.
FBNVV Expires September 3, 2023 [Page 27]
Internet-Draft site multihoming March 2023
If this is needed then PI or PA-based solutions are applicable, and
network owners must undertake some additional steps to implement
them:
. obtain a PI prefix from a RIR or LIR,
. pay a premium for the advertisement of the PI prefixes through
the carriers (as these often impose different tariffs for such
an attachment circuit).
If both of these challenges are deemed acceptable then the PI-based
solution is preferable: simple, reliable, and scalable. The
universal adoption of PI by companies of all sizes would create a
burden for Internet routing, but this issue is unlikely to be
considered a priority by an organization whose decision-making
process is already constrained by many other factors.
If at least one of the aforementioned challenges is not acceptable
then PA may be the solution of choice, even with all the
restrictions and complexities discussed in section 5.2. A notable
exception to this is if complex topology support is a requirement
and the site is served by a mobile carrier (due to the
unavailability of prefixes larger than /64 on such connections).
It may also be the case that end-to-end connectivity is not a
necessity, and may even be undesired. Here, the ULA+NPT solution
(discussed in section 5.3) satisfies a greater amount of
requirements in the majority of situations, apart from cases where
NAT66 is a strict (non-technical) requirement or the site has a
combination of complex topology and mobile connectivity (problematic
due to the small assignments on the WAN side and the 1:1 mechanism
of NPT.)
For sites having a complex topology (many links and routers), a PA-
based solution is not an option yet, because it would need automatic
PA address distribution over the site and neither [HNCP] nor DHCP-PD
have gained market acceptance for this task.
The logical steps in the design process would then be like the ones
above, but after having evaluated the PI-based solution the next
option would be ULA+NPT.
FBNVV Expires September 3, 2023 [Page 28]
Internet-Draft site multihoming March 2023
Previous availability of PI space, or perceived NAT66 regulatory
requirements might also be primary factors, and then the logic may
yet again be different.
For further use cases that have not been discussed in this document,
it is possible to get a general expectation of compatibility using
the table above.
It is however recommended to read section 5 to fine-tune the custom
requirements matrix and grade each solution accordingly.
7. Security Considerations
This document is informational.
Hence, it may not create additional security challenges.
8. IANA Considerations
This document has no request to IANA.
9. References
9.1. Normative References
[BCP38] P. Ferguson, D. Senie, "Network Ingress Filtering: Defeating
Denial of Service Attacks which employ IP Source Address
Spoofing", BCP 38, RFC 2827, DOI 10.17487/RFC2827, May
2000, <https://www.rfc-editor.org/info/rfc2827>.
[IPv6] S. Deering, R. Hinden, "Internet Protocol, Version 6 (IPv6)
Specification", RFC 8200, DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
[ND] T. Narten, E. Nordmark, W. Simpson, H. Soliman, "Neighbor
Discovery for IP version 6 (IPv6)", RFC 4861, DOI
10.17487/RFC4861, September 2007, <https://www.rfc-
editor.org/info/rfc4861>.
[SLAAC] S. Thomson, T. Narten, T. Jinmei, "IPv6 Stateless Address
Autoconfiguration", RFC 4862, DOI 10.17487/RFC4862,
September 2007, <https://www.rfc-editor.org/info/rfc4862>.
[NAT Implications] T. Hain, "Architectural Implications of NAT", RFC
2993, DOI 10.17487/RFC2993, November 2000,
<https://www.rfc-editor.org/info/rfc2993>.
FBNVV Expires September 3, 2023 [Page 29]
Internet-Draft site multihoming March 2023
[Local Protection] G. Van de Velde, T. Hain, R. Droms, B. Carpenter,
E. Klein, "Local Network Protection for IPv6", RFC 4864,
DOI 10.17487/RFC4864, May 2007, <https://www.rfc-
editor.org/info/rfc4864>.
[IAB request] D. Thaler, L. Zhang, G. Lebovitz, "IAB Thoughts on
IPv6 Network Address Translation", RFC 5902, DOI
10.17487/RFC5902, July 2010, <https://www.rfc-
editor.org/info/rfc5902>.
[NPT] M. Wasserman, F. Baker, "IPv6-to-IPv6 Network Prefix
Translation", RFC 6296, DOI 10.17487/RFC6296, June 2011,
<https://www.rfc-editor.org/info/rfc6296>.
[ULA] R. Hinden, B. Haberman, "Unique Local IPv6 Unicast Addresses",
RFC 4193, DOI 10.17487/RFC4193, October 2005,
<https://www.rfc-editor.org/info/rfc4193>.
[SASA] D. Thaler, R. Draves, A. Matsumoto, T. Chown, "Default
Address Selection for Internet Protocol Version 6 (IPv6)",
RFC 6724, DOI 10.17487/RFC6724, September 2012,
<https://www.rfc-editor.org/info/rfc6724>.
[HNCP] M. Stenberg, S. Barth, P. Pfister, "Home Networking Control
Protocol", RFC 7788, DOI 10.17487/RFC7788, April 2016,
<https://www.rfc-editor.org/info/rfc7788>.
[Node Requirements] T. Chown, J. Loughney, T. Winters, "IPv6 Node
Requirements", BCP 220, RFC 8504, DOI 10.17487/RFC8504,
January 2019, <https://www.rfc-editor.org/info/rfc8504>.
[MHMP Enterprise] F. Baker, C. Bowers, J. Linkova, "Enterprise
Multihoming Using Provider-Assigned IPv6 Addresses without
Network Prefix Translation: Requirements and Solutions",
RFC 8678 DOI 10.17487/RFC8678, December 2019,
<https://www.rfc-editor.org/info/rfc8678>.
[Multi-Homing] F. Baker, B. Carpenter, "First-Hop Router Selection
by Hosts in a Multi-Prefix Network", RFC 8028, DOI
10.17487/RFC8028, November 2016, <https://www.rfc-
editor.org/info/rfc8028>.
FBNVV Expires September 3, 2023 [Page 30]
Internet-Draft site multihoming March 2023
9.2. Informative References
[IAB report] D. Meyer, L. Zhang, K. Fall, "Report from the IAB
Workshop on Routing and Addressing", RFC 4984, DOI
10.17487/RFC4984, September 2007, <https://www.rfc-
editor.org/info/rfc4984>.
[Shim6] E. Nordmark, M. Bagnulo, "Shim6: Level 3 Multihoming Shim
Protocol for IPv6", RFC 5533, DOI 10.17487/RFC5533, June
2009, <https://www.rfc-editor.org/info/rfc5533>.
[STUN] M. Petit-Huguenin, G. Salgueiro, J. Rosenberg, D. Wing, R.
Mahy, P. Matthews, "Session Traversal Utilities for NAT
(STUN)", RFC 8489, DOI 10.17487/RFC8489, February 2020,
<https://www.rfc-editor.org/info/rfc8489>.
[MHMP] O. Troan, D. Miles, S. Matsushima, T. Okimoto, D. Wing, "IPv6
Multihoming without Network Address Translation", RFC
7157, DOI 10.17487/RFC7157, March 2014, <https://www.rfc-
editor.org/info/rfc7157>.
[Route Preferences] R. Draves, D. Thaler, "Default Router
Preferences and More-Specific Routes", RFC 4191, DOI
10.17487/RFC4191, November 2005, <https://www.rfc-
editor.org/info/rfc4191>.
[Policy by DHCP] A. Matsumoto, T. Fujisaki, T. Chown, "Distributing
Address Selection Policy Using DHCPv6", RFC 7078 DOI
10.17487/RFC7078, January 2014, <https://www.rfc-
editor.org/info/rfc7078>.
[DNS Options] J. Jeong, S. Park, L. Beloeil, S. Madanapalli, "IPv6
Router Advertisement Options for DNS Configuration", RFC
8106 DOI 10.17487/RFC8106, March 2017, <https://www.rfc-
editor.org/info/rfc8106>.
[Conditional PIO] J. Linkova, M. Stucchi, "Using Conditional Router
Advertisements for Enterprise Multihoming", RFC 8475 DOI
10.17487/RFC8475, October 2018, <https://www.rfc-
editor.org/info/rfc8475>.
[Provisioning domains] P. Pfister, E. Vyncke, T. Pauly, D. Schinazi,
W. Shao, "Discovering Provisioning Domain Names and Data",
RFC 8801 DOI 10.17487/RFC8801, July 2020,
<https://www.rfc-editor.org/info/rfc8801>.
FBNVV Expires September 3, 2023 [Page 31]
Internet-Draft site multihoming March 2023
[RFC8415] T. Mrugalski, M. Siodelski, B. Volz, A. Yourtchenko, M.
Richardson, S. Jiang, T. Lemon, T. Winters, " Dynamic Host
Configuration Protocol for IPv6 (DHCPv6)", RFC 8415 DOI
10.17487/RFC8415, November 2018, <https://www.rfc-
editor.org/info/rfc8415>.
[Temporary addressing] F. Gont, S. Krishnan, T. Narten, R. Draves,
"Temporary Address Extensions for Stateless Address
Autoconfiguration in IPv6", RFC 8981 DOI 10.17487/RFC8981,
February 2021, <https://www.rfc-editor.org/info/rfc8981>.
[nftables NAT66] Marco Cilloni, "NAT66: The good, the bad, the
ugly", <https://blog.apnic.net/2018/02/02/nat66-good-bad-
ugly>.
10. Acknowledgments
Thanks to v6ops working group for problem discussion
Authors' Addresses
Klaus Frank
Email: klaus.frank@posteo.de
Nick Buraglio
Energy Sciences Network
Email: buraglio@forwardingplane.net
Paolo Nero
Email: oselists@gmail.com
Paolo Volpato
Huawei Technologies
Email: paolo.volpato@huawei.com
Eduard Vasilenko
Huawei Technologies
Email: vasilenko.eduard@huawei.com
FBNVV Expires September 3, 2023 [Page 32]