Adaptive Load Balancing for Equal-Cost Multipath (ECMP)
draft-yang-rtgwg-ecmp-adaptive-load-balance-00
This document is an Internet-Draft (I-D).
Anyone may submit an I-D to the IETF.
This I-D is not endorsed by the IETF and has no formal standing in the
IETF standards process.
| Document | Type | Active Internet-Draft (individual) | |
|---|---|---|---|
| Authors | Jin Yang , Yuchi Tian , Weiqiang Cheng , Tingting Han , Junjie Wang , Guoying Zhang | ||
| Last updated | 2026-11-01 | ||
| RFC stream | (None) | ||
| Intended RFC status | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-yang-rtgwg-ecmp-adaptive-load-balance-00
RTGWG J. Yang
Internet-Draft Y. Tian
Intended status: Standards Track W. Cheng
Expires: 3 May 2027 T. Han
China Mobile
J. Wang
G. Zhang
Centec
30 October 2026
Adaptive Load Balancing for Equal-Cost Multipath (ECMP)
draft-yang-rtgwg-ecmp-adaptive-load-balance-00
Abstract
Equal-Cost Multipath (ECMP) forwarding assigns a flow to a next-hop
by computing a hash over packet header fields when the flow is first
seen. Because the assignment does not change after that, the load
across the paths of an ECMP group can become uneven when the set of
paths changes or when traffic shifts among existing flows. This
document defines procedures for adaptive ECMP load balancing in which
a forwarding node periodically measures per-path load, determines
whether the load is sufficiently uneven to act on, and makes a
bounded number of incremental flow placement changes toward less-used
paths. The procedures preserve per-flow packet ordering for all but
the small number of flows that are moved in any interval, and they
converge toward a balanced distribution across the group. The
procedures operate within a single forwarding node and add no new
packet formats or inter-node signaling.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 3 May 2027.
Yang, et al. Expires 3 May 2027 [Page 1]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
2. Problem Statement . . . . . . . . . . . . . . . . . . . . . . 4
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Design Objectives . . . . . . . . . . . . . . . . . . . . . . 5
5. Load Measurement . . . . . . . . . . . . . . . . . . . . . . 6
5.1. Load Metrics . . . . . . . . . . . . . . . . . . . . . . 7
5.2. Imbalance Assessment . . . . . . . . . . . . . . . . . . 7
5.3. Reassignment Limit Derivation . . . . . . . . . . . . . . 8
6. Flow Placement Adjustment . . . . . . . . . . . . . . . . . . 8
6.1. Eligible Events . . . . . . . . . . . . . . . . . . . . . 8
6.2. Target Path Selection . . . . . . . . . . . . . . . . . . 9
6.3. Rate of Path Changes . . . . . . . . . . . . . . . . . . 9
7. Interaction with Existing Forwarding Behavior . . . . . . . . 10
7.1. Coexistence with Hash-Based Path Selection . . . . . . . 10
7.2. Applicability to Link Aggregation Groups . . . . . . . . 10
7.3. MPLS Environments . . . . . . . . . . . . . . . . . . . . 10
8. Relationship to Other Work . . . . . . . . . . . . . . . . . 11
9. Applicability Considerations . . . . . . . . . . . . . . . . 11
9.1. Data Center Fabrics . . . . . . . . . . . . . . . . . . . 11
9.2. Wide Area Networks . . . . . . . . . . . . . . . . . . . 11
10. Operational Considerations . . . . . . . . . . . . . . . . . 11
10.1. Configuration Parameters . . . . . . . . . . . . . . . . 12
10.2. Flow Exclusion Policies . . . . . . . . . . . . . . . . 12
10.3. Monitoring . . . . . . . . . . . . . . . . . . . . . . . 13
10.4. Deployment Guidance . . . . . . . . . . . . . . . . . . 13
11. Conformance and Interoperability . . . . . . . . . . . . . . 14
12. Security Considerations . . . . . . . . . . . . . . . . . . . 15
13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
14. References . . . . . . . . . . . . . . . . . . . . . . . . . 15
14.1. Normative References . . . . . . . . . . . . . . . . . . 15
14.2. Informative References . . . . . . . . . . . . . . . . . 15
Yang, et al. Expires 3 May 2027 [Page 2]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction
Equal-Cost Multipath (ECMP) routing [RFC2991] lets a forwarding node
distribute IP traffic over several next-hops of equal routing cost.
The forwarding decision is commonly made by hashing selected packet
header fields and mapping each flow to one path. [RFC2992] analyzes
the hash-threshold algorithm that is widely used for this purpose and
discusses the trade-off between distribution uniformity and the
disruption that occurs when paths are added or removed.
Hash-based path selection gives a reasonable initial distribution and
preserves per-flow packet ordering, but the assignment is fixed: once
a flow is mapped to a path it stays there regardless of later changes
to the group or to traffic. As a result the load across the paths of
a group can become uneven over time. This is a known operational
effect.
[RFC7424] describes techniques for identifying large ("elephant")
flows and selectively moving them across LAG or ECMP component links
to improve utilization. That document is Informational, addresses
primarily the classification of large flows, and does not define
normative requirements for general load-aware path selection.
This document defines normative procedures for load-aware ECMP path
selection that apply to all flows in an ECMP group, not only to large
flows. The procedures rest on three elements: periodic measurement
of per-path load, a bounded number of flow placement changes per
interval to limit the rate of path changes, and operator-controlled
parameters that govern the behavior. The procedures supplement, and
do not replace, the existing hash-based path selection: they add a
load-aware feedback loop on top of the hash function. The procedures
carry MUST-level requirements on forwarding-node behavior and are
therefore on the Standards Track.
The relationship of these procedures to entropy-improving mechanisms
and to other load-balancing work is described in Section 8.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Yang, et al. Expires 3 May 2027 [Page 3]
Internet-Draft Adaptive ECMP Load Balancing October 2026
2. Problem Statement
Hash-based ECMP next-hop selection gives a one-time, deterministic
mapping from a flow identifier to a path. The mapping does not
account for runtime conditions, which leads to uneven load in at
least the following cases.
Path Addition
When a new equal-cost next-hop becomes available, for example when
a link comes up or a route is installed, only newly arriving flows
are candidates for the new path. Existing flows stay on their
assigned paths. The new path can remain under-used until enough
flows turn over naturally, which takes an unpredictable time that
depends on flow lifetimes.
Asymmetric Flow Departure
When flows end unevenly across paths, for example when several
long-lived flows on one path end while flows on other paths
continue, the load becomes uneven. The capacity freed on the path
with departing flows is not taken up by flows on the more heavily
used paths.
Traffic Rate Variation
Even when the set of active flows is stable, changes in per-flow
rate, for example a flow entering a bulk-transfer phase, can make
the load on one path diverge from the others. A static hash
assignment does not respond to such rate changes.
In each case the forwarding node has enough local information, in the
form of per-path counters and flow-table state, to detect the
imbalance, but a hash-only ECMP implementation does not act on it.
This document specifies how a forwarding node uses locally available
load data to make incremental changes to flow placement.
3. Terminology
This document uses the following terms. Where applicable, the
definitions of [RFC2991] and [RFC7424] are reused.
ECMP Group
A set of equal-cost next-hops toward the same destination prefix,
among which traffic is forwarded. A Link Aggregation Group (LAG)
is treated as the logical equivalent of an ECMP group in this
document.
Path (Component Link)
An individual next-hop within an ECMP group. The term "component
link" is used interchangeably, consistent with [RFC7424].
Yang, et al. Expires 3 May 2027 [Page 4]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Flow
A sequence of packets that share common header fields and are
treated as a unit for path selection. The flow granularity, for
example a 5-tuple or a 3-tuple, is set by the hashing policy and
is outside the scope of this document.
Load Metric
A quantitative measure of the traffic carried by a path over a
measurement window, for example a forwarded byte count, a
forwarded packet count, or a count of active flows.
Imbalance Ratio (IR)
A dimensionless measure of the degree of load skew across the
paths of an ECMP group, defined in Section 5.2.
Reassignment Limit
An upper bound on the number of flow placement changes that may be
made within one evaluation interval. It is derived from the
observed imbalance and controls the rate of change.
Evaluation Interval
The periodic window at which the forwarding node collects load
metrics, assesses imbalance, and determines whether placement
changes are warranted.
4. Design Objectives
The procedures defined in this document follow these objectives.
Minimal Disruption
The number of flows whose paths change in any interval MUST be
bounded. Unbounded path changes would cause widespread packet
reordering and degrade transport performance [RFC2992].
Proportional Response
The number of flow placement changes SHOULD be proportional to the
severity of the imbalance. When the load is within bounds, no
changes are made; when the imbalance is large, more changes are
permitted per interval to converge faster.
Incremental Convergence
Load balancing SHOULD converge over successive intervals rather
than attempting to reach a balanced state in a single step.
Incremental convergence avoids oscillation and gives stable
behavior.
Yang, et al. Expires 3 May 2027 [Page 5]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Operational Control
All parameters, including thresholds, limits, intervals, and
eligible flow classes, MUST be configurable by the operator. The
procedures MUST be able to be disabled on a per-group basis
without affecting other groups.
+---------------------+
| Evaluation Interval |
| Timer Expires |
+----------+----------+
|
v
+---------------------+
| Collect per-path |
| load metrics |
+----------+----------+
|
v
+---------------------+
| Compute Imbalance |
| Ratio (IR) |
+----------+----------+
|
+------+------+
| IR < thresh?|-----> No action; wait for
+------+------+ next interval
| no
v
+---------------------+
| Derive reassignment |
| limit from IR |
+----------+----------+
|
v
+---------------------+
| Adjust flow |
| placement (up to |
| limit) on eligible |
| events |
+---------------------+
Figure 1: High-Level Operation
5. Load Measurement
Yang, et al. Expires 3 May 2027 [Page 6]
Internet-Draft Adaptive ECMP Load Balancing October 2026
5.1. Load Metrics
A forwarding node that implements these procedures MUST maintain per-
path load metrics for each ECMP group on which adaptive load
balancing is enabled. The node MUST support at least one of the
following metric types.
* Forwarded byte count over the evaluation interval. This reflects
bandwidth use and is RECOMMENDED as the default.
* Forwarded packet count over the evaluation interval.
* Number of distinct active flows observed during the evaluation
interval.
A node MAY support a weighted combination of metrics or additional
implementation-defined metrics. The active metric type SHOULD be
configurable per group.
The evaluation interval MUST be configurable with a granularity of at
least 100 milliseconds. The default interval SHOULD be 1 second. An
interval shorter than 100 milliseconds is NOT RECOMMENDED because of
the processing cost of frequent measurement.
5.2. Imbalance Assessment
At each evaluation interval the forwarding node computes the
imbalance ratio (IR) across the N paths of the ECMP group. Let U_i
be the measured load metric for path i, where 1 <= i <= N.
U_max = max(U_1, U_2, ..., U_N)
U_min = min(U_1, U_2, ..., U_N)
U_avg = (sum of all U_i) / N
IR = (U_max - U_min) / U_avg, when U_avg > 0
IR = 0, when U_avg = 0
The IR is compared with a configurable imbalance threshold (T_imb).
When IR is below T_imb, the load is considered acceptable and no flow
placement change is made during the interval.
The default T_imb SHOULD be 0.3, which corresponds to a 30 percent
normalized deviation. An operator MAY change this value: a lower
threshold reacts to smaller imbalances at the cost of more frequent
changes, and a higher threshold reduces the frequency of changes at
the cost of tolerating more skew.
Yang, et al. Expires 3 May 2027 [Page 7]
Internet-Draft Adaptive ECMP Load Balancing October 2026
5.3. Reassignment Limit Derivation
When IR >= T_imb, the forwarding node derives a reassignment limit
(R_lim) that bounds the number of flow placement changes made during
the interval. The derivation of R_lim MUST satisfy the following
properties.
Non-decreasing
R_lim MUST be a non-decreasing function of IR, so that a larger
imbalance permits more changes per interval.
Bounded
R_lim MUST NOT exceed a configurable upper bound (R_max), which
prevents excessive changes under extreme conditions. The default
R_max SHOULD be proportional to the number of active flows in the
group, with a RECOMMENDED default of min(F/10, 16), where F is the
number of active flows. A node MAY use a fixed default, for
example 8, when the active flow count is not readily available.
Zero Below Threshold
When IR < T_imb, R_lim MUST be zero.
The function used to map IR to R_lim is an implementation choice.
Linear, piecewise-linear, and step functions are all acceptable,
provided the properties above hold.
The following is a non-normative example of a compliant function.
R_lim = 0, when IR < T_imb
R_lim = min(R_max, ceil(IR * S)), when IR >= T_imb
where S is a configurable scaling factor (default: 10).
This example is given for clarity only. An implementation may use
any function that satisfies the stated properties.
6. Flow Placement Adjustment
6.1. Eligible Events
Flow placement changes are made in response to specific forwarding-
plane events, subject to the reassignment limit for the interval.
Each change consumes one unit of the reassignment limit. The
following event types are defined.
Yang, et al. Expires 3 May 2027 [Page 8]
Internet-Draft Adaptive ECMP Load Balancing October 2026
New Flow Arrival
On the first packet of a previously unseen flow, the node MAY
assign the flow to the least-used path rather than to the path
given by the hash function. A node MUST support this event type.
Flow Expiration
When a flow entry ages out or is cleared, the resulting change in
per-path load MAY trigger the reassignment of another flow from a
more heavily used path to a less used path. Support for this
event type is RECOMMENDED.
Periodic Sweep
A background process MAY iterate over local flow state and select
flows on the most-used path for reassignment to the least-used
path. Support for this event type is OPTIONAL.
Operator-Initiated
A management action MAY request that the node make flow placement
changes, for example during a maintenance window. Support for
this event type is OPTIONAL.
The set of active event types SHOULD be configurable. When more than
one event type is enabled, any of them may trigger a change while the
reassignment limit for the interval has not been reached.
6.2. Target Path Selection
When a flow placement change is made, the target path MUST be the
path with the lowest current load metric in the group at the time of
the decision. When more than one path shares the lowest load, the
node MAY choose any of them, for example by a secondary hash or
round-robin among the candidates.
When a change is triggered by a periodic sweep or a flow expiration,
the source path from which a flow is selected MUST be the path with
the highest current load metric. The choice of which flow to move
from the source path is an implementation decision. A node SHOULD
prefer flows with lower instantaneous rate, to limit the effect of
reordering on any single flow.
6.3. Rate of Path Changes
The reassignment limit is the primary means of controlling the rate
of flow placement changes. Each change reduces the remaining limit
by one for the interval. When the remaining limit reaches zero, no
further changes are made until the next interval begins with a
freshly derived limit.
Yang, et al. Expires 3 May 2027 [Page 9]
Internet-Draft Adaptive ECMP Load Balancing October 2026
A node MUST NOT carry over unused reassignment capacity from one
interval to the next. The limit is derived anew at each interval
from the load observed for that interval. This prevents the
accumulation of unused capacity that could otherwise cause a burst of
path changes.
The combination of a bounded limit and periodic evaluation forms a
self-regulating feedback loop. Over successive intervals the per-
path load converges toward the threshold. Once IR falls below T_imb,
the limit becomes zero and changes stop until conditions change.
7. Interaction with Existing Forwarding Behavior
7.1. Coexistence with Hash-Based Path Selection
These procedures supplement, and do not replace, the existing hash-
based ECMP next-hop selection. The hash function remains the default
path selection method for all flows. A load-aware placement change
overrides the hash-based assignment only for a specific flow and only
when the derived reassignment limit permits it.
When adaptive load balancing is disabled, or when the reassignment
limit is zero, the node MUST use the standard hash-based path
selection for all flows.
7.2. Applicability to Link Aggregation Groups
A LAG selects a member link by a hash in a manner analogous to ECMP
next-hop selection. These procedures apply equally to LAG member
selection. When applied to a LAG, the references in this document to
"ECMP group" and "path" apply to the LAG and its component links
respectively.
A node that implements adaptive load balancing for both ECMP groups
and LAGs SHOULD manage them independently, since each has its own
membership, load, and parameters.
7.3. MPLS Environments
In MPLS environments, entropy labels [RFC6790] add input to the hash
and improve the uniformity of the initial flow placement. Even with
entropy labels, the fixed nature of the hash assignment means that
imbalance can develop over time, so these procedures remain
applicable as a complementary technique.
Yang, et al. Expires 3 May 2027 [Page 10]
Internet-Draft Adaptive ECMP Load Balancing October 2026
8. Relationship to Other Work
The procedures in this document make the path-selection decision
load-aware. A separate class of mechanisms instead improves the
entropy that is fed into a static hash, so that the initial
distribution is more uniform; examples are MPLS entropy labels
[RFC6790], the IPv6 flow label [RFC6438], and, for IPsec ESP traffic,
the ESP Entropy Header [I-D.yang-ipsecme-esp-entropy-header]. These
entropy-improving mechanisms and the load-aware procedures of this
document address different aspects of the same problem and are
complementary: better entropy improves the initial distribution,
while load-aware placement corrects the skew that develops after
flows are assigned. Neither removes the need for the other.
[I-D.li-adaptive-load-balance-enhancement] also addresses adaptive
load balancing for ECMP, with an emphasis on flowlet-based methods
and the difficulty of configuring a flowlet timeout. The procedures
in this document differ in that they operate as a node-local feedback
loop with an explicit imbalance assessment and a bounded,
proportional number of flow placement changes per interval, and they
are not tied to flowlet detection. The two approaches may inform
each other; this document does not depend on flowlet detection.
9. Applicability Considerations
9.1. Data Center Fabrics
Data center leaf-spine fabrics commonly use large numbers of ECMP
paths [RFC7938]. The combination of high path count, a mix of flow
sizes including long-lived storage and compute flows, and high
utilization targets makes these environments prone to hash-induced
load skew. These procedures can improve fabric-wide load balancing
without a central controller.
9.2. Wide Area Networks
In WAN environments where ECMP is used across parallel links or
tunnels, the smaller number of concurrent flows and their longer
lifetimes increase the effect of each flow placement decision on the
aggregate load. An operator that deploys these procedures in a WAN
SHOULD consider a longer evaluation interval and a lower reassignment
limit, to reflect the higher cost of a path change in a wide-area
topology.
10. Operational Considerations
Yang, et al. Expires 3 May 2027 [Page 11]
Internet-Draft Adaptive ECMP Load Balancing October 2026
10.1. Configuration Parameters
The following parameters govern the behavior described in this
document. A node MUST allow these parameters to be configured per
ECMP group, or per LAG.
+======================+=================================+==========+
| Parameter | Description | Default |
+======================+=================================+==========+
| Enable/Disable | Activate or deactivate adaptive | Disabled |
| | load balancing for the group | |
+----------------------+---------------------------------+----------+
| Evaluation | Time between consecutive load | 1 second |
| Interval | assessments | |
+----------------------+---------------------------------+----------+
| Load Metric Type | Metric used for per-path | Byte |
| | comparison | count |
+----------------------+---------------------------------+----------+
| Imbalance | IR below which no change is | 0.3 |
| Threshold (T_imb) | made | |
+----------------------+---------------------------------+----------+
| Maximum | Upper bound on flow placement | 8 |
| Reassignment | changes per interval | |
| Limit (R_max) | | |
+----------------------+---------------------------------+----------+
| Active Event | Which events may trigger a | New flow |
| Types | change | arrival |
+----------------------+---------------------------------+----------+
Table 1: Configuration Parameters
A parameter change SHOULD take effect at the start of the next
evaluation interval, to avoid inconsistent behavior during a running
interval. Disabling the feature MUST cause the node to stop making
load-aware changes immediately and to revert to hash-based path
selection for subsequent flows. Flows that were previously moved MAY
remain on their current paths; the node is not required to return
them to their hash-computed paths on disablement.
10.2. Flow Exclusion Policies
Some traffic classes require path persistence and MUST NOT be subject
to flow placement change. A node MUST support the configuration of
exclusion policies that identify flows exempt from change. Examples
of traffic that requires exclusion are the following.
Yang, et al. Expires 3 May 2027 [Page 12]
Internet-Draft Adaptive ECMP Load Balancing October 2026
* Flows that traverse stateful middleboxes such as firewalls, NAT
gateways, or intrusion prevention systems, where a path change
would invalidate session state.
* Real-time media flows, such as voice or video, that are sensitive
to jitter and reordering.
* Flows marked with a DSCP value that indicates a path affinity
requirement.
An exclusion policy SHOULD support matching on standard packet header
fields, including source and destination address prefixes, protocol
numbers, port ranges, and DSCP values. An excluded flow MUST NOT
consume reassignment capacity and MUST remain on its hash-assigned
path.
10.3. Monitoring
A node SHOULD expose the following operational state for each ECMP
group on which adaptive load balancing is enabled.
* Current per-path load metric values.
* Most recently computed imbalance ratio.
* Reassignment limit derived for the current interval and the
portion consumed.
* Cumulative count of flow placement changes since the feature was
enabled.
* Number of flows currently excluded by policy.
This state SHOULD be accessible through standard management
interfaces, such as a YANG-based management protocol [RFC8343], SNMP,
or streaming telemetry. This document does not define a data model.
10.4. Deployment Guidance
An operator is advised to adopt a conservative initial posture.
* Enable the feature on a small number of ECMP groups first and
observe the imbalance ratio and change counts before wider
rollout.
* Use a higher imbalance threshold, for example 0.5, and a lower
reassignment limit initially, then tighten the parameters as
confidence grows.
Yang, et al. Expires 3 May 2027 [Page 13]
Internet-Draft Adaptive ECMP Load Balancing October 2026
* Ensure that all stateful middlebox traversals are covered by
exclusion policies before enabling the feature.
11. Conformance and Interoperability
The procedures in this document constrain the forwarding-plane
behavior of a single node and do not define any new packet format,
header field, or inter-node protocol. The packets a conforming node
emits are bit-for-bit indistinguishable from those of a node that
performs only hash-based path selection; a flow placement change
alters which member of an ECMP group or LAG a flow egresses on, not
the contents of the flow's packets. Interoperability with neighbors,
with transport endpoints, and with existing ECMP and LAG deployments
is therefore preserved unconditionally.
The normative requirements in this document are stated as externally
observable properties so that conformance can be verified without
access to a node's internal implementation. In particular, a
conforming implementation:
* changes the path of no more than the derived reassignment limit
(Section 5.3) of flows in any evaluation interval, and makes no
changes while the imbalance ratio is below the configured
threshold (Section 5.2);
* preserves per-flow ordering for every flow that is not moved in a
given interval, and moves a flow at most as a single, atomic
change of egress member;
* never moves a flow that matches a configured exclusion policy
(Section 10.2);
* reverts to hash-based path selection for subsequent flows when the
feature is disabled (Section 10.1); and
* exposes the operational state of Section 10.3 through which the
preceding properties can be observed.
These properties are testable at a node's interfaces: an operator or
test harness can offer a controlled traffic pattern, observe per-path
load and the reported imbalance ratio, and confirm that the number
and rate of flow movements respect the configured bounds and
exclusions. The normative requirements thus give operators an
enforceable basis for procurement and conformance testing, which is
the reason this document is placed on the Standards Track even though
it introduces no on-the-wire protocol element.
Yang, et al. Expires 3 May 2027 [Page 14]
Internet-Draft Adaptive ECMP Load Balancing October 2026
12. Security Considerations
The procedures in this document operate within the local forwarding
plane of a single node. They introduce no new protocol messages,
packet formats, or inter-node signaling, and so do not expand the
control-plane attack surface.
The following data-plane considerations apply.
Stateful Middlebox Bypass
A flow placement change causes a flow's packets to use a different
path, which may bypass a stateful inspection device. An operator
MUST configure exclusion policies (Section 10.2) for all flows
subject to stateful processing.
Induced Path Changes
An attacker that can inject traffic with chosen flow identifiers
could try to concentrate load on one path and cause repeated
changes. The reassignment limit bounds the per-interval cost of
such an attempt, the imbalance threshold prevents reaction to
minor fluctuations, and the evaluation interval imposes a minimum
time between rounds of changes. Together these bound the
amplification.
Flow State Resource Consumption
A node that keeps per-flow state to support placement changes, for
example to record which flows have been moved, SHOULD bound the
size of that state to prevent memory exhaustion, especially where
flow counts are high.
13. IANA Considerations
This document has no IANA actions.
14. References
14.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
14.2. Informative References
Yang, et al. Expires 3 May 2027 [Page 15]
Internet-Draft Adaptive ECMP Load Balancing October 2026
[I-D.li-adaptive-load-balance-enhancement]
Li, Z., "Adaptive Load Balance Enhancement", Work in
Progress, Internet-Draft, draft-li-adaptive-load-balance-
enhancement-00, 2025,
<https://datatracker.ietf.org/doc/html/draft-li-adaptive-
load-balance-enhancement-00>.
[I-D.yang-ipsecme-esp-entropy-header]
Yang, J., Cheng, W., Wang, J., and G. Zhang, "An Entropy
Header for Load Balancing of IPsec ESP Traffic", Work in
Progress, Internet-Draft, draft-yang-ipsecme-esp-entropy-
header-00, 2026, <https://datatracker.ietf.org/doc/html/
draft-yang-ipsecme-esp-entropy-header-00>.
[RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and
Multicast Next-Hop Selection", RFC 2991,
DOI 10.17487/RFC2991, November 2000,
<https://www.rfc-editor.org/info/rfc2991>.
[RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path
Algorithm", RFC 2992, DOI 10.17487/RFC2992, November 2000,
<https://www.rfc-editor.org/info/rfc2992>.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
for Equal Cost Multipath Routing and Link Aggregation in
Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
<https://www.rfc-editor.org/info/rfc6438>.
[RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W., and
L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
RFC 6790, DOI 10.17487/RFC6790, November 2012,
<https://www.rfc-editor.org/info/rfc6790>.
[RFC7424] Krishnan, R., Yong, L., Ghanwani, A., So, N., and B.
Khasnabish, "Mechanisms for Optimizing Link Aggregation
Group (LAG) and Equal-Cost Multipath (ECMP) Component Link
Utilization in Networks", RFC 7424, DOI 10.17487/RFC7424,
January 2015, <https://www.rfc-editor.org/info/rfc7424>.
[RFC7938] Lapukhov, P., Premji, A., and J. Mitchell, Ed., "Use of
BGP for Routing in Large-Scale Data Centers", RFC 7938,
DOI 10.17487/RFC7938, August 2016,
<https://www.rfc-editor.org/info/rfc7938>.
[RFC8343] Bjorklund, M., "A YANG Data Model for Interface
Management", RFC 8343, DOI 10.17487/RFC8343, March 2018,
<https://www.rfc-editor.org/info/rfc8343>.
Yang, et al. Expires 3 May 2027 [Page 16]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Acknowledgements
The authors thank the members of the IETF Routing Area Working Group
for their review and feedback.
Authors' Addresses
Jin Yang
China Mobile
Beijing
100053
China
Email: yangjinwl@chinamobile.com
Yuchi Tian
China Mobile
Beijing
100053
China
Email: tianyuchi@chinamobile.com
Weiqiang Cheng
China Mobile
Beijing
100053
China
Email: chengweiqiang@chinamobile.com
Tingting Han
China Mobile
Beijing
100053
China
Email: hantingting@chinamobile.com
Junjie Wang
Centec
Suzhou
215000
China
Email: wangjj@centec.com
Yang, et al. Expires 3 May 2027 [Page 17]
Internet-Draft Adaptive ECMP Load Balancing October 2026
Guoying Zhang
Centec
Suzhou
215000
China
Email: zhanggy@centec.com
Yang, et al. Expires 3 May 2027 [Page 18]