| Internet-Draft | BIER and In Network Computing | November 2026 |
| Zhang, et al. | Expires 6 May 2027 | [Page] |
- Workgroup:
- bier
- Internet-Draft:
- draft-zzhang-bier-inc-00
- Published:
- Intended Status:
- Standards Track
- Expires:
BIER and In Network Computing
Abstract
In-Network-Computing/Collectives (INC) is an important element in Collective Communication for AI/ML. A multicast tree is typically set up for broadcasting in the forward direction and in-network computing/reduction in the reverse direction. BIER is a multicast technology that removes explicit multicast trees, and this document describes how INC can be done with BIER that does not use pre-established multicast trees.¶
Status of This Memo
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 6 May 2027.¶
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
1. Introduction
In Collective Communication, a communicator defines a specific group of processes/GPUs participating in a communication pattern, essentially acting as a communication channel or context where all members must act together (collectively) to perform operations like broadcasting, gathering, or reducing data, ensuring synchronized data movement among them.¶
Each communicator can be mapped to an IP multicast group with a set of trees, each of which rooted at a participant, used for broadcasting in the forward direction and in-network computing/reduction in the reverse direction.¶
The trees can be set up by the PIM protocol [RFC7761] or other means. However, tree setup/maintenance has long been considered a burden and scaling/convergence limit, and BIER [RFC8279] removes the trees while still providing efficient replication.¶
On the other hand, the pre-established tree is important for INC. It allows a transit node to send its reduction result upstream once it has received all downstream reduction results instead of relying on a timer. A theoretical stateless reduction not using the tree relies on a timer at each hop (because it does not know if all downstream results have been received) introduces a fixed delay at every hop.¶
Nonetheless, BIER can still be integrated with INC and provide additional INC-specific benefits, as described below.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
2. Mode of Operation
Essentially, the forward direction broadcasting does not need a pre-established explicit tree. BIER forwarding is used for that. The tree for INC purposes is established data-driven when the first packet traverses the network.¶
We first assume that each GPU is an edge BFR and allocated with a BFR-ID.¶
2.1. Forward Direction
Packets associated with INC carry the Communicator ID in the entropy field of the BIER header [RFC8296]. The (BFIR-ID, Entropy) tuple from the packet uniquely identifies a (source, group) tree (INC-tree).¶
The BIFT-ID field indicates that additional INC-related handling is needed, besides the regular BIER forwarding.¶
When an incoming BIER packet matches the INC-relevant BIFT (INC-BIFT), the (BFIR-ID, Entropy) tuple is used to look up the corresponding INC-tree. If it is found, packets are forwarded accordingly, with necessary processing related to INC. Notice that this is different from the bit-by-bit lookup with regular BIER forwarding.¶
If the tree is not found, it is created instantly on the node with the following information:¶
-
upstream node and incoming bitstring¶
-
downstream BFRs with their respective BFRs¶
-
INC-specific information¶
The downstream information is built from the bit-by-bit lookup for the regular BIER forwarding. The incoming bitstring is not expected to change in the life of the communicator barring routing changes. For simplicity, considerations for routing changes are deferred to Section 2.3.¶
2.2. Reduction Direction
Packets are also encoded in BIER. The BFIR-ID and Entropy field are set according to the corresponding forward direction packets. In particular, the BFIR-ID identifies the destination and not the sender of the packet. The initial bitstring only identifies the sender itself.¶
Another INC-BIFT ID is needed to indicate that this is for INC in the reverse direction. The corresponding INC-tree is identified from the (BFIR-ID, Entropy) tuple, and the INC processing kicks in.¶
The INC state includes a bitstring that identifies all the BFERs from which the reduction packet has been received. It is the logical OR of the bitstring from all the corresponding reduction packets. The reduction result is sent to the upstream node when the bitstring matches the incoming bitstring saved in the tree state, or when a timer fires. The bitstring is encoded into the BIER header.¶
The BFIR can determine which BFER has not sent back the reduction result based on the received bitstring - this is an additional benefit provided by BIER.¶
2.3. Routing Changes
Naturally, routing changes happen because of link/node failure/recovery and congestion situations (when adaptive routing is used). When this happens, the INC tree state also changes. How that impacts the reduction will be discussed in a future revision, but it may not be much different from the existing tree-based multicast case.¶
2.4. Leaf Switches as Edge BFRs
The above discussions assume that GPUs are edge BFRs. In a scaled setup, it may be that the edge BFRs are leaf switches, not GPUs. In this case, the leaf switch for the source of the reduction packets needs to identify the BFIR-ID.¶
3. Security Considerations
To be added.¶
4. IANA Considerations
No IANA action is needed.¶
5. Acknowledgments
The authors thank Duncan Roweth and David Hewson for their collaboration on this topic.¶
6. References
6.1. Normative References
- [RFC2119]
- Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
- [RFC8174]
- Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
- [RFC8279]
- Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., Przygienda, T., and S. Aldrin, "Multicast Using Bit Index Explicit Replication (BIER)", RFC 8279, DOI 10.17487/RFC8279, , <https://www.rfc-editor.org/info/rfc8279>.
- [RFC8296]
- Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation for Bit Index Explicit Replication (BIER) in MPLS and Non-MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, , <https://www.rfc-editor.org/info/rfc8296>.
6.2. Informative References
- [RFC7761]
- Fenner, B., Handley, M., Holbrook, H., Kouvelas, I., Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, , <https://www.rfc-editor.org/info/rfc7761>.