MPLS Working Group                                        R. Gandhi, Ed.
Internet-Draft                                                    Z. Ali
Intended status: Standards Track                             C. Filsfils
Expires: August 22, 2021                                    F. Brockners
                                                     Cisco Systems, Inc.
                                                                  B. Wen
                                                                V. Kozak
                                                                 Comcast
                                                       February 18, 2021


           MPLS Data Plane Encapsulation for In-situ OAM Data
                      draft-gandhi-mpls-ioam-sr-06

Abstract

   In-situ Operations, Administration, and Maintenance (IOAM) records
   operational and telemetry information in the data packet while the
   packet traverses a path between two nodes in the network.  This
   document defines how IOAM data fields are transported with MPLS data
   plane encapsulation using new Generic Associated Channel (G-ACh).

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on August 22, 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents



Gandhi, et al.           Expires August 22, 2021                [Page 1]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Requirement Language  . . . . . . . . . . . . . . . . . .   3
     2.2.  Abbreviations . . . . . . . . . . . . . . . . . . . . . .   3
   3.  MPLS Extensions for IOAM Data Fields  . . . . . . . . . . . .   4
     3.1.  IOAM Generic Associated Channel . . . . . . . . . . . . .   4
     3.2.  IOAM Indicator Labels . . . . . . . . . . . . . . . . . .   5
   4.  Edge-to-Edge IOAM . . . . . . . . . . . . . . . . . . . . . .   5
     4.1.  Edge-to-Edge IOAM Indicator Label . . . . . . . . . . . .   5
     4.2.  Procedure for Edge-to-Edge IOAM . . . . . . . . . . . . .   6
     4.3.  Edge-to-Edge IOAM Indicator Label Allocation  . . . . . .   7
   5.  Hop-by-Hop IOAM . . . . . . . . . . . . . . . . . . . . . . .   7
     5.1.  Hop-by-Hop IOAM Indicator Label . . . . . . . . . . . . .   7
     5.2.  Procedure for Hop-by-Hop IOAM . . . . . . . . . . . . . .   8
     5.3.  Hop-by-Hop IOAM Indicator Label Allocation  . . . . . . .   8
   6.  Considerations for IOAM Indicator Label . . . . . . . . . . .   9
     6.1.  Considerations for ECMP . . . . . . . . . . . . . . . . .   9
     6.2.  Node Capability . . . . . . . . . . . . . . . . . . . . .   9
     6.3.  MSD Considerations  . . . . . . . . . . . . . . . . . . .   9
     6.4.  Nested MPLS Encapsulation . . . . . . . . . . . . . . . .  10
   7.  MPLS Encapsulation with Control Word and Another G-ACh for
       IOAM Data Fields  . . . . . . . . . . . . . . . . . . . . . .  10
   8.  Example MPLS Encapsulations . . . . . . . . . . . . . . . . .  12
     8.1.  Example SR-MPLS Encapsulation with IOAM . . . . . . . . .  12
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
   10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  14
     11.2.  Informative References . . . . . . . . . . . . . . . . .  15
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  16
   Contributors  . . . . . . . . . . . . . . . . . . . . . . . . . .  16
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  16

1.  Introduction

   In-situ Operations, Administration, and Maintenance (IOAM) records
   operational and telemetry information within the packet while the
   packet traverses a particular network domain.  The term "in-situ"
   refers to the fact that the IOAM data fields are added to the data
   packets rather than being sent within the probe packets specifically



Gandhi, et al.           Expires August 22, 2021                [Page 2]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   dedicated to OAM or Performance Measurement (PM).  The IOAM data
   fields are defined in [I-D.ietf-ippm-ioam-data], and can be used for
   various use-cases for OAM and PM.  The IOAM data fields are further
   updated in [I-D.ietf-ippm-ioam-direct-export] for direct export use-
   cases and in [I-D.ietf-ippm-ioam-flags] for Loopback and Active
   flags.

   This document defines how IOAM data fields are transported with MPLS
   data plane encapsulations using new Generic Associated Channel
   (G-ACh).

2.  Conventions

2.1.  Requirement Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119] [RFC8174]
   when, and only when, they appear in all capitals, as shown here.

2.2.  Abbreviations

   Abbreviations used in this document:

   ECMP      Equal Cost Multi-Path

   E2E       Edge-To-Edge

   G-ACh     Generic Associated Channel

   HbH       Hop-by-Hop

   IOAM      In-situ Operations, Administration, and Maintenance

   MPLS      Multiprotocol Label Switching

   OAM       Operations, Administration, and Maintenance

   PM        Performance Measurement

   POT       Proof-of-Transit

   PSID      Path Segment Identifier

   PW        PseudoWire

   SR        Segment Routing




Gandhi, et al.           Expires August 22, 2021                [Page 3]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   SR-MPLS   Segment Routing with MPLS Data plane

3.  MPLS Extensions for IOAM Data Fields

3.1.  IOAM Generic Associated Channel

   The IOAM data fields are defined in [I-D.ietf-ippm-ioam-data].  The
   IOAM data fields are carried in the MPLS header as shown in Figure 1.
   More than one trace options can be present in the IOAM data fields.
   G-ACh [RFC5586] provides a mechanism to transport OAM and other
   control messages over MPLS data plane.  The IOAM G-ACh header
   [RFC5586] with new IOAM G-ACh type is added immediately after the
   MPLS label stack in the MPLS header as shown in Figure 1, before the
   IOAM data fields.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |0 0 0 1|Version|   Reserved    |          IOAM G-ACh           |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  |
    | Reserved      | Block Number  | IOAM-OPT-Type |IOAM HDR Length|  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  I
    |                                                               |  O
    |                                                               |  A
    ~                 IOAM Option and Data Space                    ~  M
    |                                                               |  |
    |                                                               |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |                                                               |
    |                                                               |
    |                 Payload + Padding                             |
    |                                                               |
    |                                                               |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 1: IOAM Generic Associated Channel with IOAM Data Fields

   The IOAM data fields are encapsulated using the following fields in
   the MPLS header:

   IP Version Number 0001b:  The first four octets are IP Version Field
     part of a G-ACh header, as defined in [RFC5586].

   Version:  The Version field is set to 0, as defined in [RFC4385].

   IOAM G-ACh:  Generic Associated Channel (G-ACh) Type (value TBA3) for
     IOAM [RFC5586].



Gandhi, et al.           Expires August 22, 2021                [Page 4]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   Reserved:  Reserved Bits MUST be set to zero upon transmission and
     ignored upon receipt.

   Block Number:  The Block Number can be used to aggregate the IOAM
     data collected in data plane, e.g. compute measurement metrics for
     each block of a flow.  It is also used to correlate the IOAM data
     on different nodes.

   IOAM-OPT-Type:  8-bit field defining the IOAM Option type, as defined
     in Section 8.1 of [I-D.ietf-ippm-ioam-data].

   IOAM HDR LEN:  8-bit unsigned integer.  Length of the IOAM HDR in
     4-octet units.

   IOAM Option and Data Space:  IOAM option header and data is present
     as defined by the IOAM-OPT-Type field, and is defined in Section 5
     of [I-D.ietf-ippm-ioam-data].

3.2.  IOAM Indicator Labels

   An IOAM Indicator Label is used to indicate the presence of the IOAM
   data fields in the MPLS header.  There are two IOAM types defined in
   this document: Edge-to-Edge (E2E) and Hop-by-Hop (HbH) IOAM.  If only
   edge nodes need to process IOAM data then E2E IOAM Indicator Label is
   used so that intermediate nodes can ignore it.  If both edge and
   intermediate nodes need to process IOAM data then HbH IOAM Indicator
   Label is used.  Different IOAM Indicator Labels allow to optimize the
   IOAM processing on intermediate nodes by checking if IOAM data fields
   need to be processed.

4.  Edge-to-Edge IOAM

4.1.  Edge-to-Edge IOAM Indicator Label

   The E2E IOAM Indicator Label is used to indicate the presence of the
   E2E IOAM data fields in the MPLS header as shown in Figure 2.















Gandhi, et al.           Expires August 22, 2021                [Page 5]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(1)                             | TC  |S|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    .                                                               .
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(n)                             | TC  |S|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  E2E IOAM Indicator Label             | TC  |1|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                Packet as shown in Figure 1                    |
    .                                                               .
    +---------------------------------------------------------------+

                 Figure 2: MPLS Encapsulation for E2E IOAM

   The E2E IOAM data fields carry the Option-Type(s) that require
   processing on the encapsulating and decapsulating nodes only.  The
   IOAM Option-Type carried can be IOAM Edge-to-Edge Option-Type
   [I-D.ietf-ippm-ioam-data].  The E2E IOAM data fields SHOULD NOT carry
   any IOAM Option-Type that require IOAM processing on the intermediate
   nodes as it will not be processed by them.

4.2.  Procedure for Edge-to-Edge IOAM

   The E2E IOM procedure is summarized as following:

   o  The encapsulating node inserts the E2E IOAM Indicator Label and
      one or more IOAM data fields in the MPLS header.

   o  The intermediate nodes do not process IOAM data fields.

   o  The decapsulating node "punts the timestamped copy" of the
      received packet as is including the IOAM data fields when the node
      recognizes the IOAM Indicator Label.  The copy of the packet is
      punted with receive timestamp to the slow path for IOAM data
      fields processing.  The receive timestamp is required by the
      various E2E OAM use-cases, including streaming telemetry.  Note
      that it is not necessarily punted to the control-plane.

   o  The decapsulating node processes the IOAM data fields using the
      procedures defined in [I-D.ietf-ippm-ioam-data].  An example of
      IOAM processing is to export the data fields, send data fields via
      streaming telemetry, etc.

   o  The decapsulating node also pops the IOAM Indicator Label and the
      IOAM data fields from the received packet.  The decapsulated



Gandhi, et al.           Expires August 22, 2021                [Page 6]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


      packet is forwarded downstream or terminated locally similar to
      the regular data packets.

4.3.  Edge-to-Edge IOAM Indicator Label Allocation

   The E2E IOAM Indicator Label is used to indicate the presence of the
   E2E IOAM data fields in the MPLS header.  The E2E IOAM Indicator
   Label can be allocated using one of the following three methods:

   o  Label assigned by IANA with value TBA1 from the Extended Special-
      Purpose MPLS Values [I-D.ietf-mpls-spl-terminology].

   o  Label allocated by a Controller from the global table of the
      decapsulating node.  The Controller provisions the label on both
      encapsulating and decapsulating nodes.

   o  Label allocated by the decapsulating node and signalled or
      advertised in the network.  The signaling and/or advertisement
      extension for this is outside the scope of this document.

5.  Hop-by-Hop IOAM

5.1.  Hop-by-Hop IOAM Indicator Label

   The HbH IOAM Indicator Label is used to indicate the presence of the
   HbH IOAM data fields in the MPLS header as shown in Figure 3.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(1)                             | TC  |S|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    .                                                               .
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(n)                             | TC  |S|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  HbH IOAM Indicator Label             | TC  |1|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                Packet as shown in Figure 1                    |
    .                                                               .
    +---------------------------------------------------------------+

                 Figure 3: MPLS Encapsulation for HbH IOAM

   The HbH IOAM data fields carry the Option-Type(s) that require
   processing at the intermediate and/or encapsulating and decapsulating
   nodes.  The IOAM Option-Type carried can be IOAM Pre-allocated Trace
   Option-Type, IOAM Incremental Trace Option-Type and IOAM Proof of



Gandhi, et al.           Expires August 22, 2021                [Page 7]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   Transit (POT) Option-Type, as well as Edge-to-Edge Option-Type
   [I-D.ietf-ippm-ioam-data].

5.2.  Procedure for Hop-by-Hop IOAM

   The HbH IOAM procedure is summarized as following:

   o  The encapsulating node inserts the HbH IOAM Indicator Label and
      one or more IOAM data fields in the MPLS header.

   o  The intermediate node enabled with HbH IOAM functions processes
      the data packet including the IOAM data fields as defined in
      [I-D.ietf-ippm-ioam-data] when the node recognizes the HbH IOAM
      Indicator Label present in the MPLS header.  The intermediate node
      may 'punt the timestamped copy' of the received data packet
      including the IOAM data fields as required by the IOAM data fields
      processing.  The copy of the packet is punted with receive
      timestamp to the slow path for IOAM processing.

   o  The intermediate node forwards a copy of the processed data packet
      downstream.

   o  The decapsulating node "punts the timestamped copy" of the
      received data packet as is including the IOAM data fields when the
      node recognizes the IOAM Indicator Label.  The copy of the packet
      is punted with receive timestamp to the slow path for IOAM data
      fields processing.  The receive timestamp is required by the
      various E2E OAM use-cases, including streaming telemetry.  Note
      that it is not necessarily punted to the control-plane.

   o  The decapsulating node processes the IOAM data fields using the
      procedures defined in [I-D.ietf-ippm-ioam-data].  An example of
      IOAM processing is to export the data fields, send data fields via
      streaming telemetry, etc.

   o  The decapsulating node also pops the IOAM Indicator Label and the
      IOAM data fields from the received packet.  The decapsulated
      packet is forwarded downstream or terminated locally similar to
      the regular data packets.

5.3.  Hop-by-Hop IOAM Indicator Label Allocation

   The HbH IOAM Indicator Label is used to indicate the presence of the
   HbH IOAM data fields in the MPLS header.  The HbH IOAM Indicator
   Label can be allocated using one of the following three methods:

   o  Label assigned by IANA with value TBA2 from the Extended Special-
      Purpose MPLS Values [I-D.ietf-mpls-spl-terminology].



Gandhi, et al.           Expires August 22, 2021                [Page 8]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   o  Label allocated by a Controller from the network-wide global
      table.  The Controller provisions the labels on all nodes
      participating in IOAM functions along the data traffic path.

   o  Labels allocated by the intermediate and decapsulating nodes and
      signalled or advertised in the network.  The signaling and/or
      advertisement extension for this is outside the scope of this
      document.

6.  Considerations for IOAM Indicator Label

6.1.  Considerations for ECMP

   The encapsulating node needs to make sure the IOAM data fields do not
   start with a well-known IP Version Number (e.g. 0x4 for IPv4 and 0x6
   for IPv6) as that can alter the hashing function for ECMP that uses
   the IP header.  This is achieved by using the IOAM G-ACh with IP
   Version Number 0001b after the MPLS label stack [RFC5586].

   Note that the hashing function for ECMP that uses the labels from the
   MPLS header may now include the IOAM Indicator Label.

   When entropy label [RFC6790] is used for hashing function for ECMP,
   the procedure defined in this document does not alter the hashing
   function.

6.2.  Node Capability

   The decapsulating node that has to pop the IOAM Indicator Label, data
   fields, and perform the IOAM function may not be capable of
   supporting it.  The encapsulating node needs to know if the
   decapsulating node can support the IOAM function.  The signaling
   extension for this capability exchange is outside the scope of this
   document.

   The intermediate node that is not capable of supporting the IOAM
   functions defined in this document, can simply skip the IOAM
   processing of the MPLS header.

6.3.  MSD Considerations

   The SR path computation needs to know the Maximum SID Depth (MSD)
   that can be imposed at each node/link of a given SR path [RFC8664].
   This ensures that the SID stack depth of a computed path does not
   exceed the number of SIDs the node is capable of imposing.  The MSD
   used for path computation MUST include the IOAM Indicator Label.





Gandhi, et al.           Expires August 22, 2021                [Page 9]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


6.4.  Nested MPLS Encapsulation

   The data packets with IOAM data fields carry only one IOAM Indicator
   Label in the MPLS header.  Any intermediate node that adds additional
   MPLS encapsulation in the MPLS header may further update the IOAM
   data fields in the header without inserting another IOAM Indicator
   Label.  When a packet is received with a HbH IOAM Indicator Label,
   the nested MPLS encapsulating node can add a HbH and/or E2E IOAM
   Option-Type.  However, when a packet is received with an E2E IOAM
   Indicator Label, the nested MPLS encapsulating node SHOULD NOT add a
   HbH IOAM Option-Type, as intermediate nodes will not process it.

7.  MPLS Encapsulation with Control Word and Another G-ACh for IOAM Data
    Fields

   The IOAM data fields, including IOAM G-ACh header are added in the
   MPLS encapsulation immediately after the MPLS header.  Any Control
   Word [RFC4385] or another G-ACh [RFC5586] MUST be added after the
   IOAM data fields in the packet as shown in the Figure 4 and Figure 5,
   respectively.  This allows the intermediate nodes to easily access
   the HbH IOAM data fields located immediately after the MPLS header.
   The decapsulating node can remove the MPLS encapsulation including
   the IOAM data fields and then process the Control Word or another
   G-ACh following it.



























Gandhi, et al.           Expires August 22, 2021               [Page 10]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | IOAM Indicator Label                  | TC  |1|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |0 0 0 1|Version| Reserved      | IOAM G-ACh                    |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  |
    | Reserved      | Block Number  | IOAM-OPT-Type |IOAM HDR Length|  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  I
    |                                                               |  O
    |                                                               |  A
    ~                 IOAM Option and Data Space                    ~  M
    |                                                               |  |
    |                                                               |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |0 0 0 0| Specified by PW Encapsulation [RFC4385]               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                                                               |
    ~                 Payload + Padding                             ~
    |                                                               |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 4: Example MPLS Encapsulation with Generic PW Control Word
                                 with IOAM

























Gandhi, et al.           Expires August 22, 2021               [Page 11]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | IOAM Indicator Label                  | TC  |1|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |0 0 0 1|Version| Reserved      | IOAM G-ACh                    |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  |
    | Reserved      | Block Number  | IOAM-OPT-Type |IOAM HDR Length|  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  I
    |                                                               |  O
    |                                                               |  A
    ~                 IOAM Option and Data Space                    ~  M
    |                                                               |  |
    |                                                               |  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
    |0 0 0 1|Version| Reserved      | Channel Type                  |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    |                                                               |
    ~                 Payload + Padding                             ~
    |                                                               |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 5: Example MPLS Encapsulation with Another G-ACh with IOAM

8.  Example MPLS Encapsulations

8.1.  Example SR-MPLS Encapsulation with IOAM

   Segment Routing (SR) technology leverages the source routing paradigm
   [RFC8660].  A node steers a packet through a controlled set of
   instructions, called segments, by pre-pending the packet with an SR
   header.  In the SR with MPLS data plane (SR-MPLS), the SR header is
   instantiated through a label stack.

   An example of data packet with SR-MPLS encapsulation containing Path
   Segment Identifier (PSID) [I-D.ietf-spring-mpls-path-segment] and E2E
   IOAM data fields is shown in Figure 6.  The PSID allows to identify
   the path associated with the data traffic being monitored for IOAM on
   the decapsulating node.










Gandhi, et al.           Expires August 22, 2021               [Page 12]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(1)                             | TC  |S|      TTL      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    .                                                               .
    .                                                               .
    .                                                               .
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Label(n)                             | TC  |S|      TTL      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  PSID                                 | TC  |S|      TTL      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Extension Label (15)                 | TC  |S|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  E2E IOAM Indicator Label TBA1        | TC  |1|  TTL          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                Packet as shown in Figure 1                    |
    .                                                               .
    +---------------------------------------------------------------+

     Figure 6: Example SR-MPLS Encapsulation with E2E IOAM Data Fields

9.  Security Considerations

   The security considerations of IOAM in general are discussed in
   [I-D.ietf-ippm-ioam-data].

   IOAM is considered a "per domain" feature, where one or several
   operators decide on leveraging and configuring IOAM according to
   their needs.  Still, operators need to properly secure the IOAM
   domain to avoid malicious configuration and use, which could include
   injecting malicious IOAM packets into a domain.

   Routers that support G-ACh are subject to the same security
   considerations as defined in [RFC4385] and [RFC5586].

10.  IANA Considerations

   IANA maintains the "Special-Purpose Multiprotocol Label Switching
   (MPLS) Label Values" registry (see <https://www.iana.org/assignments/
   mpls-label-values/mpls-label-values.xml>).  IANA is requested to
   allocate IOAM Indicator Label value from the "Extended Special-
   Purpose MPLS Label Values" registry:







Gandhi, et al.           Expires August 22, 2021               [Page 13]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


           +--------+--------------------------+---------------+
           |  Value |  Description             |  Reference    |
           +--------+--------------------------+---------------+
           | TBA1   | E2E IOAM Indicator Label | This document |
           +--------+--------------------------+---------------+
           | TBA2   | HbH IOAM Indicator Label | This document |
           +--------+--------------------------+---------------+

                   Table 1: IOAM Indicator Label Values

   IANA maintains G-ACh Type Registry (see
   <https://www.iana.org/assignments/g-ach-parameters/g-ach-
   parameters.xhtml>).  IANA is requested to allocate a value for IOAM
   G-ACh Type from "MPLS Generalized Associated Channel (G-ACh) Types
   (including Pseudowire Associated Channel Types)" registry.

                +-------+-----------------+---------------+
                | Value | Description     | Reference     |
                +-------+-----------------+---------------+
                | TBA3  | IOAM G-ACh Type | This document |
                +-------+-----------------+---------------+

                         Table 2: IOAM G-ACh Type

11.  References

11.1.  Normative References

   [I-D.ietf-ippm-ioam-data]
              Brockners, F., Bhandari, S., and T. Mizrahi, "Data Fields
              for In-situ OAM", draft-ietf-ippm-ioam-data-11 (work in
              progress), November 2020.

   [I-D.ietf-ippm-ioam-direct-export]
              Song, H., Gafni, B., Zhou, T., Li, Z., Brockners, F.,
              Bhandari, S., Sivakolundu, R., and T. Mizrahi, "In-situ
              OAM Direct Exporting", draft-ietf-ippm-ioam-direct-
              export-02 (work in progress), November 2020.

   [I-D.ietf-ippm-ioam-flags]
              Mizrahi, T., Brockners, F., Bhandari, S., Sivakolundu, R.,
              Pignataro, C., Kfir, A., Gafni, B., Spiegel, M., and J.
              Lemon, "In-situ OAM Flags", draft-ietf-ippm-ioam-flags-03
              (work in progress), October 2020.







Gandhi, et al.           Expires August 22, 2021               [Page 14]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4385]  Bryant, S., Swallow, G., Martini, L., and D. McPherson,
              "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for
              Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385,
              February 2006, <https://www.rfc-editor.org/info/rfc4385>.

   [RFC5586]  Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed.,
              "MPLS Generic Associated Channel", RFC 5586,
              DOI 10.17487/RFC5586, June 2009,
              <https://www.rfc-editor.org/info/rfc5586>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

11.2.  Informative References

   [I-D.ietf-mpls-spl-terminology]
              Andersson, L., Kompella, K., and A. Farrel, "Special
              Purpose Label terminology", draft-ietf-mpls-spl-
              terminology-06 (work in progress), January 2021.

   [I-D.ietf-spring-mpls-path-segment]
              Cheng, W., Li, H., Chen, M., Gandhi, R., and R. Zigler,
              "Path Segment in MPLS Based Segment Routing Network",
              draft-ietf-spring-mpls-path-segment-03 (work in progress),
              September 2020.

   [RFC6790]  Kompella, K., Drake, J., Amante, S., Henderickx, W., and
              L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
              RFC 6790, DOI 10.17487/RFC6790, November 2012,
              <https://www.rfc-editor.org/info/rfc6790>.

   [RFC8660]  Bashandy, A., Ed., Filsfils, C., Ed., Previdi, S.,
              Decraene, B., Litkowski, S., and R. Shakir, "Segment
              Routing with the MPLS Data Plane", RFC 8660,
              DOI 10.17487/RFC8660, December 2019,
              <https://www.rfc-editor.org/info/rfc8660>.

   [RFC8664]  Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W.,
              and J. Hardwick, "Path Computation Element Communication
              Protocol (PCEP) Extensions for Segment Routing", RFC 8664,
              DOI 10.17487/RFC8664, December 2019,
              <https://www.rfc-editor.org/info/rfc8664>.



Gandhi, et al.           Expires August 22, 2021               [Page 15]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


Acknowledgements

   The authors would like to thank Patrick Khordoc, Shwetha Bhandari and
   Vengada Prasad Govindan for the discussions on IOAM.  The authors
   would also like to thank Tarek Saad, Loa Andersson, Greg Mirsky,
   Stewart Bryant, Xiao Min, and Cheng Li for providing many useful
   comments.  The authors would also like to thank Mach Chen, Andrew
   Malis, Matthew Bocci, and Nick Delregno for the MPLS-RT reviews.

Contributors

   Sagar Soni
   Cisco Systems, Inc.

   Email: sagsoni@cisco.com



Authors' Addresses

   Rakesh Gandhi (editor)
   Cisco Systems, Inc.
   Canada

   Email: rgandhi@cisco.com


   Zafar Ali
   Cisco Systems, Inc.

   Email: zali@cisco.com


   Clarence Filsfils
   Cisco Systems, Inc.
   Belgium

   Email: cf@cisco.com


   Frank Brockners
   Cisco Systems, Inc.
   Hansaallee 249, 3rd Floor
   DUESSELDORF, NORDRHEIN-WESTFALEN  40549
   Germany

   Email: fbrockne@cisco.com




Gandhi, et al.           Expires August 22, 2021               [Page 16]


Internet-Draft       In-situ OAM for MPLS Data plane       February 2021


   Bin Wen
   Comcast

   Email: Bin_Wen@cable.comcast.com


   Voitek Kozak
   Comcast

   Email: Voitek_Kozak@comcast.com









































Gandhi, et al.           Expires August 22, 2021               [Page 17]