INTERNET-DRAFT                                                  J. Kumar
Intended Status: Standards Track                                J. Lemon
                                                                Y. Peleg
                                                           Broadcom Inc.
                                                             K. Kompella
                                                        Juniper Networks
Expires: September 12, 2019                               March 11, 2019


                     MPLS Inband Network Telemetry
                        draft-kumar-mpls-mint-00


Abstract

   MPLS Inband Network Telemetry (MINT) records flow specific
   information from end stations and/or switches across a network.  This
   document describes the method to collect data on a per hop basis
   across a network and perform localized or end to end analytics
   operations on the data.  This document also describes the use of the
   Extension Label, value 15, for specifying extended special purpose
   labels for specifying presence of MINT data.


Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html


Copyright and License Notice




Kumar, et al.                                                   [Page 1]


INTERNET DRAFT                    IFA                     March 11, 2019


   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.



Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1  Terminology . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.2 Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . .  5
     1.3 Applicability  . . . . . . . . . . . . . . . . . . . . . . .  5
     1.4 Motivation . . . . . . . . . . . . . . . . . . . . . . . . .  6
   2. Requirements  . . . . . . . . . . . . . . . . . . . . . . . . .  6
     2.1 Encapsulation Requirements . . . . . . . . . . . . . . . . .  6
     2.2 Operational Requirements . . . . . . . . . . . . . . . . . .  6
   3. MINT Domains  . . . . . . . . . . . . . . . . . . . . . . . . .  6
     3.1 MINT Domain  . . . . . . . . . . . . . . . . . . . . . . . .  8
     3.2 MINT Function Nodes  . . . . . . . . . . . . . . . . . . . .  8
       3.2.1 Initiating Function Node . . . . . . . . . . . . . . . .  8
       3.2.2 Transit Function Node  . . . . . . . . . . . . . . . . .  9
       3.2.3 Terminating Function Node  . . . . . . . . . . . . . . .  9
       3.2.4 Metadata Fragmentation Function  . . . . . . . . . . . .  9
     3.3 MINT Cloning, Truncation, and Drop . . . . . . . . . . . . . 10
     3.4 MINT Label Stack . . . . . . . . . . . . . . . . . . . . . . 10
       3.4.1 MINT Metadata Header . . . . . . . . . . . . . . . . . . 12
       3.4.2 MINT Checksum Header . . . . . . . . . . . . . . . . . . 13
       3.4.3 MINT Metadata Fragmentation (MF) Header  . . . . . . . . 14
     3.5 MINT Metadata  . . . . . . . . . . . . . . . . . . . . . . . 15
       3.5.1 Global Name Space (GNS) Identifier . . . . . . . . . . . 15
       3.5.2 Local Name Space (LNS) Identifier  . . . . . . . . . . . 16
       3.5.3 Device ID  . . . . . . . . . . . . . . . . . . . . . . . 16
     3.6 MINT Network Overhead  . . . . . . . . . . . . . . . . . . . 16
     3.7 MINT Analytics . . . . . . . . . . . . . . . . . . . . . . . 16
     3.8 MINT Packet Format . . . . . . . . . . . . . . . . . . . . . 17
       3.8.1 MINT Packet Format with TS Flag Set  . . . . . . . . . . 17
     3.9 MINT Load Balancing  . . . . . . . . . . . . . . . . . . . . 18
   4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 19
     4.1.  Extended Special-Purpose MPLS Label Values Registry  . . . 19



Kumar, et al.                                                   [Page 2]


INTERNET DRAFT                    IFA                     March 11, 2019


   5. Security Considerations . . . . . . . . . . . . . . . . . . . . 19
   6. References  . . . . . . . . . . . . . . . . . . . . . . . . . . 19
     6.1  Normative References  . . . . . . . . . . . . . . . . . . . 19
     6.2 Informative References . . . . . . . . . . . . . . . . . . . 19
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20














































Kumar, et al.                                                   [Page 3]


INTERNET DRAFT                    IFA                     March 11, 2019


1.  Introduction

   This document describes MPLS Inband Network Telemetry (MINT), which
   is a mechanism to enable the collection of metadata for an analyzed
   MPLS flow.

   The sequence of per hop metadata in the packet preceded by a MINT
   header is referred to as a MINT stack.  The presence of a MINT stack
   is indicated by an extended special purpose label.  The "Extension
   Label", value 15, is used to indicate a following label, which will
   be the MINT label in this case.

   The MINT header and metadata definition are the same as defined in
   [I-D.draft-kumar-ippm-ifa-01].



1.1  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

   LSP: Label Switched Path

   MPLS: Multi Path Label Switching

   MINT: MPLS Inband Network Telemetry

   MTU: Maximum Transmit Unit

   MINT Domain: A set of MPLS nodes within a single administration with
   all nodes participating in MINT.

   MINT Node: A single node in a MINT domain.

   MINT Metadata: Information inserted by a MINT node.

   MINT Stack: Ordered collection of per hop metadata.

   MINT Label: IANA allocated extended special purpose label.  Indicates
   the presence of a MINT stack.

   MINT Initiator: A MINT node at the beginning edge of a MINT domain
   that is responsible for inserting an Extension label, a MINT label,
   and the initial MINT metadata.

   MINT Transit: A MINT node within the MINT domain, responsible for



Kumar, et al.                                                   [Page 4]


INTERNET DRAFT                    IFA                     March 11, 2019


   adding MINT metadata for the current node.

   MINT Terminator: A MINT node at the ending edge of a MINT domain that
   is responsible for removing the Extension label, the MINT label, and
   the MINT Stack and exporting the MINT Stack to a collector.


1.2 Scope

   This document describes MINT deployment, header definitions, packet
   format, and data path functions.

   MINT deployment involves defining a MINT Domain and understanding the
   requirements in terms of traffic overhead and points of data
   collection.  Given that MINT inserts two extra labels in the label
   stack, consideration MUST be given to network devices' capability to
   parse the depth of the label stack.  The scope of MINT is from an end
   station and/or ToR, through any/all nodes in the MPLS path, and
   terminating in a network switch and/or an end station. In case of a
   service provider network, the scope of MINT MAY encompass one carrier
   to another carrier.  Special care MUST be taken for leaking the MINT
   stack across a MINT domain for security reasons.

   MINT can create a synthetic stream of MPLS traffic and use it to
   collect metadata along the path.  This sampled stream is later
   discarded.  MINT can also insert metadata on a per packet basis in
   live traffic.

   This draft defines an identification mechanism using a extended
   special purpose label for MINT packets.

1.3 Applicability

   MINT is capable of providing traffic analysis for a given LSP.  The
   LSP may be tunneled in another LSP in part of the MINT domain.  In
   such cases more then one MINT stack MAY be present in the MINT
   packet.

   A very important requirement of MINT traffic is preserving the same
   path for the flow.  Since inserting a MINT label will change the
   label stack and possibly keys to load balancing functions, a MINT
   domain SHOULD use an entropy label as defined in [RFC 6790] to
   preserve the flow path.

   Inserting additional an label and metadata will increase the packet
   size and may impact path MTU.  MINT provides a metadata fragmentation
   function to keep the packet size below path MTU.




Kumar, et al.                                                   [Page 5]


INTERNET DRAFT                    IFA                     March 11, 2019


1.4 Motivation

   The main motivation for MINT is to collect analyzed metadata from
   packets within a flow for a given application.

   Each hop in the flow path MAY insert metadata in the packet or MAY
   export metadata to a collector.

2. Requirements

   MINT requirements are the same as IFA requirements and are defined
   with operational efficiency, performance of the network, and cost of
   hardware in mind.

2.1 Encapsulation Requirements

   MINT packets MUST be clearly marked and identifiable so that a
   networking element in the flow path can insert metadata or perform
   other MINT operations.

   MINT packets need to be easily identified for performance reasons.  A
   new extended special purpose label is requested for MINT.

   MINT encapsulation MAY support the ability to fragment metadata.

   MINT encapsulation MAY support direct export of metadata instead of
   inserting it into the packet.



2.2 Operational Requirements

   MINT MUST preserve the flow path across the network.  This can be
   done either by using specific key fields for load balancing functions
   or by using an entropy label.

   MINT MUST support cloning, live traffic, checksum, fragmentation,
   global name spaces, and local name spaces using the MINT header
   definitions.



3. MINT Domains

   MINT performs flow analysis, and possible actions on the flow data,
   in-band.  Once a flow is enabled for analysis, a MINT node with the
   role of "Initiator" makes a copy of the flow or samples the live
   traffic flow, or tags a live traffic flow for analysis and data



Kumar, et al.                                                   [Page 6]


INTERNET DRAFT                    IFA                     March 11, 2019


   collection.  Copying of a flow is done by sampling or cloning the
   flow.  These new packets are representative packets of the original
   flow and possess the exact same characteristics as the original flow.
    This means that MINT packets traverse the same path in the network
   and same queues in the networking element as the original packet
   would.  Figure 1 shows the MINT based Telemetry Framework.  The
   terminating node is responsible for terminating the MINT flow by
   summarizing the metadata of the entire path and sending it to a
   collector.


                              +----------+
                              |          |
                              |Collector |--------------+
                              |          |              |
                              +----------+              |
                                                        |
                                                        |
                                                        |
                                                        |
                                                        |
                                                        |
                                                        |
                                                        |
     +--------------+        +------------+        +----+-----------+
     |Initiator Node|        |Transit Node|        |Terminating Node|
     |   +------+   |        |  +------+  |        |     +------+   |
     |   | MINT |   |------->|  | MINT |  |------->|     | MINT |   |
     |   +------+   |MINTflow|  +------+  |MINTflow|     +------+   |
     +--------------+        +------------+        +----------------+

          Figure 1 MINT Domain Framework without fragmentation



















Kumar, et al.                                                   [Page 7]


INTERNET DRAFT                    IFA                     March 11, 2019


                              +----------+
                              |          |
                              |Collector |<-------------+
                       +----->|          |              |
                       |      +----------+              |
                       |                                |
                       |                                |
                       |                                |
                       |                                |
                       |                                |
                       |                                |
                       |                                |
                       |                                |
     +--------------+  |     +------------+        +----+----------+
     |Initiator Node|  |     |Transit Node|        |Fragmenter Node|
     |   +------+   |  |     |  +------+  |        |    +------+   |
     |   | MINT |   |------->|  | MINT |  |------->|    | MINT |   |
     |   +------+   |MINTflow|  +------+  |MINTflow|    +------+   |
     +--------------+  |     +------------+        +---------------+
                       |                                  |MINT
                       |                                  |flow
                       |                                  v
                       |  +---------------+        +------+-----+
                       |  |Terminator Node|        |Transit Node|
                       +--|    +------+   |        |  +------+  |
                          |    | MINT |   |<-------|  | MINT |  |
                          |    +------+   |MINTflow|  +------+  |
                          +---------------+        +------------+
       Figure 2 MINT Domain Framework with metadata fragmentation
3.1 MINT Domain

   A MINT Domain is the domain of interest where MINT monitoring is
   enabled. A MINT Domain MUST have designated MINT function nodes.
   Initiating and Terminating function MINT nodes are always at the edge
   of the MINT domain.  Internal nodes in the MINT Domain are always
   Transit function nodes.

3.2 MINT Function Nodes

   There are three types of MINT functional nodes with respect to a
   specific set of flows.  Each node MAY perform metadata fragmentation
   function as well.

3.2.1 Initiating Function Node

   An end station, a switch, or any other middleware can perform the
   MINT initiating function.  It is advantageous to keep this role
   closest to the application to maximize flow visibility.  A MINT



Kumar, et al.                                                   [Page 8]


INTERNET DRAFT                    IFA                     March 11, 2019


   initiating function node performs the following functions for a flow:

   - Samples the flow traffic of interest based on a configuration.
   - Converts the traffic into a MINT flow by adding a MINT header to
   each sample.
   - Updates the packet with initiating function node metadata.
   - MAY mandate a specific template ID metadata by all networking
   elements.
   - MAY mandate tail stamping of metadata by all networking elements.

3.2.2 Transit Function Node

   A MINT transit node is responsible for inserting transit node
   metadata in the MINT packets in the specified flow.

3.2.3 Terminating Function Node

   A MINT terminating node is responsible for the following for a flow:
   - Inserts terminating node metadata in a MINT packet.
   - Performs a local analytics function on one or more segments of
   metadata, e.g., threshold breach for residence time, congestion
   notifications, and so on.
   - Filters a MINT flow in case of cloned traffic.
   - Sends a copy or report of the packet to a collector.
   - Removes the MINT headers and forwards the packet in case of live
   traffic.

3.2.4 Metadata Fragmentation Function
   There are cases where the size of metadata may grow too big for link
   MTU or path MTU, or where it imposes excessive overhead for the
   terminating function node to remove it.  This is especially true in
   networks with a large number of hops between the initiator function
   node and the terminating function node.  This is also true where the
   size of per hop metadata itself is large.  For such cases, MINT
   defines a metadata fragmentation function.  The metadata
   fragmentation function allows removal of metadata from the packet and
   the sending of a copy/report of the packet to the collector.
   Correlation of metadata fragments and recreation of metadata stack
   for the entire flow path is done by the collector.

   There is no dedicated node performing the metadata fragmentation
   function.  As a MINT packet traverses the hops in a MINT Domain, any
   node MAY detect the need to fragment the packet's metadata stack and
   perform metadata fragmentation.

   Metadata fragmentation is done if the MINT header in the packet has
   "MDF" bit set and the current length of the metadata would exceed the
   maximum length after the addition of metadata by the current node.  A



Kumar, et al.                                                   [Page 9]


INTERNET DRAFT                    IFA                     March 11, 2019


   node MAY create a copy of the packet or create a MINT report, remove
   the existing metadata stack from the packet, insert its own metadata,
   and finally forward the packet.  A node MAY also update the MINT MDF
   (Meta Data Fragment) header, fragment identifier, and current length.

   The maximum length in a MINT header, if set to 0, MAY trigger the
   metadata fragmentation special function.  This mechanism can be used
   to generate MINT reports at each hop and never insert metadata in the
   packet.  If the maximum length is set to 0, a node MAY NOT insert
   MINT metadata, SHOULD create a MINT report or copy of the packet
   including its own metadata, and MUST increment the MINT MDF header
   fragment identifier field.

3.3 MINT Cloning, Truncation, and Drop

   MINT allows cloning of live traffic.  It is expected that cloned
   traffic will have the same network path characterization as the
   original traffic, i.e., follow the same network path, use the same
   queues, etc.

   Cloned traffic can be truncated to accommodate the MTU of the MINT
   Domain.

   Cloned traffic MUST be dropped by the terminating function node of
   the MINT Domain.

3.4 MINT Label Stack

   The MINT label stack is described below.  An extended special-purpose
   label (ESPL) is used to identify a MINT packet in the MPLS label
   stack at the level to be measured.




















Kumar, et al.                                                  [Page 10]


INTERNET DRAFT                    IFA                     March 11, 2019


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      ESPL:
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |            Label = 15 (ESPL)          |0 0 0|0|   TTL=0       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      MINT Extension Label:
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |            Label = 16 (MINTi)         |0 0 0|1|   TTL=0       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      MINT Header:
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Ver=2  |  GNS  |NextHdr = IP_xx|R|R|R|M|T|I|T|C|   Max Length  |
      |       |       |               | | | |F|S| |A| |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                   Figure 3 MINT Label Headers Format


   ESPL:
   A label value of 15 identifies an extended special-purpose label
   (ESPL), which indicates that the following label is an extension
   label.  The EXP bits SHOULD be set to 0.  The S bit MUST be set to 0
   to indicate that more labels follow.  The TTL for the ESPL MUST be
   zero to ensure that it is not used inadvertently for forwarding.

   MINT Extension Label:
   A label value of 16 identifies an MINT extension label, which
   indicates that header following this label is a MINT header.  The EXP
   bits SHOULD be set to 0.  The S bit MUST be set to 1 to indicate that
   no more labels follow.  The TTL for the MINT extension label MUST be
   zero to ensure that it is not used inadvertently for forwarding.

   MINT Header:
   (1) Version (4 bits). Specifies the version of MINT header.  The
   version MUST be set to 2 for this.

   (2) GNS (4 bits) - Global Name Space.  Specifies the MINT Domain
   scoped name space for the MINT metadata.

   (3) Protocol Type (8 bits) - IP Header protocol type.  This indicates
   what protocol follows MINT.

   (4) Flags (8 bits)
        0: R - Reserved.  MUST be initialized to 0 on transmission and



Kumar, et al.                                                  [Page 11]


INTERNET DRAFT                    IFA                     March 11, 2019


        ignored on receipt.

        1: R - Reserved.  MUST be initialized to 0 on transmission and
        ignored on receipt.

        2: R - Reserved.  MUST be initialized to 0 on transmission and
        ignored on receipt.

        3: MF - Metadata Fragment.  Indicates the presence of the
        optional metadata fragment header.  This header is inserted and
        initialized by the initiator node.  If the MF bit is set, nodes
        in the path MAY perform fragmentation of metadata stack if the
        current length exceeds the maximum length.

        4: TS - Tail Stamp.  Indicates the MINT Domain is requiring tail
        stamping of metadata.

        5: I - Inband.  Indicates this is live traffic.  Strip and
        forward MUST be performed by the terminator node if this bit is
        set.

        6: TA - Turn Around.  Indicates that the MINT packet needs to be
        turned around at the terminating node of the MINT Domain and
        sent back to the source if a return path is known.  This bit MAY
        be used for probe packets where probes are collection
        bidirectional information in the network.  This is analogous to
        an echo request and echo reply.  A packet with the TA bit set
        collects metadata in one direction, and after it is turned
        around by the terminating function node, collects metadata in
        the reverse direction.

        7: C - Checksum.  Indicates the presence of the optional
        checksum header.  The checksum MUST be computed and updated for
        the MINT header and metadata at each node that modifies the
        header and/or metadata.  A node MAY perform checksum validation
        before updating the checksum.

   (5) Max Length (8 bits).  Specifies the maximum allowed length of the
   metadata stack in multiples of 4 octets.  This field is initialized
   by the initiator node.  Each node in the path MUST compare the
   current length with the max length, and if the current length equals
   or exceeds the max length, the transit nodes MUST stop inserting
   metadata.


3.4.1 MINT Metadata Header

   The MINT metadata header is always present.



Kumar, et al.                                                  [Page 12]


INTERNET DRAFT                    IFA                     March 11, 2019


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | Request Vector| Action Vector |   Hop Limit   | Current Length|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                  Figure 4 MINT Metadata Header Format


   Request Vector (8 bits) - This vector specifies the presence of
   fields as specified by GNS.  Fields are always 4-octet aligned.  This
   field can be made extensible by defining a new GNS for a MINT Domain.

   Action Vector (8 bits) - This vector specifies node-local or end-to-
   end action on the MINT packets.

   Hop Limit (8 bits) - Specifies the maximum allowed hops in a MINT
   Domain.  This field is initialized by the initiator node.  The hop
   limit MUST be decremented at each hop.  If the incoming hop limit is
   0, current nodes MUST NOT insert metadata.  A value of 0xFF means
   that the Hop limit check MUST be ignored.

   Current Length (8 bits) - Specifies the current length of the
   metadata in multiples of 4 octets.

3.4.2 MINT Checksum Header

   The MINT checksum header is optional.  Presence of the checksum
   header is indicated by the C bit in the flags field of the MINT
   header.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |          Checksum             |           Reserved            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 5 MINT Checksum Header Format


   Checksum (16 bits) - The checksum covers the MINT header and metadata
   stack.  An initiator function node MAY compute the full checksum
   including MINT header and metadata.  Other nodes MAY compute delta
   checksum for the inserted/deleted metadata.

   Reserved (16 bits) - Reserved.  MUST be initialized to 0 on
   transmission and ignored on receipt.



Kumar, et al.                                                  [Page 13]


INTERNET DRAFT                    IFA                     March 11, 2019


3.4.3 MINT Metadata Fragmentation (MF) Header

   The MINT metadata fragmentation (MF) header is optional.  Presence of
   the fragmentation header is indicated by the MF bit in the flags
   field of the MINT header.


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                  Packet ID                        |   MF ID |L|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                     Figure 6 MINT MF Header Format


   Packet ID (26 bits) - Packet identification value generated by the
   initiator node.  This value is node scoped.

   Metadata Fragment ID (5 bits) - The initiator MUST initialize this
   value to 0.  A node performing metadata fragmentation function MUST
   increment the value by 1.

   L (1 bit) - This bit is set by the node creating the last metadata
   fragment.  This will ALWAYS be the terminating function node.  If
   incoming hop limit is 0, the terminating function node will still
   generate a copy/report of the packet and MUST set the L bit.  A
   collector MUST implement mechanism to recover from lost
   packets/reports with L bit set.

   The MF header is a fixed overhead of 4 octets per packet.  A network
   operator MUST identify the need for using MINT metadata
   fragmentation.  The following network conditions can be considered:

   - If a MINT packet may exceed the link or path MTU of the flow path

   - If there are large number of hops in a flow path that could trigger
   link or path MTU breach

   - If the length of metadata creates excessive overhead for
   terminating function node to delete the metadata

   - If each hop needs to generate its own MINT report (postcard
   mechanism)


   With 26 bits of packet id, a maximum datagram lifetime (MDL) of 3
   seconds, and an average Internet mix (IMIX) packet size of 512 bytes,



Kumar, et al.                                                  [Page 14]


INTERNET DRAFT                    IFA                     March 11, 2019


   we get 183.25 Gbps of MINT traffic bit rate per node before the
   packet identifier wraps around.  The collector can use [device id,
   packet id, MF id, L] to rebuild the fragmented packet.

   5 bits of MF id will support 32 metadata fragments.

3.5 MINT Metadata

   The MINT metadata is the information inserted by each hop after the
   MINT header.  The MINT metadata can be inserted at the following
   offsets:

   - Payload Stamping: Immediately after the layer 4 header.  This is
   the default setting.
   - Tail Stamping: After the end of the packet, but before the FCS.
   This is controlled by the TS bit in the flags field of the MINT
   header.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  LNS  |                     Device ID                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      |                                                               |
      ~                LNS/GNS defined metadata (contd)               ~
      .                                                               .
      .                                                               .
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                     Figure 7 MINT Metadata Format

   The MINT metadata header contains a set fields as defined by the name
   space identifier.  Two types of name space identifiers are proposed.

3.5.1 Global Name Space (GNS) Identifier

   A Global Name Space (GNS) is specified in the MINT header by the
   initiator node.  The scope of a GNS is a MINT Domain.  All networking
   elements in a MINT Domain MUST insert metadata as per the GNS ID
   specified in the MINT header.  This is defined as the "Uniform Mode"
   of deployment.

   A GNS value of 0xF indicates that metadata in a MINT Domain is
   defined by the LNS of each hop.

   The advantage of using the uniform mode is having a simple and
   uniform metadata stack.  This means less load on a collector for



Kumar, et al.                                                  [Page 15]


INTERNET DRAFT                    IFA                     March 11, 2019


   parsing.

   The disadvantage is that metadata fields are supported based on the
   least capable networking element in the MINT Domain.

3.5.2 Local Name Space (LNS) Identifier

   A Local Name Space (LNS) is specified in the metadata header.  A GNS
   value of 0xF in the MINT header indicates the presence of an LNS.
   This is defined as the "Non-uniform Mode" of deployment.

   A switch pipeline MUST parse the GNS field in the MINT header.  The
   parsing result will dictate the name space ID that the hop needs to
   comply with.

   The advantage of using the non-uniform mode is having a flexible
   metadata stack.  This allows each hop to include the most relevant
   data for that hop.

   The disadvantage is more complex parsing by a collector.

3.5.3 Device ID

   A 28-bit unique identifier for the device inserting the metadata.  If
   a GNS other than 0xF is present, then the device ID can be expanded
   to a 32 bit value.  This is to support including an IPv4 loopback
   address as a Device ID.


3.6 MINT Network Overhead

   A common problem associated with inserting metadata on a per packet
   per flow basis is the amount of traffic overhead on the network.
   MINT is defined to minimize the overhead on the network.

   MINT Base Header          : 4 octets
   MINT Metadata Header      : 4 octets
   MINT Checksum Header      : 4 octets
   MINT Fragmentation Header : 4 octets


   Minimum Overhead:
      MINT header          : 4 octets
      MINT Metadata Header : 4 octets

      Total Min Overhead  : 8 octets per packet

3.7 MINT Analytics



Kumar, et al.                                                  [Page 16]


INTERNET DRAFT                    IFA                     March 11, 2019


   There are two kinds of actions considered in this proposal.

   (1) Action Bit MAP in MINT Header - This is encoded in the MINT
   header.  Each node in the path MAY use the action bitmap to insert or
   not insert the metadata based on exceeding a locally-specified
   threshold.  Not inserting the metadata is indicated by setting the
   field value to -1 (all 1s).

   (2) Terminating Node Actions - A terminating node may decide to
   perform threshold or other actions on the set of metadata in the
   packet.  This information is not encoded in the MINT header.


3.8 MINT Packet Format

   The MINT header is treated as a layer 3 extension header.  MINT
   header and metadata stack length is reflected in IP total length
   field.  IPv6 extension headers are ordered. The MINT header MUST be
   the last extension header in the IPv6 extension header chain.
   Similarly in case of IPv4 AH/ESP/WESP extension headers, MINT header
   MUST be the last extension header.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                       MPLS Label Stack                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                             ESPL                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                       MINT Extension Label                    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                           MINT Header                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                   MINT Metadata Header                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                    MINT Metadata Stack                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                          Payload                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                            FCS                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      Figure 8 MINT Packet Format

3.8.1 MINT Packet Format with TS Flag Set



Kumar, et al.                                                  [Page 17]


INTERNET DRAFT                    IFA                     March 11, 2019


   In case the Tail Stamp flag is set in the MINT header, the MINT
   metadata header and metadata stack are inserted at the end of the
   packet just before the FCS.  Each node inserts metadata at the bottom
   of MINT metadata stack.

   One of the key advantages of using TS is to support legacy devices
   and/or appliances that need to look at the layer 5 data.  The IP
   length and IP header checksum are updated at each hop inserting
   metadata.  This is the same as without the TS flag.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                       MPLS Label Stack                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                             ESPL                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                       MINT Extension Label                    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                           MINT Header                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                         Payload                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                   MINT Metadata Stack                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      ~                                                               ~
      |                   MINT Metadata Header                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                            FCS                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                  Figure 9 MINT Packet Format with TS

3.9 MINT Load Balancing

   As stated in [RFC7325], regarding use of MPLS fields in multipath
   load balancing:

     Special-purpose labels (label values 0-15) MUST NOT be used.
     Extended special-purpose labels (any label following label 15) MUST
     NOT be used.

   Accordingly, the addition of an ESPL and a MINT Extension Label will
   not cause a different multipath computation from that which is
   calculated in absence of these labels.



Kumar, et al.                                                  [Page 18]


INTERNET DRAFT                    IFA                     March 11, 2019


4. IANA Considerations

   This document requests the following IANA Actions.

4.1.  Extended Special-Purpose MPLS Label Values Registry

   This registry defines a new code point to be added to the Extended
   Special-Purpose MPLS Label Values Registry to identify that a MINT
   extension label is present.
   The following new code point is defined in this draft:

      16  MINT Extension Label

5. Security Considerations

   A successful attack on an OAM protocol can prevent the detection of
   failures or anomalies, or create a false illusion of nonexistent
   ones.

   The metadata elements of MINT can be used by attackers to collect
   information about the network hops.

   Adding MINT headers or adding to MINT metadata can be used to consume
   resources within the path being monitored or by a collector.

   Adding MINT headers or adding to MINT metadata can be used to force
   exceeding the MTU for the path being monitored resulting in
   fragmentation and/or packet drops.

   MINT is expected to be deployed within controlled network domains,
   containing attacks to that controlled domain.  Limiting or preventing
   monitoring or attacks using IFA requires limiting or preventing
   unauthorized access to the domain in which MINT is to be used, and
   preventing leaking IFA metadata beyond the controlled domain.

6. References

6.1  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.



6.2 Informative References

   [RFC6790]  Kompella, K., "The Use of Entropy Labels in MPLS
   Forwarding", RFC 6790, November 2012, <http://www.rfc-



Kumar, et al.                                                  [Page 19]


INTERNET DRAFT                    IFA                     March 11, 2019


   editor.org/info/rfc6790>.

   [RFC7274]  Kompella, K., "Allocating and Retiring Special-Purpose
   MPLS Labels", RFC 7274, June 2014, <http://www.rfc-
   editor.org/info/rfc7274>.

   [RFC7325]  Villamizar, C., "MPLS Forwarding Compliance and
   Performance Requirements", RFC 7325, August 2014, <http://www.rfc-
   editor.org/info/rfc7325>.

   [I-D.draft-kumar-ippm-ifa-01]  Kumar, J., "Inband Flow Analyzer",
   March 2019, <https://tools.ietf.org/html/draft-kumar-ippm-ifa-01>
   (work in progress).

Authors' Addresses

   Jai Kumar
   Broadcom Inc.
   Email: jai.kumar@broadcom.com

   John Lemon
   Broadcom Inc.
   Email: john.lemon@broadcom.com

   Yoav Peleg
   Broadcom Inc.
   Email: yoav.peleg@broadcom.com

   Kireeti Kompella
   Juniper Networks
   Email: kireeti@juniper.net




















Kumar, et al.                                                  [Page 20]