IPPM Working Group                                      G. Fioccola, Ed.
Internet-Draft                                               M. Cociglio
Intended status: Experimental                             Telecom Italia
Expires: September 2, 2018                                      A. Sapio
                                                                R. Sisto
                                                   Politecnico di Torino
                                                           March 1, 2018

 Multipoint Alternate Marking method for passive and hybrid performance


   The Alternate Marking method, as presented in RFC 8321 [RFC8321], can
   be applied only to point-to-point flows because it assumes that all
   the packets of the flow measured on one node are measured again by a
   single second node.  This document aims to generalize and expand this
   methodology to measure any kind of unicast flows, whose packets can
   follow several different paths in the network, in wider terms a
   multipoint-to-multipoint network.  For this reason the technique here
   described is called Multipoint Alternate Marking.  Some definitions
   here introduced extend the scope of RFC 5644 [RFC5644] in the context
   of alternate marking schema.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 2, 2018.

Fioccola, et al.        Expires September 2, 2018               [Page 1]

Internet-Draft                Multipoint AM                   March 2018

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Correlation with RFC5644  . . . . . . . . . . . . . . . . . .   3
   3.  Flow classification . . . . . . . . . . . . . . . . . . . . .   4
   4.  Multipoint Performance Measurement  . . . . . . . . . . . . .   6
     4.1.  Monitoring Network  . . . . . . . . . . . . . . . . . . .   6
     4.2.  Multipoint Packet Loss  . . . . . . . . . . . . . . . . .   7
     4.3.  Network Clustering  . . . . . . . . . . . . . . . . . . .   8
       4.3.1.  Algorithm for Cluster partition . . . . . . . . . . .   9
     4.4.  Multipoint Delay and Delay Variation  . . . . . . . . . .  11
       4.4.1.  Single and Double Marking measurement . . . . . . . .  11
       4.4.2.  Mean Delay  . . . . . . . . . . . . . . . . . . . . .  11
       4.4.3.  Hashing selection method  . . . . . . . . . . . . . .  11
   5.  Examples of application . . . . . . . . . . . . . . . . . . .  13
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  13
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  13
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  14
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  14

1.  Introduction

   The alternate marking method, as presented until now, is applicable
   to a point-to-point path; so the extension proposed in this document
   explains the most general case of multipoint-to-multipoint path.

   The Alternate Marking methodology described in RFC 8321 [RFC8321] has
   the property to synchronize measurements in different points
   maintaining the coherence of the counters.  So it is possible to show
   what is happening in every marking period for each monitored flow.

Fioccola, et al.        Expires September 2, 2018               [Page 2]

Internet-Draft                Multipoint AM                   March 2018

   The monitoring parameters are the packet counter and timestamps of a
   flow for each marking period.

   There are some applications of the alternate marking method where
   there are a lot of monitored flows and nodes.

   For instance, by considering n measurement points and n monitored
   flows, the order of magnitude of the packet counters for each time
   interval is n*n*2 (1 per color).

   Multipoint Alternate Marking aims to reduce this value and makes the
   performance monitoring more flexible in case a detailed analysis is
   not needed.  It can be applied only to unicast flows.

   In some circumstances it is possible to monitor a Multipoint Network
   by analyzing the Network Clustering, without examining in depth.  In
   case there is packet loss or the delay is too high the filtering
   criteria could be specified more in order to perform a per flow
   detailed analysis, as described in RFC 8321 [RFC8321].

   An application could be the software defined network (SDN) paradigm
   where the SDN Controllers are the brains of the network and can
   manage flow control to the switches and routers and, in the same way,
   can calibrate the performance measurements depending on the

2.  Correlation with RFC5644

   RFC 5644 [RFC5644] is limited to active measurements using a single
   source packet or stream, and observations of corresponding packets
   along the path (spatial), at one or more destinations (one-to-group),
   or both.  Instead, the scope of this memo is to define multiparty
   metrics for passive and hybrid measurements in a group-to-group
   topology with multiple sources and destinations.

   RFC 5644 [RFC5644] introduces metric names that can be reused also
   here but have to be extended and rephrased to be applied to the
   alternate marking schema:

   o  the multiparty metrics are not only one-to-group metrics but can
      be also group-to-group metrics;

   o  the spatial metrics, used for measuring the performance of
      segments of a source to destination path, are applied here to
      group-to-group segments (called Clusters).

Fioccola, et al.        Expires September 2, 2018               [Page 3]

Internet-Draft                Multipoint AM                   March 2018

3.  Flow classification

   A flow is identified by all the packets having a set of common
   characteristics.  This definition is inspired by RFC 7011 [RFC7011].

   As an example, by considering a flow as all the packets sharing the
   same source IP address or the same destination IP address, it is easy
   to understand that the resulting pattern will not be a point-to-point
   connection, but a point-to-multipoint or multipoint-to-point

   In general a flow can be defined by a set of selection rules used to
   match a subset of the packets processed by the network device.  These
   rules specify a set of headers fields (Identification Fields) and the
   relative values that must be found in matching packets.

   The choice of the identification fields directly affects the type of
   paths that the flow would follow in the network.  In fact, it is
   possible to relate a set of identification fields with the pattern of
   the resulting graphs, as listed in Figure 1.

   A TCP 5-tuple usually identifies flows following either a single path
   or a point-to-point multipath (in case of load balancing).  On the
   contrary, a single source address selects flows following a point-to-
   multipoint, while a multipoint-to-point can be the result of a
   matching on a single destination address.  In case a selection rule
   and its reverse are used for bidirectional measurements, they can
   correspond to a point-to-multipoint in one direction and a
   multipoint-to-point in the opposite direction.

   In this way the flows to be monitored are selected into the
   monitoring points using packet selection rules, that can also change
   the pattern of the monitored network.

   The alternate marking method is applicable only to a single path (and
   partially to a one-to-one multipath), so the extension proposed in
   this document is suitable also for the most general case of
   multipoint-to-multipoint, which embraces all the other patterns of
   Figure 1.

          point-to-point single path
              +------+      +------+      +------+
          ---<>  R1  <>----<>  R2  <>----<>  R3  <>---
              +------+      +------+      +------+

Fioccola, et al.        Expires September 2, 2018               [Page 4]

Internet-Draft                Multipoint AM                   March 2018

          point-to-point multipath
                          <>  R2  <>
                         / +------+ \
                        /            \
              +------+ /              \ +------+
          ---<>  R1  <>                <>  R4  <>---
              +------+ \              / +------+
                        \            /
                         \ +------+ /
                          <>  R3  <>

                                     <>  R4  <>---
                                    / +------+
                          +------+ /
                         <>  R2  <>
                        / +------+ \
              +------+ /            \ +------+
          ---<>  R1  <>              <>  R5  <>---
              +------+ \              +------+
                        \ +------+
                         <>  R3  <>
                          +------+ \
                                    \ +------+
                                     <>  R6  <>---

          ---<>  R1  <>
              +------+ \
                        \ +------+
                        <>  R4  <>
                        / +------+ \
              +------+ /            \ +------+
          ---<>  R2  <>              <>  R4  <>---
              +------+              / +------+
                          +------+ /
                         <>  R5  <>
                        / +------+
              +------+ /
          ---<>  R3  <>

Fioccola, et al.        Expires September 2, 2018               [Page 5]

Internet-Draft                Multipoint AM                   March 2018

              +------+                +------+
          ---<>  R1  <>              <>  R6  <>---
              +------+ \            / +------+
                        \ +------+ /
                         <>  R4  <>
                          +------+ \
              +------+              \ +------+
          ---<>  R2  <>             <>  R7  <>---
              +------+ \            / +------+
                        \ +------+ /
                         <>  R5  <>
                        / +------+ \
              +------+ /            \ +------+
          ---<>  R3  <>              <>  R8  <>---
              +------+                +------+

                       Figure 1: Flow classification

4.  Multipoint Performance Measurement

   By Using the "traditional" alternate marking method only point-to-
   point paths can be monitored.  To have an IP (TCP/UDP) flow that
   follows a point-to-point path we have to define, with a specific
   value, 5 identification fields (IP Source, IP Destination, Transport
   Protocol, Source Port, Destination Port).

   Multipoint Alternate Marking enables the performance measurement for
   multipoint flows selected by identification fields without any
   constraints (even the entire network production traffic).  It is also
   possible to use multiple marking points for the same monitored flow.

4.1.  Monitoring Network

   The Monitoring Network is deduced from the Production Network, by
   identifying the nodes of the graph that are the measurement points,
   and the links that are the connections between measurement points.

   There are some tecniques that can help with the building of the
   monitoring network (as an example it is possible to mention

   So a graph model of the monitoring network can be built according to
   the alternate marking method: the monitored interfaces and links are
   identified.  Only the measurement points and links where the traffic
   has flowed have to be represented in the graph.

Fioccola, et al.        Expires September 2, 2018               [Page 6]

Internet-Draft                Multipoint AM                   March 2018

   The following figure shows a simple example of a Monitoring Network

                                                   <>  R6  <>---
                                                  / +------+
                           +------+     +------+ /
                          <>  R2  <>---<>  R4  <>
                         / +------+ \   +------+ \
                        /            \            \ +------+
              +------+ /   +------+   \ +------+   <>  R7  <>---
          ---<>  R1  <>---<>  R3  <>---<>  R5  <>   +------+
              +------+ \   +------+ \   +------+ \
                        \            \            \ +------+
                         \            \            <>  R8  <>---
                          \            \            +------+
                           \            \
                            \            \ +------+
                             \            <>  R9  <>---
                              \            +------+
                                \ +------+
                                 <>  R10 <>---

                    Figure 2: Monitoring Network Graph

   Each monitoring point is characterized by the packet counter that
   refers only to a marking period of the monitored flow.

   The same is applicable also for the delay but it will be described in
   the following sections.

4.2.  Multipoint Packet Loss

   Since all the packets of the considered flow leaving the network have
   previously entered the network, the number of packets counted by all
   the input nodes is always greater or equal than the number of packets
   counted by all the output nodes.

   And in case of no packet loss occurring in the marking period, if all
   the input and output points of the network domain to be monitored are
   measurement points, the number of packets is the same on all the
   ingress interfaces and on all the egress interfaces.  The

Fioccola, et al.        Expires September 2, 2018               [Page 7]

Internet-Draft                Multipoint AM                   March 2018

   intermediate measurement points have only the task to split the

   It is possible to define the Network Packet Loss (for 1 flow, for 1
   period): <<In a packet network, the number of lost packets is the
   number of packets counted by the input nodes minus the number of
   packets counted by the output nodes>>.  This is true for every packet
   flow in each marking period.

   The Monitored Network Packet Loss with n input nodes and m output
   nodes is given by:

   PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)


   PL is the Network Packet Loss (number of lost packets)

   PIi is the Number of packets flowed through the i-th Input node in
   this period

   POj is the Number of packets flowed through the j-th Output node in
   this period

4.3.  Network Clustering

   The previous Equation can determine the number of packets lost
   globally in the monitored network, exploiting only the data provided
   by the counters in the input and output nodes.

   In addition it is also possible to leverage the data provided by the
   other counters in the network to converge on the smallest
   identifiable subnetworks where the losses occur.  These subnetworks
   are named Clusters.

   A Cluster graph is a subnetwork of the entire Monitoring Network
   graph that still satisfies the packet loss equation where PL in this
   case is the number of packets lost in the Cluster.

   For this reason a Cluster should contain all the arcs emanating from
   its input nodes and all the arcs terminating at its output nodes.
   This ensures that we can count all the packets (and only those)
   exiting an input node again at the output node, whatever path they

   In a completely monitored network (a network where every network
   interface is monitored), each network device corresponds to a Cluster

Fioccola, et al.        Expires September 2, 2018               [Page 8]

Internet-Draft                Multipoint AM                   March 2018

   and each physical link corresponds to two Clusters (one for each

   Clusters can have different sizes depending on flow filtering
   criteria adopted.

   Moreover, sometimes Clusters can be optionally simplified.  For
   example when two monitored interfaces are divided by a single router
   (one is the input interface and the other is the output interface and
   the router has only these two interfaces), instead of counting
   exactly twice, upon entering and leaving, it is possible to consider
   a single measurement point (in this case we do not care of the
   internal packet loss of the router).

4.3.1.  Algorithm for Cluster partition

   A simple algorithm can be applied in order to split our monitoring
   network into Clusters.  It is a two-step algorithm:

   o  Group the links where there is the same starting node;

   o  Join the grouped links with at least one ending node in common.

   In our monitoring network graph example it is possible to identify 4
   Clusters, by applying the previous algorithm:

          Cluster 1
                          <>  R2  <>---
                         / +------+
              +------+ /   +------+
          ---<>  R1  <>---<>  R3  <>---
              +------+ \   +------+
                                \ +------+
                                 <>  R10 <>---

Fioccola, et al.        Expires September 2, 2018               [Page 9]

Internet-Draft                Multipoint AM                   March 2018

          Cluster 2
              +------+     +------+
          ---<>  R2  <>---<>  R4  <>---
              +------+ \   +------+
              +------+   \ +------+
          ---<>  R3  <>---<>  R5  <>---
              +------+ \   +------+
                            \ +------+
                             <>  R9  <>---

          Cluster 3
                         <>  R6  <>---
                        / +------+
              +------+ /
          ---<>  R4  <>
              +------+ \
                        \ +------+
                         <>  R7  <>---

          Cluster 4
          ---<>  R5  <>
              +------+ \
                        \ +------+
                         <>  R8  <>---

                        Figure 3: Clusters example

   There are Clusters with more than 2 nodes and two-nodes Clusters.  In
   the two-nodes Clusters the loss is on the link (Cluster 4).  In more-
   than-2-nodes Clusters the loss is on the Cluster but we cannot know
   in which link (Cluster 1, 2, 3).

   Obviously, by combining some Clusters in a new connected subnetwork
   (called Super Cluster) the Packet Loss Rule is still true.

Fioccola, et al.        Expires September 2, 2018              [Page 10]

Internet-Draft                Multipoint AM                   March 2018

4.4.  Multipoint Delay and Delay Variation

   The same line of reasoning can be applied to Delay and Delay
   Variation.  It is important to highlight that both delay and delay
   variation measurements make sense in a multipoint path.  It is
   possible to perform delay and delay variation measurements on
   multipoint paths basis or on single packets basis.  In the latter
   case, you can use multipoint path just to easily couple packects
   between inputs and output nodes of a multipoint path, as it is
   described in the following sections.

4.4.1.  Single and Double Marking measurement

   Delay and delay variation measurements relative to a picked packet
   (both single and double marked) cannot be performed in the Multipoint
   scenario, since they would not be representative of the entire flow.
   The packets can follow different paths with various delays and in
   general it is very difficult to recognize a marked packet in a
   multipoint-to-multipoint path.

4.4.2.  Mean Delay

   Mean delay and delay variation measurements can also be generalized
   to the case of multipoint flows.  It is possible to compute the
   average one-way delay of packets, in one block, in a cluster or in
   the entire monitored network.

   The average latency can be measured as the difference between the
   weighted averages of the mean timestamps of the sets of output and
   input nodes.

4.4.3.  Hashing selection method

   RFC 5474 [RFC5474] and RFC 5475 [RFC5475] introduce sampling and
   filtering techniques for IP Packet Selection.

   The hash-based selection methodologies for delay measurement can work
   in a multipoint-to-multipoint path and can be used both coupled to
   mean delay or stand alone.

   [I-D.mizrahi-ippm-compact-alternate-marking] introduces how to use
   the Hash method combined with alternate marking method for point-to-
   point flows.  It is also called Mixed Hashed Marking: the coupling of
   marking method and hashing technique is very useful because the
   marking batches anchor the samples selected with hashing and this
   simplifies the correlation of the hashing packets along the path.

Fioccola, et al.        Expires September 2, 2018              [Page 11]

Internet-Draft                Multipoint AM                   March 2018

   It is possible to use a basic hash or a dynamic hash method.  One of
   the challenges of the basic approach is that the frequency of the
   sampled packets may vary considerably.  For this reason the dynamic
   approach has been introduced for point-to-point flow in order to have
   the desired and almost fixed number of samples for each measurement
   period.  In the hash-based sampling, alternate marking is used to
   create periods, so that hash-based samples are divided into batches,
   allowing to anchor the selected samples to their period.  Moreover in
   the dynamic hash-based sampling, by dynamically adapting the length
   of the hash value, the number of samples is bounded in each marking
   period.  This can be realized by choosing the maximum number of
   samples (NMAX) to be catched in a marking period.  The algorithm
   starts with only few hash bits, that permit to select a greater
   percentage of packets (e.g. with 0 bit of hash all the packets are
   sampled, with 1 bit of hash half of the packets are sampled, and so
   on).  When the number of selected packets reaches NMAX, a hashing bit
   is added.  As a consequence, the sampling proceeds at half of the
   original rate and also the packets already selected that don't match
   the new hash are discarded.  This step can be repeated iteratively.
   It is assumed that each sample includes the timestamp (used for delay
   measurement) and the hash value, allowing the management system to
   match the samples received from the two measurement points.  The
   dynamic process statistically converges at the end of a marking
   period and the final number of selected samples is between NMAX/2 and
   NMAX.  Therefore, the dynamic approach paces the sampling rate,
   allowing to bound the number of sampled packets per sampling period.

   In a multipoint environment the behaviour is similar to point-to
   point flow.  In particular, in the context of multipoint-to-
   multipoint flow, the dynamic hash could be the solution to perform
   delay measurements on specific packets and to overcome the single and
   double marking limitations.

   The management system receives the samples including the timestamps
   and the hash value from all the MPs, and this happens both for point-
   to-point and for multipoint-to-multipoint flow.  Then the longest
   hash used by MPs is deduced and it is applied to couple timestamps of
   same packets of 2 MPs of a point-to-point path or of input and output
   MPs of a Cluster (or a Super Cluster or the entire network).  But
   some considerations are needed: if there isn't packet loss the set of
   input samples is always equal to the set of output samples.  In case
   of packet loss the set of output samples can be a subset of input
   samples but the method still works because, at the end, it is easy to
   couple the input and output timestamps of each catched packet using
   the hash (in particular the "unused part of the hash" that should be
   different for each packet).

Fioccola, et al.        Expires September 2, 2018              [Page 12]

Internet-Draft                Multipoint AM                   March 2018

5.  Examples of application

   There are three application fields where it may be useful to take
   into consideration the Multipoint Alternate Marking:

   o  VPN: The IP traffic is selected on IP source basis in both
      directions.  At the end point WAN interface all the output traffic
      is counted in a single flow.  The input traffic is composed by all
      the other flows aggregated for source address.  So, by considering
      n end-points, the monitored flows are n (each flow with 1 ingress
      point and (n-1) egress points) instead of n*(n-1) flows (each
      flow, with 1 ingress point and 1 egress point);

   o  Mobile Backhaul: LTE traffic is selected, in the Up direction, by
      the EnodeB source address and, in Down direction, by the EnodeB
      destination address because the packets are sent from the Mobile
      Packet Core to the EnodeB.  So the monitored flow is only one per
      EnodeB in both directions;

   o  OTT(Over The Top) services: The traffic is selected, in the Down
      direction by the source addresses of the packets sent by OTT
      Servers.  In the opposite direction (Up) by the destination IP
      addresses of the same Servers.  So the monitoring is based on a
      single flow per OTT Servers in both directions.

6.  Security Considerations


7.  Acknowledgements


8.  IANA Considerations


9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

Fioccola, et al.        Expires September 2, 2018              [Page 13]

Internet-Draft                Multipoint AM                   March 2018

   [RFC5644]  Stephan, E., Liang, L., and A. Morton, "IP Performance
              Metrics (IPPM): Spatial and Multicast", RFC 5644,
              DOI 10.17487/RFC5644, October 2009,

   [RFC8321]  Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli,
              L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi,
              "Alternate-Marking Method for Passive and Hybrid
              Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321,
              January 2018, <https://www.rfc-editor.org/info/rfc8321>.

9.2.  Informative References

              Alvarez-Hamelin, J., Morton, A., and J. Fabini, "Advanced
              Unidirectional Route Assessment", draft-amf-ippm-route-01
              (work in progress), October 2017.

              Mizrahi, T., Arad, C., Fioccola, G., Cociglio, M., Chen,
              M., Zheng, L., and G. Mirsky, "Compact Alternate Marking
              Methods for Passive Performance Monitoring", draft-
              mizrahi-ippm-compact-alternate-marking-00 (work in
              progress), October 2017.

   [RFC5474]  Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,
              Grossglauser, M., and J. Rexford, "A Framework for Packet
              Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,
              March 2009, <https://www.rfc-editor.org/info/rfc5474>.

   [RFC5475]  Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
              Raspall, "Sampling and Filtering Techniques for IP Packet
              Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,

   [RFC7011]  Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
              "Specification of the IP Flow Information Export (IPFIX)
              Protocol for the Exchange of Flow Information", STD 77,
              RFC 7011, DOI 10.17487/RFC7011, September 2013,

Authors' Addresses

Fioccola, et al.        Expires September 2, 2018              [Page 14]

Internet-Draft                Multipoint AM                   March 2018

   Giuseppe Fioccola (editor)
   Telecom Italia
   Via Reiss Romoli, 274
   Torino  10148

   Email: giuseppe.fioccola@telecomitalia.it

   Mauro Cociglio
   Telecom Italia
   Via Reiss Romoli, 274
   Torino  10148

   Email: mauro.cociglio@telecomitalia.it

   Amedeo Sapio
   Politecnico di Torino
   Corso Duca degli Abruzzi, 24
   Torino  10129

   Email: amedeo.sapio@polito.it

   Riccardo Sisto
   Politecnico di Torino
   Corso Duca degli Abruzzi, 24
   Torino  10129

   Email: riccardo.sisto@polito.it

Fioccola, et al.        Expires September 2, 2018              [Page 15]