Active and Passive Metrics and Methods (and everything in-between, or Hybrid)
draft-ietf-ippm-active-passive-00

The information below is for an old version of the document
Document Type Active Internet-Draft (ippm WG)
Author Al Morton 
Last updated 2015-06-30
Stream Internet Engineering Task Force (IETF)
Formats pdf htmlized bibtex
Reviews
Stream WG state WG Document
Document shepherd No shepherd assigned
IESG IESG state I-D Exists
Consensus Boilerplate Unknown
Telechat date
Responsible AD (None)
Send notices to (None)
Network Working Group                                          A. Morton
Internet-Draft                                                 AT&T Labs
Intended status: Informational                             June 30, 2015
Expires: January 1, 2016

 Active and Passive Metrics and Methods (and everything in-between, or
                                Hybrid)
                   draft-ietf-ippm-active-passive-00

Abstract

   This memo provides clear definitions for Active and Passive
   performance assessment.  The construction of Metrics and Methods can
   be described as Active or Passive.  Some methods may use a subset of
   both active and passive attributes, and we refer to these as Hybrid
   Methods.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 1, 2016.

Copyright Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of

Morton                   Expires January 1, 2016                [Page 1]
Internet-Draft             Active and Passive                  June 2015

   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
   2.  Purpose and Scope . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Terms and Definitions . . . . . . . . . . . . . . . . . . . .   3
     3.1.  Performance Metric  . . . . . . . . . . . . . . . . . . .   3
     3.2.  Method of Measurement . . . . . . . . . . . . . . . . . .   4
     3.3.  Observation Point . . . . . . . . . . . . . . . . . . . .   4
     3.4.  Active Methods  . . . . . . . . . . . . . . . . . . . . .   4
     3.5.  Active Metric . . . . . . . . . . . . . . . . . . . . . .   4
     3.6.  Passive Methods . . . . . . . . . . . . . . . . . . . . .   5
     3.7.  Passive Metric  . . . . . . . . . . . . . . . . . . . . .   5
     3.8.  Hybrid Methods and Metrics  . . . . . . . . . . . . . . .   6
   4.  Discussion  . . . . . . . . . . . . . . . . . . . . . . . . .   6
     4.1.  Graphical Representation  . . . . . . . . . . . . . . . .   6
     4.2.  Discussion of PDM . . . . . . . . . . . . . . . . . . . .   8
     4.3.  Discussion of "Coloring" Method . . . . . . . . . . . . .   9
   5.  Security considerations . . . . . . . . . . . . . . . . . . .   9
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  10
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  10
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   The adjectives "active" and "passive" have been used for many years
   to distinguish two different classes of Internet performance
   assessment.  The first Passive and Active Measurement (PAM)
   Conference was held in 2000, but the earliest proceedings available
   on-line are from the second PAM conference in 2001
   [https://www.ripe.net/ripe/meetings/pam-2001].

   The notions of "active" and "passive" are well-established.  In
   general:

      An Active metric or method depends on a dedicated measurement
      packet stream and observations of the stream.

      A Passive metric or method depends *solely* on observation of one
      or more existing packet streams.  The streams only serve
      measurement when they are observed for that purpose, and are
      present whether measurements take place or not.

Morton                   Expires January 1, 2016                [Page 2]
Internet-Draft             Active and Passive                  June 2015

   As new techniques for assessment emerge it is helpful to have clear
   definitions of these notions.  This memo provides more detailed
   definitions, defines a new category for combinations of traditional
   active and passive techniques, and discusses means to evaluate new
   techniques as they emerge.

   This memo provides definitions for Active and Passive Metrics and
   Methods based on long usage in the Internet measurement community,
   and especially the Internet Engineering Task Force.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

2.  Purpose and Scope

   The scope of this memo is to define and describe Active and Passive
   versions of metrics and methods which are consistent with the long-
   time usage of these adjectives in the Internet measurement community
   and especially the Internet Engineering Task Force.  Since the
   science of measurement is expanding, we provide a category for
   combinations of the traditional extremes, treating Active and Passive
   as a continuum and designating combinations of their attributes as
   Hybrid methods.

   Further, this memo's purpose includes describing multiple dimensions
   in which to evaluate methods as they emerge.

3.  Terms and Definitions

   This section defines the key terms of the memo.  Some definitions use
   the notion of "stream of interest" which is synonymous with
   "population of interest" defined in clause 6.1.1 of ITU-T
   Recommendation Y.1540.  The definitions are consistent with
   [I-D.zheng-ippm-framework-passive].

3.1.  Performance Metric

   The standard definition of a quantity, produced in an assessment of
   performance and/or reliability of the network, which has an intended
   utility and is carefully specified to convey the exact meaning of a
   measured value.  (This definition is consistent with that of
   Performance Metric in RFC 2330 and RFC 6390).

Morton                   Expires January 1, 2016                [Page 3]
Internet-Draft             Active and Passive                  June 2015

3.2.  Method of Measurement

   The procedure or set of operations having the object of determining a
   Measured Value or Measurement Result.

3.3.  Observation Point

   See section 2 of [RFC7011] for this definition (a location in the
   network where packets can be observed), and related definitions.  The
   comparable term defined in IETF literature on Active measurement is
   Measurement Point, see section 4.1 of [RFC5835].  Two terms have come
   into use describing similar actions at the identified point in the
   network path.

3.4.  Active Methods

   Active measurement methods have the following attributes:

   1.  Commonly, the packet stream of interest is generated as the basis
       of measurement.  Another packet stream may be generated to
       increase traffic load, but the loading stream itself may not be
       measured.

   2.  The packets in the stream of interest have fields (or are
       augmented or modified to include fields) which are dedicated to
       measurement.  Since measurement usually requires determining the
       corresponding packets at multiple measurement points, a sequence
       number is the most common information dedicated to measurement.

   3.  The Source and Destination of the packet stream of interest are
       usually known a' priori.

   4.  The characteristics of the packet stream of interest are known at
       the Source at least, and may be communicated to Destination as
       part of the method.

   When adding traffic to the network for measurement, Active Methods
   influence the quantities measured to some degree, and those
   performing tests should take steps to quantify the effect(s) and/or
   minimize such effects.

3.5.  Active Metric

   An Active Metric incorporates one or more of the aspects of Active
   Methods in the metric definition.

   For example, IETF metrics for IP performance (developed according to
   the [RFC2330] framework) include the Source packet stream

Morton                   Expires January 1, 2016                [Page 4]
Internet-Draft             Active and Passive                  June 2015

   characteristics as metric input parameters, and also specify the
   packet characteristics (Type-P) and Source and Destination IP
   addresses (with their implications on both stream treatment and
   interfaces associated with measurement points).

3.6.  Passive Methods

   Passive measurement methods are

   o  based solely on observations of undisturbed and unmodified packet
      stream of interest

   o  dependent on the existence of one or more packet streams to supply
      the stream of interest

   o  dependent on the presence of the packet stream of interest at one
      or more designated observation points.

   Some passive methods simply observe and collect information on all
   packets that pass Observation Point(s), while others filter the
   packets as a first step and only collect information on packets that
   match the filter criteria, and thereby narrow the stream of interest.

   It is common that passive methods are conducted at one or more
   Observation Points.  Passive methods to assess Performance Metrics
   often require multiple observation points, e.g., to assess latency of
   packet transfer across a network path between two Observation Points.
   In this case, the observed packets must include enough information to
   determine the corresponding packets at different Observation Points.

   Communication of the observations (in some form) to a collector is an
   essential aspect of Passive Methods.  In some configurations, the
   traffic load associated with results export to a collector may
   influence the network performance.  However, the collection of
   results is not unique to Passive Methods, and the load from
   management and operations of measurement systems must always be
   considered for potential effects on the measured values.

3.7.  Passive Metric

   Passive Metrics apply to observations of packet traffic (traffic
   flows in [RFC7011]).

   Passive performance metrics are assessed independent of the packets
   or traffic flows, and solely through observation.  Some refer to such
   assessments as "out-of-band".

Morton                   Expires January 1, 2016                [Page 5]
Internet-Draft             Active and Passive                  June 2015

   One example of passive performance metrics for IP packet transfer can
   be found in ITU-T Recommendation Y.1540, where the metrics are
   defined on the basis of reference events as packet pass reference
   points, so the metrics are agnostic to the distinction between active
   and passive when the necessary packet correspondence can be derived
   from the observed stream of interest when required.

3.8.  Hybrid Methods and Metrics

   Methods of Measurement which use a combination of Active Methods and
   Passive Methods, to assess Active Metrics, Passive Metrics, or new
   metrics derived from the a' priori knowledge and observations of the
   stream of interest.  ITU-T Recommendation Y.1540 defines metrics are
   applicable to the hybrid category, since packet correspondence at
   different observation/reference points could be derived from "fields
   which are dedicated to measurement", but otherwise the methods are
   passive.

   With respect to the stream of interest, Hybrid methods fit in the
   continuum as follows, in terms of what happens at the Source(or
   Observation Point nearby):

   o  If you generate the stream of interest => Active

   o  If you augment of modify a stream of interest => Hybrid

   o  If you solely observe a stream of interest => Passive

4.  Discussion

   This section illustrates the definitions and presents some examples.

4.1.  Graphical Representation

   If we compare the Active and Passive Methods, there are at least two
   dimensions on which methods can be evaluated.  This evaluation space
   may be useful when a method is a combination of the two alternative
   methods.

   The two dimensions (initially chosen) are:

   1.  The degree to which the stream of interest effects overall
       network conditions experienced by that stream and other streams.
       This is a key dimension for Active measurement error analysis.
       (Comment: There is also the notion of time averages - a
       measurement stream may have significant effect while it is
       present, but the stream is only generated 0.1% of the time.  On
       the other hand, observations alone have no effect on network

Morton                   Expires January 1, 2016                [Page 6]
Internet-Draft             Active and Passive                  June 2015

       performance.  To keep things simple, we consider the stream
       effect only when it is present.)

   2.  The degree to which stream characteristics are know a' priori.
       There are methodological advantages of knowing the source stream
       characteristics, and having complete control of the stream
       characteristics.  For example, knowing the number of packets in a
       stream allows more efficient operation of the measurement
       receiver, and so is an asset for active measurement methods.
       Passive methods (with no sample filter) have few clues available
       to anticipate what the protocol first packet observed will use or
       how many packets will comprise the flow, but once the standard
       protocol of a flow is known the possibilities narrow (for some
       compliant flows).  Therefore this is a key dimension for Passive
       measurement error analysis.

   There are a few examples we can plot on a two-dimensional space.  We
   can anchor the dimensions with reference point descriptions.

   Effect of the measured stream on network conditions
   ^ Max
   |* Active using max capacity stream
   |
   |
   |
   |
   |* Active using stream with load of typical user
   |
   |
   |
   |* Active using extremely sparse, randomized stream
   |                             * PDM                        Passive
   | Min                                                            *
   +----------------------------------------------------------------|
   |                                                                |
   Stream                                                   No Stream
   Characteristics                                    Characteristics
   completely                                                   Known
   known

   We recognize that method categorization could be based on additional
   dimensions, but this would require a different graphical approach.

   For example, "effect of stream of interest on network conditions"
   could easily be further qualified into:

Morton                   Expires January 1, 2016                [Page 7]
Internet-Draft             Active and Passive                  June 2015

   1.  effect on the performance of the stream of interest itself: for
       example, choosing a packet marking or DSCP resulting in domain
       treatment as a real-time stream (as opposed to default/best-
       effort marking.

   2.  effect on unmeasured streams that share the path and/or
       bottlenecks: for example, an extremely sparse measured stream of
       minimal size packets typically has little effect on other flows
       (and itself), while a stream designed to characterize path
       capacity may effect all other flows passing through the capacity
       bottleneck (including itself).

   3.  effect on network conditions resulting in network adaptation: for
       example, a network monitoring load and congestion conditions
       might change routing, placing some flows to alternate paths to
       mitigate the congestion.

   At present, we have combined 1 and 2 on one axis, as examination of
   examples indicates strong correlation of affects on this pair, and
   network adaptation is not addressed.  As suggestions emerge we will
   re-examine the possibilities.

   It is apparent that different methods of IP network measurement can
   produce different results, even when measuring the same path at the
   same time.  The two dimensions of the graph help to understand how
   the results might change with the method chosen.  For example, an
   Active Method to assess throughput adds some amount of traffic to the
   network which might result in lower throughput for all streams.
   However, a Passive Method to assess throughput can also err on the
   low side due to unknown limitations of the hosts providing traffic,
   competition for host resources, limitations of the network interface,
   or private sub-networks that are not an intentional part of the path,
   etc.  And Hybrid Methods could easily suffer from both forms of
   error.  Another example of potential errors stems from the pitfalls
   of using an Active stream with known bias, such as a periodic stream
   defined in [RFC3432].  The strength of modelling periodic streams
   (like VoIP) is a potential weakness when extending the measured
   results to other application whose streams are non-periodic.  The
   solutions are to model the application streams more exactly with an
   Active Method, or accept the risks and potential errors with the
   Passive Method discussed above.

4.2.  Discussion of PDM

   In [I-D.ietf-ippm-6man-pdm-option], an IPv6 Option Header for
   Performance and Diagnostic Measurements (PDM) is described which
   (when added to the stream of interest at strategic interfaces)
   supports performance measurements.  This method processes a user

Morton                   Expires January 1, 2016                [Page 8]
Internet-Draft             Active and Passive                  June 2015

   traffic stream and adds "fields which are dedicated to measurement".
   Thus:

   o  The method may have a small effect on the measured stream and
      other streams in the network.

   o  The measured stream has unknown characteristics until it is
      processed to add the PDM Option header.

   We conclude that this is a Hybrid method, having at least one
   characteristic of both active and passive methods.

4.3.  Discussion of "Coloring" Method

   Draft [I-D.tempia-opsawg-p3m], proposed to color packets by re-
   writing a field of the stream at strategic interfaces to support
   performance measurements.  This method processes a user traffic
   stream and inserts "fields which are dedicated to measurement".
   Thus:

   o  The method may have a small effect on the measured stream and
      other streams in the network (smaller than PDM above).

   o  The measured stream has unknown characteristics until it is
      processed to add the coloring in the header, and the stream could
      be measured and time-stamped during that process.

   We note that [I-D.chen-ippm-coloring-based-ipfpm-framework] proposes
   a method similar to [I-D.tempia-opsawg-p3m], and ippm-list discussion
   indicates [I-D.chen-ippm-coloring-based-ipfpm-framework] may be
   covered by the same IPR as [I-D.tempia-opsawg-p3m].

   We conclude that this is a Hybrid method, having at least one
   characteristic of both active and passive methods.

5.  Security considerations

   When considering privacy of those involved in measurement or those
   whose traffic is measured, there is sensitive information
   communicated and observed at observation and measurement points
   described above.  We refer the reader to the privacy considerations
   described in the Large Scale Measurement of Broadband Performance
   (LMAP) Framework [I-D.ietf-lmap-framework], which covers active and
   passive measurement techniques and supporting material on measurement
   context.

Morton                   Expires January 1, 2016                [Page 9]
Internet-Draft             Active and Passive                  June 2015

6.  IANA Considerations

   This memo makes no requests for IANA consideration.

7.  Acknowledgements

   Thanks to Mike Ackermann for asking the right question, and for
   several suggestions on terminology.  Brian Trammell provided key
   terms and references for the passive category.  Tiziano Ionta
   reviewed the draft and suggested the classification for the
   "coloring" method of measurement.  Nalini Elkins identified several
   areas for clarification following her review.  Bill Jouris reviewed
   01 editorially and suggested several improvements.

8.  References

8.1.  Normative References

   [RFC2330]  Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
              "Framework for IP Performance Metrics", RFC 2330, May
              1998.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3432]  Raisanen, V., Grotefeld, G., and A. Morton, "Network
              performance measurement with periodic streams", RFC 3432,
              November 2002.

   [RFC5835]  Morton, A. and S. Van den Berghe, "Framework for Metric
              Composition", RFC 5835, April 2010.

   [RFC7011]  Claise, B., Trammell, B., and P. Aitken, "Specification of
              the IP Flow Information Export (IPFIX) Protocol for the
              Exchange of Flow Information", STD 77, RFC 7011, September
              2013.

8.2.  Informative References

   [I-D.ietf-lmap-framework]
              Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
              Aitken, P., and A. Akhter, "A framework for Large-Scale
              Measurement of Broadband Performance (LMAP)", draft-ietf-
              lmap-framework-14 (work in progress), April 2015.

Morton                   Expires January 1, 2016               [Page 10]
Internet-Draft             Active and Passive                  June 2015

   [I-D.ietf-ippm-6man-pdm-option]
              Elkins, N. and M. Ackermann, "IPv6 Performance and
              Diagnostic Metrics (PDM) Destination Option", draft-ietf-
              ippm-6man-pdm-option-00 (work in progress), June 2015.

   [I-D.tempia-opsawg-p3m]
              Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda,
              "A packet based method for passive performance
              monitoring", draft-tempia-opsawg-p3m-04 (work in
              progress), February 2014.

   [I-D.chen-ippm-coloring-based-ipfpm-framework]
              Chen, M., Zheng, L., Mirsky, G., and G. Fioccola, "IP Flow
              Performance Measurement Framework", draft-chen-ippm-
              coloring-based-ipfpm-framework-03 (work in progress),
              February 2015.

   [I-D.zheng-ippm-framework-passive]
              Zheng, L., Elkins, N., Lingli, D., Ackermann, M., and G.
              Mirsky, "Framework for IP Passive Performance
              Measurements", draft-zheng-ippm-framework-passive-03 (work
              in progress), February 2015.

Author's Address

   Al Morton
   AT&T Labs
   200 Laurel Avenue South
   Middletown, NJ
   USA

   Email: acmorton@att.com

Morton                   Expires January 1, 2016               [Page 11]