Active and Passive Metrics and Methods (and everything in-between, or Hybrid)
draft-ietf-ippm-active-passive-01
The information below is for an old version of the document.
| Document | Type | Active Internet-Draft (ippm WG) | |
|---|---|---|---|
| Author | Al Morton | ||
| Last updated | 2015-09-06 | ||
| Stream | Internet Engineering Task Force (IETF) | ||
| Formats | plain text xml htmlized pdfized bibtex | ||
| Reviews |
SECDIR Last Call review
(of
-04)
Has Nits
GENART Last Call review
(of
-04)
Almost Ready
OPSDIR Last Call review
(of
-04)
Has Nits
|
||
| Stream | WG state | WG Document | |
| Document shepherd | (None) | ||
| IESG | IESG state | I-D Exists | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ietf-ippm-active-passive-01
Network Working Group A. Morton
Internet-Draft AT&T Labs
Intended status: Informational September 6, 2015
Expires: March 9, 2016
Active and Passive Metrics and Methods (and everything in-between, or
Hybrid)
draft-ietf-ippm-active-passive-01
Abstract
This memo provides clear definitions for Active and Passive
performance assessment. The construction of Metrics and Methods can
be described as Active or Passive. Some methods may use a subset of
both active and passive attributes, and we refer to these as Hybrid
Methods.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 9, 2016.
Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
Morton Expires March 9, 2016 [Page 1]
Internet-Draft Active and Passive September 2015
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 3
3. Terms and Definitions . . . . . . . . . . . . . . . . . . . . 3
3.1. Performance Metric . . . . . . . . . . . . . . . . . . . 3
3.2. Method of Measurement . . . . . . . . . . . . . . . . . . 4
3.3. Observation Point . . . . . . . . . . . . . . . . . . . . 4
3.4. Active Methods . . . . . . . . . . . . . . . . . . . . . 4
3.5. Active Metric . . . . . . . . . . . . . . . . . . . . . . 4
3.6. Passive Methods . . . . . . . . . . . . . . . . . . . . . 5
3.7. Passive Metric . . . . . . . . . . . . . . . . . . . . . 5
3.8. Hybrid Methods and Metrics . . . . . . . . . . . . . . . 6
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.1. Graphical Representation . . . . . . . . . . . . . . . . 7
4.2. Discussion of PDM . . . . . . . . . . . . . . . . . . . . 9
4.3. Discussion of "Coloring" Method . . . . . . . . . . . . . 10
5. Security considerations . . . . . . . . . . . . . . . . . . . 10
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.1. Normative References . . . . . . . . . . . . . . . . . . 11
8.2. Informative References . . . . . . . . . . . . . . . . . 11
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12
1. Introduction
The adjectives "active" and "passive" have been used for many years
to distinguish two different classes of Internet performance
assessment. The first Passive and Active Measurement (PAM)
Conference was held in 2000, but the earliest proceedings available
on-line are from the second PAM conference in 2001
[https://www.ripe.net/ripe/meetings/pam-2001].
The notions of "active" and "passive" are well-established. In
general:
An Active metric or method depends on a dedicated measurement
packet stream and observations of the stream.
A Passive metric or method depends *solely* on observation of one
or more existing packet streams. The streams only serve
measurement when they are observed for that purpose, and are
present whether measurements take place or not.
Morton Expires March 9, 2016 [Page 2]
Internet-Draft Active and Passive September 2015
As new techniques for assessment emerge it is helpful to have clear
definitions of these notions. This memo provides more detailed
definitions, defines a new category for combinations of traditional
active and passive techniques, and discusses means to evaluate new
techniques as they emerge.
This memo provides definitions for Active and Passive Metrics and
Methods based on long usage in the Internet measurement community,
and especially the Internet Engineering Task Force. This memo also
describes the comnination of fundamental Active and Passive
categories, which are called Hybrid Methods and Metrics.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. Purpose and Scope
The scope of this memo is to define and describe Active and Passive
versions of metrics and methods which are consistent with the long-
time usage of these adjectives in the Internet measurement community
and especially the Internet Engineering Task Force. Since the
science of measurement is expanding, we provide a category for
combinations of the traditional extremes, treating Active and Passive
as a continuum and designating combinations of their attributes as
Hybrid methods.
Further, this memo's purpose includes describing multiple dimensions
in which to evaluate methods as they emerge.
3. Terms and Definitions
This section defines the key terms of the memo. Some definitions use
the notion of "stream of interest" which is synonymous with
"population of interest" defined in clause 6.1.1 of ITU-T
Recommendation Y.1540 [Y.1540]. The definitions are consistent with
[I-D.zheng-ippm-framework-passive].
3.1. Performance Metric
The standard definition of a quantity, produced in an assessment of
performance and/or reliability of the network, which has an intended
utility and is carefully specified to convey the exact meaning of a
measured value. (This definition is consistent with that of
Performance Metric in RFC 2330 and RFC 6390).
Morton Expires March 9, 2016 [Page 3]
Internet-Draft Active and Passive September 2015
3.2. Method of Measurement
The procedure or set of operations having the object of determining a
Measured Value or Measurement Result.
3.3. Observation Point
See section 2 of [RFC7011] for this definition (a location in the
network where packets can be observed), and related definitions. The
comparable term defined in IETF literature on Active measurement is
Measurement Point, see section 4.1 of [RFC5835]. Two terms have come
into use describing similar actions at the identified point in the
network path.
3.4. Active Methods
Active measurement methods have the following attributes:
1. Commonly, the packet stream of interest is generated as the basis
of measurement. Another packet stream may be generated to
increase traffic load, but the loading stream itself may not be
measured.
2. The packets in the stream of interest have fields (or are
augmented or modified to include fields) which are dedicated to
measurement. Since measurement usually requires determining the
corresponding packets at multiple measurement points, a sequence
number is the most common information dedicated to measurement.
3. The Source and Destination of the packet stream of interest are
usually known a' priori.
4. The characteristics of the packet stream of interest are known at
the Source at least, and may be communicated to Destination as
part of the method.
When adding traffic to the network for measurement, Active Methods
influence the quantities measured to some degree, and those
performing tests should take steps to quantify the effect(s) and/or
minimize such effects.
3.5. Active Metric
An Active Metric incorporates one or more of the aspects of Active
Methods in the metric definition.
For example, IETF metrics for IP performance (developed according to
the [RFC2330] framework) include the Source packet stream
Morton Expires March 9, 2016 [Page 4]
Internet-Draft Active and Passive September 2015
characteristics as metric input parameters, and also specify the
packet characteristics (Type-P) and Source and Destination IP
addresses (with their implications on both stream treatment and
interfaces associated with measurement points).
3.6. Passive Methods
Passive measurement methods are
o based solely on observations of undisturbed and unmodified packet
stream of interest
o dependent on the existence of one or more packet streams to supply
the stream of interest
o dependent on the presence of the packet stream of interest at one
or more designated observation points.
Some passive methods simply observe and collect information on all
packets that pass Observation Point(s), while others filter the
packets as a first step and only collect information on packets that
match the filter criteria, and thereby narrow the stream of interest.
It is common that passive methods are conducted at one or more
Observation Points. Passive methods to assess Performance Metrics
often require multiple observation points, e.g., to assess latency of
packet transfer across a network path between two Observation Points.
In this case, the observed packets must include enough information to
determine the corresponding packets at different Observation Points.
Communication of the observations (in some form) to a collector is an
essential aspect of Passive Methods. In some configurations, the
traffic load associated with results export to a collector may
influence the network performance. However, the collection of
results is not unique to Passive Methods, and the load from
management and operations of measurement systems must always be
considered for potential effects on the measured values.
3.7. Passive Metric
Passive Metrics apply to observations of packet traffic (traffic
flows in [RFC7011]).
Passive performance metrics are assessed independent of the packets
or traffic flows, and solely through observation. Some refer to such
assessments as "out-of-band".
Morton Expires March 9, 2016 [Page 5]
Internet-Draft Active and Passive September 2015
One example of passive performance metrics for IP packet transfer can
be found in ITU-T Recommendation Y.1540 [Y.1540], where the metrics
are defined on the basis of reference events as packet pass reference
points, so the metrics are agnostic to the distinction between active
and passive when the necessary packet correspondence can be derived
from the observed stream of interest as required.
3.8. Hybrid Methods and Metrics
Hybrid Methods are Methods of Measurement which use a combination of
Active Methods and Passive Methods, to assess Active Metrics, Passive
Metrics, or new metrics derived from the a' priori knowledge and
observations of the stream of interest. ITU-T Recommendation Y.1540
[Y.1540] defines metrics are applicable to the hybrid category, since
packet correspondence at different observation/reference points could
be derived from "fields which are dedicated to measurement", but
otherwise the methods are passive.
There are several types of Hybrid methods, as categorized below.
With respect to a *single* stream of interest, Hybrid Type I methods
fit in the continuum as follows, in terms of what happens at the
Source (or Observation Point nearby):
o If you generate the stream of interest => Active
o If you augment of modify a stream of interest => Hybrid Type I
o If you solely observe a stream of interest => Passive
We define Hybrid Type II as follows: Methods that employ two or more
different streams of interest with some degree of mutual coordination
(one or more Active streams and one or more undisturbed and
unmodified packet streams) to collect both Active and Passive Metrics
and enable enhanced characterization from additional joint analysis.
[I-D.trammell-ippm-hybrid-ps] presents a problem statement for Hybrid
Type II methods and metrics. Note that one or more Hybrid Type I
streams could be substituted for the Active streams or undisturbed
streams in the mutually coordinated set. It is the Type II Methods
where unique Hybrid Metrics are atnticipated to emerge.
Methods based on a combination of a single (generated) Active stream
and Passive observations applied to the stream of interest at
intermediate observation points are also a type of Hybrid Methods.
However, [RFC5644] already defines these as Spatial Metrics and
Methods. It is possible to replace the Active stream of [RFC5644]
with a Hybrid Type I stream and measure Spatial Metrics (but this was
un-anticipated when [RFC5644] was developed).
Morton Expires March 9, 2016 [Page 6]
Internet-Draft Active and Passive September 2015
The Table below illustrates the categorization of methods (where
"Synthesis" refers to a combination of Active and Passive Method
attributes).
| Single Stream | Multiple Simultaneous
| of Interest | Streams of Interest
| | from Different Methods
====================================================================
Single Fundamental | Active or Passive |
Method | |
Synthesis of | Hybrid Type I |
Fundamental Methods | |
Multiple Methods | Spatial Metrics | Hybrid Type II
| [RFC 5644] |
4. Discussion
This section illustrates the definitions and presents some examples.
4.1. Graphical Representation
If we compare the Active and Passive Methods, there are at least two
dimensions on which methods can be evaluated. This evaluation space
may be useful when a method is a combination of the two alternative
methods.
The two dimensions (initially chosen) are:
1. The degree to which the stream of interest effects overall
network conditions experienced by that stream and other streams.
This is a key dimension for Active measurement error analysis.
(Comment: There is also the notion of time averages - a
measurement stream may have significant effect while it is
present, but the stream is only generated 0.1% of the time. On
the other hand, observations alone have no effect on network
performance. To keep things simple, we consider the stream
effect only when it is present.)
2. The degree to which stream characteristics are know a' priori.
There are methodological advantages of knowing the source stream
characteristics, and having complete control of the stream
characteristics. For example, knowing the number of packets in a
stream allows more efficient operation of the measurement
receiver, and so is an asset for active measurement methods.
Passive methods (with no sample filter) have few clues available
Morton Expires March 9, 2016 [Page 7]
Internet-Draft Active and Passive September 2015
to anticipate what the protocol first packet observed will use or
how many packets will comprise the flow, but once the standard
protocol of a flow is known the possibilities narrow (for some
compliant flows). Therefore this is a key dimension for Passive
measurement error analysis.
There are a few examples we can plot on a two-dimensional space. We
can anchor the dimensions with reference point descriptions.
Effect of the measured stream on network conditions
^ Max
|* Active using max capacity stream
|
|
|
|
|* Active using stream with load of typical user
|
|
|
|* Active using extremely sparse, randomized stream
| * PDM Passive
| Min *
+----------------------------------------------------------------|
| |
Stream No Stream
Characteristics Characteristics
completely Known
known
We recognize that method categorization could be based on additional
dimensions, but this would require a different graphical approach.
For example, "effect of stream of interest on network conditions"
could easily be further qualified into:
1. effect on the performance of the stream of interest itself: for
example, choosing a packet marking or DSCP resulting in domain
treatment as a real-time stream (as opposed to default/best-
effort marking.
2. effect on unmeasured streams that share the path and/or
bottlenecks: for example, an extremely sparse measured stream of
minimal size packets typically has little effect on other flows
(and itself), while a stream designed to characterize path
Morton Expires March 9, 2016 [Page 8]
Internet-Draft Active and Passive September 2015
capacity may effect all other flows passing through the capacity
bottleneck (including itself).
3. effect on network conditions resulting in network adaptation: for
example, a network monitoring load and congestion conditions
might change routing, placing some flows to alternate paths to
mitigate the congestion.
At present, we have combined 1 and 2 on one axis, as examination of
examples indicates strong correlation of affects on this pair, and
network adaptation is not addressed. As suggestions emerge we will
re-examine the possibilities.
It is apparent that different methods of IP network measurement can
produce different results, even when measuring the same path at the
same time. The two dimensions of the graph help to understand how
the results might change with the method chosen. For example, an
Active Method to assess throughput adds some amount of traffic to the
network which might result in lower throughput for all streams.
However, a Passive Method to assess throughput can also err on the
low side due to unknown limitations of the hosts providing traffic,
competition for host resources, limitations of the network interface,
or private sub-networks that are not an intentional part of the path,
etc. And Hybrid Methods could easily suffer from both forms of
error. Another example of potential errors stems from the pitfalls
of using an Active stream with known bias, such as a periodic stream
defined in [RFC3432]. The strength of modelling periodic streams
(like VoIP) is a potential weakness when extending the measured
results to other application whose streams are non-periodic. The
solutions are to model the application streams more exactly with an
Active Method, or accept the risks and potential errors with the
Passive Method discussed above.
4.2. Discussion of PDM
In [I-D.ietf-ippm-6man-pdm-option], an IPv6 Option Header for
Performance and Diagnostic Measurements (PDM) is described which
(when added to the stream of interest at strategic interfaces)
supports performance measurements. This method processes a user
traffic stream and adds "fields which are dedicated to measurement".
Thus:
o The method may have a small effect on the measured stream and
other streams in the network.
o The measured stream has unknown characteristics until it is
processed to add the PDM Option header.
Morton Expires March 9, 2016 [Page 9]
Internet-Draft Active and Passive September 2015
We conclude that this is a Hybrid Type I method, having at least one
characteristic of both active and passive methods.
4.3. Discussion of "Coloring" Method
Draft [I-D.tempia-opsawg-p3m], proposed to color packets by re-
writing a field of the stream at strategic interfaces to support
performance measurements. This method processes a user traffic
stream and inserts "fields which are dedicated to measurement".
Thus:
o The method may have a small effect on the measured stream and
other streams in the network (smaller than PDM above).
o The measured stream has unknown characteristics until it is
processed to add the coloring in the header, and the stream could
be measured and time-stamped during that process.
We note that [I-D.chen-ippm-coloring-based-ipfpm-framework] proposes
a method similar to [I-D.tempia-opsawg-p3m], and ippm-list discussion
indicates [I-D.chen-ippm-coloring-based-ipfpm-framework] may be
covered by the same IPR as [I-D.tempia-opsawg-p3m].
We conclude that this is a Hybrid Type I method, having at least one
characteristic of both active and passive methods.
5. Security considerations
When considering privacy of those involved in measurement or those
whose traffic is measured, there is sensitive information
communicated and observed at observation and measurement points
described above. We refer the reader to the privacy considerations
described in the Large Scale Measurement of Broadband Performance
(LMAP) Framework [I-D.ietf-lmap-framework], which covers active and
passive measurement techniques and supporting material on measurement
context.
6. IANA Considerations
This memo makes no requests for IANA consideration.
7. Acknowledgements
Thanks to Mike Ackermann for asking the right question, and for
several suggestions on terminology. Brian Trammell provided key
terms and references for the passive category, and suggested ways to
expand the Hybrid description and types. Phil Eardley suggested some
hybrid scenaios for categorization as part of his review. Tiziano
Morton Expires March 9, 2016 [Page 10]
Internet-Draft Active and Passive September 2015
Ionta reviewed the draft and suggested the classification for the
"coloring" method of measurement. Nalini Elkins identified several
areas for clarification following her review. Bill Jouris suggested
several editorial improvements.
8. References
8.1. Normative References
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998,
<http://www.rfc-editor.org/info/rfc2330>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
performance measurement with periodic streams", RFC 3432,
DOI 10.17487/RFC3432, November 2002,
<http://www.rfc-editor.org/info/rfc3432>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644,
DOI 10.17487/RFC5644, October 2009,
<http://www.rfc-editor.org/info/rfc5644>.
[RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for
Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April
2010, <http://www.rfc-editor.org/info/rfc5835>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<http://www.rfc-editor.org/info/rfc7011>.
8.2. Informative References
[I-D.ietf-lmap-framework]
Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., and A. Akhter, "A framework for Large-Scale
Measurement of Broadband Performance (LMAP)", draft-ietf-
lmap-framework-14 (work in progress), April 2015.
Morton Expires March 9, 2016 [Page 11]
Internet-Draft Active and Passive September 2015
[I-D.ietf-ippm-6man-pdm-option]
Elkins, N. and M. Ackermann, "IPv6 Performance and
Diagnostic Metrics (PDM) Destination Option", draft-ietf-
ippm-6man-pdm-option-00 (work in progress), June 2015.
[I-D.tempia-opsawg-p3m]
Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda,
"A packet based method for passive performance
monitoring", draft-tempia-opsawg-p3m-04 (work in
progress), February 2014.
[I-D.chen-ippm-coloring-based-ipfpm-framework]
Chen, M., Zheng, L., Mirsky, G., and G. Fioccola, "IP Flow
Performance Measurement Framework", draft-chen-ippm-
coloring-based-ipfpm-framework-04 (work in progress), July
2015.
[I-D.zheng-ippm-framework-passive]
Zheng, L., Elkins, N., Lingli, D., Ackermann, M., and G.
Mirsky, "Framework for IP Passive Performance
Measurements", draft-zheng-ippm-framework-passive-03 (work
in progress), February 2015.
[I-D.trammell-ippm-hybrid-ps]
Trammell, B., Zheng, L., Berenguer, S., and M. Bagnulo,
"Hybrid Measurement using IPPM Metrics", draft-trammell-
ippm-hybrid-ps-01 (work in progress), February 2014.
[Y.1540] ITU-T Recommendation Y.1540, , "Internet protocol data
communication service - IP packet transfer and
availability performance parameters", March 2011.
Author's Address
Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ
USA
Email: acmorton@att.com
Morton Expires March 9, 2016 [Page 12]