Export of Delay Performance Metrics in IP Flow Information eXport (IPFIX)
draft-ietf-opsawg-ipfix-on-path-telemetry-14
| Document | Type | Active Internet-Draft (opsawg WG) | |
|---|---|---|---|
| Authors | Thomas Graf , Benoît Claise , Alex Huang Feng | ||
| Last updated | 2024-11-04 | ||
| Replaces | draft-tgraf-opsawg-ipfix-on-path-telemetry | ||
| RFC stream | Internet Engineering Task Force (IETF) | ||
| Intended RFC status | Proposed Standard | ||
| Formats | |||
| Reviews |
TSVART Last Call review
by Martin Duke
Ready w/issues
OPSDIR Last Call Review due 2024-10-24
Incomplete
|
||
| Additional resources |
Related Implementations
Mailing list discussion |
||
| Stream | WG state | Waiting for WG Chair Go-Ahead | |
| Document shepherd | (None) | ||
| IESG | IESG state | I-D Exists | |
| Consensus boilerplate | Yes | ||
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ietf-opsawg-ipfix-on-path-telemetry-14
Network Working Group T. Graf
Internet-Draft Swisscom
Intended status: Standards Track B. Claise
Expires: 7 May 2025 Huawei
A. Huang Feng
INSA-Lyon
3 November 2024
Export of Delay Performance Metrics in IP Flow Information eXport
(IPFIX)
draft-ietf-opsawg-ipfix-on-path-telemetry-14
Abstract
This document specifies new IP Flow Information Export (IPFIX)
Information Elements to export the On-Path Telemetry measured delay
on the OAM transit and decapsulating nodes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 7 May 2025.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
Graf, et al. Expires 7 May 2025 [Page 1]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . 6
3.1. IP One-Way Delay Hybrid Type I Passive Performance
Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.1. Summary . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.2. Description . . . . . . . . . . . . . . . . . . . . . 8
3.1.3. Reference . . . . . . . . . . . . . . . . . . . . . . 8
3.1.4. Change Controller . . . . . . . . . . . . . . . . . . 8
3.1.5. Version of Registry Format . . . . . . . . . . . . . 9
3.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 9
3.2.1. Reference Definition . . . . . . . . . . . . . . . . 9
3.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 9
3.3. Method of Measurement . . . . . . . . . . . . . . . . . . 9
3.3.1. Reference Methods . . . . . . . . . . . . . . . . . . 10
3.3.2. Packet Stream Generation . . . . . . . . . . . . . . 10
3.3.3. Traffic Filtering (Observation) Details . . . . . . . 10
3.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 10
3.3.5. Runtime Parameters and Data Format . . . . . . . . . 10
3.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4.2. Reference Definition . . . . . . . . . . . . . . . . 11
3.4.3. Administrative Items . . . . . . . . . . . . . . . . 14
3.4.4. Comments and Remarks . . . . . . . . . . . . . . . . 14
4. IPFIX Information Elements . . . . . . . . . . . . . . . . . 14
5. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
6.1. Performance Metrics . . . . . . . . . . . . . . . . . . . 16
6.2. IPFIX Entities . . . . . . . . . . . . . . . . . . . . . 16
6.2.1. pathDelayMeanDeltaMicroseconds . . . . . . . . . . . 17
6.2.2. pathDelayMinDeltaMicroseconds . . . . . . . . . . . . 18
6.2.3. pathDelayMaxDeltaMicroseconds . . . . . . . . . . . . 18
6.2.4. pathDelaySumDeltaMicroseconds . . . . . . . . . . . . 19
7. Operational Considerations . . . . . . . . . . . . . . . . . 19
7.1. Time Accuracy . . . . . . . . . . . . . . . . . . . . . . 19
7.2. Mean Delay . . . . . . . . . . . . . . . . . . . . . . . 19
Graf, et al. Expires 7 May 2025 [Page 2]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
7.3. Reduced-size encoding . . . . . . . . . . . . . . . . . . 19
7.4. Measurement Interval . . . . . . . . . . . . . . . . . . 20
7.5. In-Packet OAM Application . . . . . . . . . . . . . . . . 20
8. Security Considerations . . . . . . . . . . . . . . . . . . . 21
9. Implementation Status . . . . . . . . . . . . . . . . . . . . 21
9.1. FD.io VPP . . . . . . . . . . . . . . . . . . . . . . . . 21
9.2. Huawei VRP . . . . . . . . . . . . . . . . . . . . . . . 21
9.3. Fluvia . . . . . . . . . . . . . . . . . . . . . . . . . 22
9.4. Pmacct Data Collection . . . . . . . . . . . . . . . . . 22
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
11.1. Normative References . . . . . . . . . . . . . . . . . . 22
11.2. Informative References . . . . . . . . . . . . . . . . . 24
Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . 27
A.1. Aggregated On-Path Dealay Examples . . . . . . . . . . . 27
A.1.1. Template Record and Data Set with Mean Delta . . . . 27
A.1.2. Template Record and Data Set with Sum Delta . . . . . 29
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 31
1. Introduction
Network operators usually gather and maintain some forms of
statistical delay view of their networks (or segments of their
networks). That view is meant to help with understanding where in
the network, for which customer traffic or services, how much, and
why abnormal delay is being accumulated. To that aim, delay-related
data need to be reported from devices covering both data and control
planes. In order to understand which customer traffic is affected,
delay-related data need to be reported in the context of the customer
data-plane context. That enables network operators to quickly
identify when the control-plane updates the current path with a
different set of intermediate hops (that is, a change of the
forwarding path) and interfaces, how the path delay changes for which
customer traffic.
With On-Path Telemetry, described in the Network Telemetry Framework
[RFC9232] and applied in In Situ Operations, Administration, and
Maintenance (IOAM) Deployment [RFC9378] and Alternate Marking
Deployment Framework [I-D.ietf-ippm-alt-mark-deployment], the path
delay between two endpoints is measured by inserting a timestamp in
the packet.
Graf, et al. Expires 7 May 2025 [Page 3]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
At least two modes of On-Path Telemetry can be distinguished.
Passport mode, where only the last hop in the forwarding path of the
On-Path Telemetry domain exposes all the metrics, and postcard mode,
where the metrics are also exposed in transit nodes. In both modes
the forwarding path exposes performance metrics allowing to determine
how much delay has been accumulated on which hop. The proposal in
this document makes more sense for the postcard mode.
In order to export the delay-related metrics via IPIFX [RFC7011],
this document defines four new IPFIX Information Elements (IEs),
exposing the On-Path delay on OAM transit and decapsulating nodes,
following the postcard mode principles. Since these IPFIX IEs are
performance metrics [RFC8911], they must be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
Following the guidelines for Registered Performance Metric Requesters
and Reviewers [RFC8911], the different characteristics of the
performance metrics (Identifier, Name, URI, Status, Requester,
Revision, Revision Date, Description, etc.) must be clearly specified
in the "IANA Performance Metric Registry [IANA-PERF-METRIC] in order
for the measurement results using the Performance Metrics to be
comparable even if they are performed using different implementations
and in different networks. The first performance metric
characteristic is the selection of a meaningful name, following the
"MetricType_Method_SubTypeMethod_... Spec_Units_Output" naming
convention (See Section 7.1.2 of [RFC8911]).
+------------------------------------+-------------------------------+
| Performance Metric | IPFIX Information Element |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMeanDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Mean (TBD1)|(TBD5) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMinDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Min (TBD2) |(TBD6) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelayMaxDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Max (TBD3) |(TBD7) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |pathDelaySumDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Sum (TBD4) |(TBD8) |
+------------------------------------+-------------------------------+
Table 1: Mapping Between IPFIX IEs and Performance Metrics
Assuming time synchronization on devices, the delay is measured by
calculating the difference between the timestamp imposed with On-Path
Telemetry in the packet at the OAM encapsulating node and the
Graf, et al. Expires 7 May 2025 [Page 4]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
timestamp exported in the IPFIX flow record from the OAM transit and
decapsulating nodes. The lowest, highest, mean, and/or the sum of
measured path delay can be exported, thanks to the different IPFIX IE
specifications.
On-Path Telemetry Domain
.........................................
. .
. D1 .
. x------> .
. .
. D2 .
. x--------------------> .
. .
. D3 .
. x-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1 Encapsulating Transit Transit Decapsulating Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
Figure 1: Delay use case. Packets flow from host 1 to host 2.
In the use case shown in Figure 1 using On-path Telemetry to export
the delay metrics, the node R2 exports the delay D1, the node R3
exports the delay D2 and the decapsulating node R4 exports the total
delay D3 using IPFIX.
The advantage of this solution is that the delay metrics (min, max,
and mean) can be computed on the router, and aggregated directly
within the Flow Record, saving export bandwidth and computation on
the Collector. For the computation of the min, max, and mean delay
metric to be computed locally on the router, the exporter Metering
Process requires some local caching/processing computation (for each
new packets in the flow), specifically the mean value. A less
computational heavy solution for the router is the export of the
delay sum instead of the delay mean; on the Collector, the delay mean
can easily be computed by a single division operation (using the
packet count). The alternative, with no delay monitoring on the
router, requires the export of every single packet as a separate Flow
Record, including the timestamps information, as described in
[I-D.ietf-opsawg-ipfix-alt-mark] for Alternate Marking, for the
Collector to compute delay metrics (min, max, and mean), before
recomputing the aggregated Flow Record.
Graf, et al. Expires 7 May 2025 [Page 5]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document makes use of the terms defined in [RFC7011] and
[RFC9378].
The following terms are used as defined in [RFC7011]:
* IPFIX
* IPFIX Information Elements (IEs)
* Flow
* Flow Record
* Exporter
The following terms are used as defined in [RFC8911]:
* Performance Metric
* Registered Performance Metric
* Performance Metrics Registry
The following terms are used as defined in Section 5 of
[I-D.ietf-opsawg-oam-characterization]:
* Encapsulating node
* Transit node
* Decapsulating node
The following terms are used as defined in Section 3.8 of [RFC7799]:
* Hybrid Type I Passive
3. Performance Metrics
This section defines the new performance metrics following the
template defined in Section 11 of [RFC8911].
Graf, et al. Expires 7 May 2025 [Page 6]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
IANA Note (to be removed): RFC 8192 Section 4 was taken a guiding
example.
3.1. IP One-Way Delay Hybrid Type I Passive Performance Metrics
This section specifies four performance metrics for the Hybrid Type I
Passive assessment of IP One-Way Delay, to be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
All column entries besides the Identifier, Name, URI, Description,
Reference Description (Output only) categories are the same; thus,
this section defines four closely related performance metrics. As a
result, IANA has assigned corresponding URIs to each of the four
registered performance metrics.
3.1.1. Summary
This category includes multiple indexes of the registered performance
metrics: the element Identifier and Metric Name.
3.1.1.1. ID (Identifier)
IANA has allocated the numeric Identifiers TBD1, TBD2, TBD3, and TBD4
for the four Named Metric Entries in the following section.
RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4.
3.1.1.2. Name
TBD1: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
TBD2: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
TBD3: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
TBD4: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace [RFC-to-be].
3.1.1.3. URI
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
Graf, et al. Expires 7 May 2025 [Page 7]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
URI: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace RFC-to-be.
3.1.2. Description
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean: This
metric assesses the mean of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min: This
metric assesses the minimum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max: This
metric assesses the maximum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum: This
metric assesses the sum of one-way delays of all successfully
forwarded IP packets constituting a single Flow. We consider the
measurement of one-way delay based on a single Observation Point
(OP) [RFC7011] somewhere in the network.
RFC EDITOR NOTE: please replace RFC-to-be.
3.1.3. Reference
[RFC-to-be]
RFC EDITOR NOTE: please replace RFC-to-be.
3.1.4. Change Controller
IETF
Graf, et al. Expires 7 May 2025 [Page 8]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
3.1.5. Version of Registry Format
1.0
3.2. Metric Definition
This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable
document reference and values of input factors, called "Fixed
Parameters".
3.2.1. Reference Definition
Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-
Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC
7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-
editor.org/info/rfc7679>. [RFC7679]
Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC
6049, DOI 10.17487/RFC6049, January 2011, <https://www.rfc-
editor.org/info/rfc6049>. [RFC6049]
Section 3.4 of [RFC7679] provides the reference definition of the
singleton (single value) one-way delay metric. Section 4.4 of
[RFC7679] provides the reference definition expanded to cover a
multi-value sample. Note that terms such as "singleton" and "sample"
are defined in Section 2 of [RFC2330].
With the OP [RFC7011] typically located between the hosts
participating in the IP Flow, the one-way delay metric requires one
individual measurement between the OP and sourcing host, such that
the Spatial Composition [RFC6049] of the measurements yields a one-
way delay singleton.
This document specifies how to export the performance metric using
IPFIX.
3.2.2. Fixed Parameters
None
3.3. Method of Measurement
This category includes columns for references to relevant sections of
the RFC(s) and any supplemental information needed to ensure an
unambiguous method for implementations.
Graf, et al. Expires 7 May 2025 [Page 9]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
3.3.1. Reference Methods
The foundational methodology for this metric is defined in Section 4
of [RFC7323] using the Timestamps option with modifications that
allow application at a mid-path OP [RFC7011].
3.3.2. Packet Stream Generation
The timestamp when the packet is being received at OAM encapsulating
node. Format depends on On-Path Telemetry implementation. For IOAM,
Section 4.4.1 of [RFC9197] describes what kind of timestamps are
supported. Section 4.4.2.3 and 4.4.2.4 describe where the timestamp
is being inserted. For the Enhanced Alternate Marking Method,
Section 2 of [I-D.zhou-ippm-enhanced-alternate-marking] and
Section 3.2 of [I-D.fz-spring-srv6-alt-mark] defines timestamp
encoding and granularity.
3.3.3. Traffic Filtering (Observation) Details
Runtime Parameters (in the following sections) may be used for
Traffic Filtering.
3.3.4. Sampling Distribution
This metric requires a partial sample of all packets that qualify
according to the Traffic Filter criteria.
3.3.5. Runtime Parameters and Data Format
Runtime Parameters are input factors that must be determined,
configured into a measurement system, and reported with the results
for the context to be complete.
The hybrid type I metering parameters must be reported to provide the
complete measurement context. As an example, if the IPFIX Metering
Process is used, then the IPFIX Metering Process parameters (IPFIX
Template Record, potential traffic filters, and potential sampling
method and parameters) that generate the Flow Records must be
reported to provide the complete measurement context. At a minimum,
the following fields are required:
Src: The IP address of the host in the host A Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see Section 4 of [RFC6991]).
Dst: The IP address of the host in the host B Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see Section 4 of [RFC6991]).
Graf, et al. Expires 7 May 2025 [Page 10]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
T0: T time, the start of a measurement interval (format "date/time"
as specified in Section 5.6 of [RFC3339]; see also "date-and-time"
in Section 3 of [RFC6991]). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", a start time is
unspecified and Tf is to be interpreted as the duration of the
measurement interval. The start time is controlled through other
means.
Tf: A time, the end of a measurement interval (format "date/time" as
specified in Section 5.6 of [RFC3339]; see also "date-and-time" in
Section 3 of [RFC6991]). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", an ending time
and date is ignored and Tf is interpreted as the duration of the
measurement interval.
3.3.6. Roles
host A: Launches an IP packet to start the Flow.
host B: Receives the IP packet to start the Flow.
Encapsulating Node: Receives the IP Flow packets and encapsulates
the timestamp into the packet.
Transit Node: Receives the IP Flow packets and measures the delay
between the timestamp in the packet and the timestamp when the
packet was received.
Decapsulating Node: Receives the IP Flow packets and computes the
delay between the timestamp in the packet and the timestamp when
the packet was received and removes the OAM header from the
packet.
3.4. Output
This category specifies all details of the output of measurements
using the metric.
3.4.1. Type
OWDelay Types are discussed in the subsections below.
3.4.2. Reference Definition
For all output types:
OWDelay_HybridType1_Passive_IP: The one-way delay of one IP packet
is a Singleton
Graf, et al. Expires 7 May 2025 [Page 11]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
For each <statistic> Singleton one of the following subsections
applies.
3.4.2.1. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
Similar to Section 7.4.2.2 of [RFC8912], the mean SHALL be calculated
using the conditional distribution of all packets with a finite value
of one-way delay (undefined delays are excluded) -- a single value,
as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.2.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.2.3 of [RFC6049].
Mean: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
3.4.2.2. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
Similar to Section 7.4.2.3 of [RFC8912], the minimum SHALL be
calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.2 of [RFC6049] for details on calculating this
statistic; see also Section 4.3.3 of [RFC6049].
Min: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
Graf, et al. Expires 7 May 2025 [Page 12]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
3.4.2.3. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
Similar to Section 7.4.2.4 of [RFC8912], the maximum SHALL be
calculated using the conditional distribution of all packets with a
finite value of one-way delay (undefined delays are excluded) -- a
single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.2 of [RFC6049] for a closely related method for
calculating this statistic; see also Section 4.3.3 of [RFC6049]. The
formula is as follows:
Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
where all packets n = 1 through N have finite singleton delays.
Max: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
3.4.2.4. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
The sum SHALL be calculated using the conditional distribution of all
packets with a finite value of one-way delay (undefined delays are
excluded) -- a single value, as follows:
See Section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see Section 5
of [RFC6703] for background on this analysis choice.
See Section 4.3.5 of [RFC6049] for details on calculating this
statistic. However in this case FiniteDelay or MaxDelay MAY be used.
Sum: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(similar to the decimal64 in YANG, Section 9.3 of [RFC6020]) with
a resolution of 0.000000001 seconds (1.0 ns), and with lossless
conversion to/from the 64-bit NTP timestamp as per Section 6 of
[RFC5905].
Graf, et al. Expires 7 May 2025 [Page 13]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
3.4.2.5. Metric Units
* Mean
* Min
* Max
* Sum
The one-way delay of the IP Flow singleton is expressed in seconds.
3.4.2.6. Calibration
A clock synchronization between the nodes of the monitored OAM domain
is needed to compute representative delay measurements at the transit
and decapsulating nodes. NTP, as defined in [RFC5905], can be used
for synchronizing the clocks of the monitored nodes.
3.4.3. Administrative Items
3.4.3.1. Status
Current
3.4.3.2. Requester
This RFC
RFC EDITOR NOTE: please replace This RFC text by the RFC issued from
this document
3.4.3.3. Revision
1.0
3.4.3.4. Revision Date
RFC Date
3.4.4. Comments and Remarks
none
4. IPFIX Information Elements
This section specifies the following new IPFIX IEs:
Graf, et al. Expires 7 May 2025 [Page 14]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
pathDelayMeanDeltaMicroseconds
32-bit unsigned integer that identifies the mean path delay of all
packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node).
pathDelayMinDeltaMicroseconds
32-bit unsigned integer that identifies the lowest path delay of
all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node).
pathDelayMaxDeltaMicroseconds
32-bit unsigned integer that identifies the highest path delay of
all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node).
pathDelaySumDeltaMicroseconds
64-bit unsigned integer that identifies the sum of the path delay
of all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node).
5. Use Cases
The measured On-Path delay can be aggregated with Flow Aggregation as
defined in [RFC7015] to the following device and control-plane
dimensions to determine:
* With node id and egressInterface(14), on which node which logical
egress interfaces have been contributing to how much delay.
* With node id and egressPhysicalInterface(253), on which node which
physical egress interfaces have been contributing to how much
delay.
* With ipNextHopIPv4Address(15) or ipNextHopIPv6Address(62), the
forwarding path to which next-hop IP contributed to how much
delay.
* With mplsTopLabelIPv4Address(47) or destinationIPv6Address and
srhActiveSegmentIPv6(495), the forwarding path to which MPLS top
label IPv4 address or IPv6 destination address and SRv6 active
segment contributed to how much delay.
Graf, et al. Expires 7 May 2025 [Page 15]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
* BGP communities [RFC1997] are often used for setting a path
priority or service selection. With
bgpDestinationExtendedCommunityList(488) or
bgpDestinationCommunityList(485) or
bgpDestinationLargeCommunityList(491) which group of prefixes
accumulated at which node how much delay.
* With destinationIPv4Address(13), destinationTransportPort(11),
protocolIdentifier (4) and sourceIPv4Address(8), or equivalent
IPFIX IEs for IPv6, the forwarding path delay on each node from
each IPv4 source address to a specific application in the network.
Let us consider the example depicted in Figure 1 from Section 1 as
topology example. Below example table shows the aggregated delay per
each node, ingressInterface,(10) egressInterface(14),
destinationIPv6Address(28) and srhActiveSegmentIPv6(495).
+-----------+-----------+------+-------------+-------------+------------+
| ingress | egress | Node | destination | srhActive | Path Delay |
| Interface | Interface | | IPv6Address | SegmentIPv6 | |
+-----------+-----------+------+-------------+-------------+------------+
| 271 | 276 | R1 | 2001:db8::2 | 2001:db8::4 | 0 us |
+-----------+-----------+------+-------------+-------------+------------+
| 301 | 312 | R2 | 2001:db8::3 | 2001:db8::4 | 22 us |
+-----------+-----------+------+-------------+-------------+------------+
| 22 | 27 | R3 | 2001:db8::4 | 2001:db8::4 | 42 us |
+-----------+-----------+------+-------------+-------------+------------+
| 852 | 854 | R4 | 2001:db8::4 | 2001:db8::4 | 122 us |
+-----------+-----------+------+-------------+-------------+------------+
Table 2: Example table of measured delay. Ascending by delay.
6. IANA Considerations
6.1. Performance Metrics
This document requests IANA to add four new performance metrics under
the "Performance Metrics" registry [RFC8911] with the four templates
defined in Section 3.
6.2. IPFIX Entities
This document requests IANA to register new IPFIX IEs (see table 3)
under the "IPFIX Information Elements" registry [RFC7012] available
at "IANA IP Flow Information Export (IPFIX) Entities Registry
[IANA-IPFIX] and assign the following initial code points.
Graf, et al. Expires 7 May 2025 [Page 16]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
+-------+--------------------------------+
|Element| Name |
| ID | |
+-------+--------------------------------+
| TBD5 | pathDelayMeanDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD6 | pathDelayMinDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD7 | pathDelayMaxDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD8 | pathDelaySumDeltaMicroseconds |
| | |
+-------+--------------------------------+
Table 3: New IPFIX IEs in the "IPFIX Information Elements" Registry
Note to the RFC-Editor:
* Please replace TBD5 - TBD8 with the values allocated by IANA
* Please replace all instances of [RFC-to-be] in this section with
the RFC number assigned to this document
6.2.1. pathDelayMeanDeltaMicroseconds
Name: pathDelayMeanDeltaMicroseconds
ElementID: TBD5
Description: This Information Element identifies the mean path delay
of all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be]
Additional Information:
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry.
Graf, et al. Expires 7 May 2025 [Page 17]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
6.2.2. pathDelayMinDeltaMicroseconds
Name: pathDelayMinDeltaMicroseconds
ElementID: TBD6
Description: This Information Element identifies the lowest path
delay of all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node), according to
the OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min in
the IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be]
Additional Information:
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min in the
IANA Performance Metric Registry.
6.2.3. pathDelayMaxDeltaMicroseconds
Name: pathDelayMaxDeltaMicroseconds
ElementID: TBD7
Description: This Information Element identifies the highest path
delay of all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be]
Additional Information:
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max in the
IANA Performance Metric Registry.
Graf, et al. Expires 7 May 2025 [Page 18]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
6.2.4. pathDelaySumDeltaMicroseconds
Name: pathDelaySumDeltaMicroseconds
ElementID: TBD8
Description: This Information Element identifies the sum of the path
delay of all packets in the Flow, in microseconds, between the OAM
encapsulating node and the local node with the OAM domain (either
an OAM transit node or an OAM decapsulating node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum in the
IANA Performance Metric Registry.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
Reference: [RFC-to-be]
Additional Information:
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum in the
IANA Performance Metric Registry.
7. Operational Considerations
7.1. Time Accuracy
The same recommendation as defined in Section 4.5 of [RFC5153] for
IPFIX applies in terms of clock precision to this document as well.
7.2. Mean Delay
The mean (average) path delay can be calculated by dividing the
pathDelaySumDeltaMicroseconds(TBD8) by the packetDeltaCount(2) at the
IPFIX data collection in order to offload the IPFIX Exporter from
calculating the mean for every Flow at export time.
7.3. Reduced-size encoding
Unsigned64 has been chosen as type for pathDelaySumDeltaMicroseconds
to support cases with large delay numbers and where many packets are
being accounted. As an example, a specific Flow Record with path
delay of 100 milliseconds cannot observe more than 42949 packets
without overflowing the unsigned32 counter. The procedure described
in Section 6.2 of [RFC7011] may be applied to reduce network
bandwidth between the IPFIX Exporter and Collector if unsigned32
would be large enough without wrapping around.
Graf, et al. Expires 7 May 2025 [Page 19]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
7.4. Measurement Interval
The delay metrics are computed for the Flow Record life time. For
long-running Flow, we might miss the temporal distribution of the
delay (for example, a longer delay only at the beginning of Flow).
If this is an operational problem, the IPFIX Metering Process might
be configured with a smaller expiration timeout (see Section 5.1.1.
Flow Expiration[RFC5470]).
7.5. In-Packet OAM Application
Multiple methods can be used to compute the delay performance metrics
defined in this document. Some examples of such methods are IOAM
[RFC9197] and Enhanced Alternate Marking
[I-D.zhou-ippm-enhanced-alternate-marking].
For IOAM, these performance metrics can be computed using the Edge-
to-Edge and the Direct Exporting Option-Type.
IOAM Edge-to-Edge Option-Type, as described in Section 4.6 of
[RFC9197], can use bits 2 and 3. In this case, timestamps are
encoded as defined in Section 4.4.2.3 and 4.4.2.4 of [RFC9197]. This
timestamp can be used to compute the delay between the encapsulating
node and the decapsulating node.
IOAM Direct Exporting Option-Type, as described in [RFC9326], can use
the Extension-Flag defined in [I-D.ahuang-ippm-dex-timestamp-ext] to
insert a timestamp in the encapsulating node. The timestamp is
encoded as defined in Section 4.4.2.3 and 4.4.2.4 of [RFC9197]. This
timestamp can be used to compute the delay between the inserted
timestamp and the transit and decapsulating node.
For the Enhanced Alternate Marking Method, Section 2 of
[I-D.zhou-ippm-enhanced-alternate-marking] and Section 3.2 of
[I-D.fz-spring-srv6-alt-mark] defines that, within the metaInfo, a
nanosecond timestamp can be encoded in the encapsulating node and be
read at the intermediate and decapsulating node to calculate the on-
path delay. [RFC9343] defines how this can be applied to the IPv6
options header and [I-D.fz-spring-srv6-alt-mark] defines how this can
be applied to the SRv6 Segment Routing Header.
Given that the delay measurements are computed with the timestamp
introduced on the encapsulating node, regardless of the approach,
implementations should document at which point of the forwarding
plane this timestamp is introduced (e.g. the time at which the packet
was received by the node, the time at which the packet was
transmitted by the node, etc). Based on this information, different
actions can be taken.
Graf, et al. Expires 7 May 2025 [Page 20]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
8. Security Considerations
The IPFIX Information Elements introduced in this document do not
directly introduce security issues. Rather, they define a set of
performance metrics that may, for privacy or business issues, be
considered sensitive information.
For example, exporting delay metrics may make attacks possible for
the receiver of this information; this would otherwise only be
possible for direct observers of the reported Flows along the data
path.
The underlying protocol used to exchange the information described
here must therefore apply appropriate procedures to guarantee the
integrity and confidentiality of the exported information. These
protocols are defined in separate documents, specifically the IPFIX
protocol document [RFC7011].
9. Implementation Status
Note to the RFC-Editor: Please remove this section before publishing.
9.1. FD.io VPP
INSA Lyon implemented the following IEs as part of a prototype in the
FD.io VPP (Vector Packet Processing) platform:
* pathDelayMeanDeltaMicroseconds
* pathDelayMaxDeltaMicroseconds
* pathDelayMinDeltaMicroseconds
* pathDelaySumDeltaMicroseconds
The open source code can be obtained here: [INSA-Lyon-VPP] and was
validated at the IETF 116 hackathon.
9.2. Huawei VRP
Huawei implemented the following IEs as part of a production
implementation in the VRP platform:
* pathDelayMeanDeltaMicroseconds
* pathDelayMaxDeltaMicroseconds
* pathDelayMinDeltaMicroseconds
Graf, et al. Expires 7 May 2025 [Page 21]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
* pathDelaySumDeltaMicroseconds
The implementation was validated at the IETF 116 hackathon.
9.3. Fluvia
NTT Com implemented the following IEs in the Fluvia Exporter:
* pathDelayMeanDeltaMicroseconds
* pathDelayMaxDeltaMicroseconds
* pathDelayMinDeltaMicroseconds
* pathDelaySumDeltaMicroseconds
The open source code can be obtained here: [NTT-Fluvia] and was
validated at the IETF 118 hackathon.
9.4. Pmacct Data Collection
Paolo Lucente implemented the IE pathDelayMeanDeltaMicroseconds by
dividing IE pathDelaySumDeltaMicroseconds by IE packetDeltaCount in
the open source Network Telemetry data collection project pmacct.
The source code can be obtained here: [Paolo-Lucente-Pmacct] and was
validated at the IETF 116 hackathon.
10. Acknowledgements
The authors would like to thank Al Morton (Rest in Peace Al), Justin
Iurman, Giuseppe Fioccola and Yannick Buchs for their review and
valuable comments. Special thanks to Paul Aitken (as IPFIX
Designated Expert), Greg Mirsky (as IP Performance Metrics Designated
Expert), and to Med Boucadair for his very detailed feedback.
11. References
11.1. Normative References
[I-D.ietf-opsawg-oam-characterization]
Pignataro, C. and A. Farrel, "Guidelines for Charactering
"OAM"", Work in Progress, Internet-Draft, draft-ietf-
opsawg-oam-characterization-03, 29 August 2024,
<https://datatracker.ietf.org/doc/html/draft-ietf-opsawg-
oam-characterization-03>.
Graf, et al. Expires 7 May 2025 [Page 22]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet:
Timestamps", RFC 3339, DOI 10.17487/RFC3339, July 2002,
<https://www.rfc-editor.org/info/rfc3339>.
[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
<https://www.rfc-editor.org/info/rfc5905>.
[RFC6049] Morton, A. and E. Stephan, "Spatial Composition of
Metrics", RFC 6049, DOI 10.17487/RFC6049, January 2011,
<https://www.rfc-editor.org/info/rfc6049>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model
for IP Flow Information Export (IPFIX)", RFC 7012,
DOI 10.17487/RFC7012, September 2013,
<https://www.rfc-editor.org/info/rfc7012>.
[RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Scheffenegger, Ed., "TCP Extensions for High Performance",
RFC 7323, DOI 10.17487/RFC7323, September 2014,
<https://www.rfc-editor.org/info/rfc7323>.
[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
Ed., "A One-Way Delay Metric for IP Performance Metrics
(IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January
2016, <https://www.rfc-editor.org/info/rfc7679>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
Graf, et al. Expires 7 May 2025 [Page 23]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
[RFC8911] Bagnulo, M., Claise, B., Eardley, P., Morton, A., and A.
Akhter, "Registry for Performance Metrics", RFC 8911,
DOI 10.17487/RFC8911, November 2021,
<https://www.rfc-editor.org/info/rfc8911>.
[RFC8912] Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza,
"Initial Performance Metrics Registry Entries", RFC 8912,
DOI 10.17487/RFC8912, November 2021,
<https://www.rfc-editor.org/info/rfc8912>.
11.2. Informative References
[I-D.ahuang-ippm-dex-timestamp-ext]
Feng, A. H., Francois, P., Claise, B., and T. Graf,
"Timestamp extension for In Situ Operations,
Administration, and Maintenance (IOAM) Direct Export",
Work in Progress, Internet-Draft, draft-ahuang-ippm-dex-
timestamp-ext-00, 15 February 2023,
<https://datatracker.ietf.org/doc/html/draft-ahuang-ippm-
dex-timestamp-ext-00>.
[I-D.fz-spring-srv6-alt-mark]
Fioccola, G., Zhou, T., Cociglio, M., Mishra, G. S., wang,
X., and G. Zhang, "Application of the Alternate Marking
Method to the Segment Routing Header", Work in Progress,
Internet-Draft, draft-fz-spring-srv6-alt-mark-09, 9 August
2024, <https://datatracker.ietf.org/doc/html/draft-fz-
spring-srv6-alt-mark-09>.
[I-D.ietf-ippm-alt-mark-deployment]
Fioccola, G., Keyi, Z., Graf, T., Nilo, M., and L. Zhang,
"Alternate Marking Deployment Framework", Work in
Progress, Internet-Draft, draft-ietf-ippm-alt-mark-
deployment-02, 9 October 2024,
<https://datatracker.ietf.org/doc/html/draft-ietf-ippm-
alt-mark-deployment-02>.
[I-D.ietf-opsawg-ipfix-alt-mark]
Graf, T., Fioccola, G., Zhou, T., Cociglio, M., and M.
Nilo, "IP Flow Information Export (IPFIX) Alternate-
Marking Information Elements", Work in Progress, Internet-
Draft, draft-ietf-opsawg-ipfix-alt-mark-01, 3 November
2024, <https://datatracker.ietf.org/doc/html/draft-ietf-
opsawg-ipfix-alt-mark-01>.
[I-D.zhou-ippm-enhanced-alternate-marking]
Zhou, T., Fioccola, G., Liu, Y., Cociglio, M., Pang, R.,
Xiong, L., Lee, S., and W. Li, "Enhanced Alternate Marking
Graf, et al. Expires 7 May 2025 [Page 24]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
Method", Work in Progress, Internet-Draft, draft-zhou-
ippm-enhanced-alternate-marking-15, 27 May 2024,
<https://datatracker.ietf.org/doc/html/draft-zhou-ippm-
enhanced-alternate-marking-15>.
[IANA-IPFIX]
"IANA IP Flow Information Export (IPFIX) Entities
Registry",
<https://www.iana.org/assignments/ipfix/ipfix.xhtml>.
[IANA-PERF-METRIC]
"IANA Performance Metric Registry",
<https://www.iana.org/assignments/performance-metrics/
performance-metrics.xhtml>.
[INSA-Lyon-VPP]
"INSA Lyon, FD.io VPP implementation",
<https://github.com/network-analytics/vpp-srh-onpath-
telemetry>.
[NTT-Fluvia]
"NTT Com, Fluvia Exporter",
<https://github.com/nttcom/fluvia/>.
[Paolo-Lucente-Pmacct]
"Paolo Lucente, Pmacct open source Network Telemetry Data
Collection", <https://github.com/pmacct/pmacct>.
[RFC1997] Chandra, R., Traina, P., and T. Li, "BGP Communities
Attribute", RFC 1997, DOI 10.17487/RFC1997, August 1996,
<https://www.rfc-editor.org/info/rfc1997>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330,
DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>.
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393,
DOI 10.17487/RFC3393, November 2002,
<https://www.rfc-editor.org/info/rfc3393>.
[RFC5153] Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P.
Aitken, "IP Flow Information Export (IPFIX) Implementation
Guidelines", RFC 5153, DOI 10.17487/RFC5153, April 2008,
<https://www.rfc-editor.org/info/rfc5153>.
Graf, et al. Expires 7 May 2025 [Page 25]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export", RFC 5470,
DOI 10.17487/RFC5470, March 2009,
<https://www.rfc-editor.org/info/rfc5470>.
[RFC6020] Bjorklund, M., Ed., "YANG - A Data Modeling Language for
the Network Configuration Protocol (NETCONF)", RFC 6020,
DOI 10.17487/RFC6020, October 2010,
<https://www.rfc-editor.org/info/rfc6020>.
[RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
IP Network Performance Metrics: Different Points of View",
RFC 6703, DOI 10.17487/RFC6703, August 2012,
<https://www.rfc-editor.org/info/rfc6703>.
[RFC6991] Schoenwaelder, J., Ed., "Common YANG Data Types",
RFC 6991, DOI 10.17487/RFC6991, July 2013,
<https://www.rfc-editor.org/info/rfc6991>.
[RFC7015] Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation
for the IP Flow Information Export (IPFIX) Protocol",
RFC 7015, DOI 10.17487/RFC7015, September 2013,
<https://www.rfc-editor.org/info/rfc7015>.
[RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi,
Ed., "Data Fields for In Situ Operations, Administration,
and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197,
May 2022, <https://www.rfc-editor.org/info/rfc9197>.
[RFC9232] Song, H., Qin, F., Martinez-Julia, P., Ciavaglia, L., and
A. Wang, "Network Telemetry Framework", RFC 9232,
DOI 10.17487/RFC9232, May 2022,
<https://www.rfc-editor.org/info/rfc9232>.
[RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T.
Mizrahi, "In Situ Operations, Administration, and
Maintenance (IOAM) Direct Exporting", RFC 9326,
DOI 10.17487/RFC9326, November 2022,
<https://www.rfc-editor.org/info/rfc9326>.
[RFC9343] Fioccola, G., Zhou, T., Cociglio, M., Qin, F., and R.
Pang, "IPv6 Application of the Alternate-Marking Method",
RFC 9343, DOI 10.17487/RFC9343, December 2022,
<https://www.rfc-editor.org/info/rfc9343>.
Graf, et al. Expires 7 May 2025 [Page 26]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
[RFC9378] Brockners, F., Ed., Bhandari, S., Ed., Bernier, D., and T.
Mizrahi, Ed., "In Situ Operations, Administration, and
Maintenance (IOAM) Deployment", RFC 9378,
DOI 10.17487/RFC9378, April 2023,
<https://www.rfc-editor.org/info/rfc9378>.
Appendix A. IPFIX Encoding Examples
This appendix represents two different encodings for the newly
introduced IEs. Taking Figure 1 from Section 1 as topology example.
Below example Table 4 shows the aggregated delay with
ingressInterface, egressInterface, destinationIPv6Address and
srhActiveSegmentIPv6.
+------ +------+-----------+-----------+------+---------+---------+---------+---------+
|ingress|egress|destination|srhActive |packet|pathDelay|pathDelay|pathDelay|pathDelay|
|Inter |Inter |IPv6Address|SegmentIPv6|Delta |MeanDelta|MinDelta |MaxDelta |SumDelta |
|face |face | | |Count |Micro.. |Micro.. |Micro.. |Micro.. |
+-------+------+-----------+-----------+------+---------+---------+---------+---------+
| 271 | 276 |2001:db8::4|2001:db8::2| 5 | 36 us | 22 us | 74 us | 180 us |
+-------+------+-----------+-----------+------+---------+---------+---------+---------+
Table 4: Aggregated delay with egressInterface and srhActiveSegmentIPv6
A.1. Aggregated On-Path Dealay Examples
A.1.1. Template Record and Data Set with Mean Delta
With encoding in Figure 2, the mean (average) path delay is
calculated on the exporting node.
* Ingress interface => ingressInterface
* Egress interface => egressInterface
* IPv6 destination address => destinationIPv6Address
* Active SRv6 Segment => srhIPv6ActiveSegment
* Packet Delta Count => packetDeltaCount
* Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (TBD6)
* Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (TBD7)
* Mean One-Way Delay => pathDelayMeanDeltaMicroseconds (TBD5)
Graf, et al. Expires 7 May 2025 [Page 27]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 2 | Length = 40 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| ingressInterface = 10 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| egressInterface = 14 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv6Address = 28 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| srhIPv6ActiveSegment = 495 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelayMeanDelta.. = TBD5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelayMinDelta.. = TBD6 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelayMaxDelta.. = TBD7 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Template Record for pathDelayMeanDeltaMicroseconds
The data set is represented as follows:
Graf, et al. Expires 7 May 2025 [Page 28]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 256 | Length = 60 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ingressInterface = 271 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| egressInterface = 276 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv6Address = |
| ... |
| ... |
| 2001:db8::2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| srhIPv6ActiveSegment = ... |
| ... |
| ... |
| 2001:db8::4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| packetDeltaCount = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelayMeanDeltaMicroseconds = 36 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelayMinDeltaMicroseconds = 22 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelayMaxDeltaMicroseconds = 74 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: Data Set Encoding for pathDelayMeanDeltaMicroseconds
A.1.2. Template Record and Data Set with Sum Delta
With encoding in Figure 4, the mean (average) path delay is
calculated on the IPFIX data collection.
* Ingress interface => ingressInterface
* Egress interface => egressInterface
* IPv6 destination address => destinationIPv6Address
* Active SRv6 Segment => srhIPv6ActiveSegment
* Packet Delta Count => packetDeltaCount
* Minimum One-Way Delay => pathDelayMinDeltaMicroseconds (TBD6)
Graf, et al. Expires 7 May 2025 [Page 29]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
* Maximum One-Way Delay => pathDelayMaxDeltaMicroseconds (TBD7)
* Sum of One-Way Delay => pathDelaySumDeltaMicroseconds (TBD8)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 2 | Length = 40 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 257 | Field Count = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| ingressInterface = 10 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| egressInterface = 14 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv6Address = 28 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| srhIPv6ActiveSegment = 495 | Field Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 5 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelayMinDelta.. = TBD6 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelayMaxDelta.. = TBD7 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| pathDelaySumDelta.. = TBD8 | Field Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Template Record for pathDelaySumDeltaMicroseconds
The data set is represented as follows:
Graf, et al. Expires 7 May 2025 [Page 30]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SET ID = 257 | Length = 60 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ingressInterface = 271 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| egressInterface = 276 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| destinationIPv6Address = |
| ... |
| ... |
| 2001:db8::2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| srhIPv6ActiveSegment = ... |
| ... |
| ... |
| 2001:db8::4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| packetDeltaCount = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelayMinDeltaMicroseconds = 22 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelayMaxDeltaMicroseconds = 74 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| pathDelaySumDeltaMicroseconds = 180 |
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Data Set Encoding for pathDelaySumDeltaMicroseconds
Authors' Addresses
Thomas Graf
Swisscom
Binzring 17
CH-8045 Zurich
Switzerland
Email: thomas.graf@swisscom.com
Benoit Claise
Huawei
Email: benoit.claise@huawei.com
Graf, et al. Expires 7 May 2025 [Page 31]
Internet-Draft Delay Performance Metrics for IPFIX November 2024
Alex Huang Feng
INSA-Lyon
Lyon
France
Email: alex.huang-feng@insa-lyon.fr
Graf, et al. Expires 7 May 2025 [Page 32]