Skip to main content

IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
draft-ietf-ippm-6man-pdm-option-09

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft that was ultimately published as RFC 8250.
Authors Nalini Elkins , Rob Hamilton , michael ackermann
Last updated 2017-04-13 (Latest revision 2017-03-13)
RFC stream Internet Engineering Task Force (IETF)
Formats
Reviews
Additional resources Mailing list discussion
Stream WG state Submitted to IESG for Publication
Document shepherd Al Morton
Shepherd write-up Show Last changed 2016-07-09
IESG IESG state Became RFC 8250 (Proposed Standard)
Consensus boilerplate Yes
Telechat date (None)
Needs a YES. Needs 9 more YES or NO OBJECTION positions to pass.
Responsible AD Spencer Dawkins
Send notices to "Bill Cerveny" <ietf@wjcerveny.com>, "Al Morton" <acmorton@att.com>
IANA IANA review state IANA OK - Actions Needed
draft-ietf-ippm-6man-pdm-option-09
INTERNET-DRAFT                                                 N. Elkins
                                                         Inside Products
                                                             R. Hamilton
                                              Chemical Abstracts Service
                                                            M. Ackermann
Intended Status: Proposed Standard                         BCBS Michigan
Expires: September 14, 2017                               March 13, 2017

    IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
                   draft-ietf-ippm-6man-pdm-option-09

Abstract

   To assess performance problems,  measurements based on optional
   sequence numbers and timing may be embedded in each packet.  Such
   measurements may be interpreted in real-time or after the fact. An
   implementation of the existing IPv6 Destination Options extension
   header, the Performance and Diagnostic Metrics (PDM) Destination
   Options extension header as well as the field limits, calculations,
   and usage of the PDM in measurement are included in this document.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

 

Elkins                 Expires September 14, 2017               [Page 1]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

Copyright and License Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph
   3: This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.     

 

Elkins                 Expires September 14, 2017               [Page 2]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

Table of Contents

   1  Background  . . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1 Terminology  . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.2 End User Quality of Service (QoS)  . . . . . . . . . . . . .  4
     1.3 Need for a Packet Sequence Number (PSN)  . . . . . . . . . .  5
     1.4 Rationale for defined solution . . . . . . . . . . . . . . .  5
     1.5 PDM Works in Collaboration with Other Headers  . . . . . . .  6
     1.6 IPv6 Transition Technologies . . . . . . . . . . . . . . . .  7
   2 Measurement Information Derived from PDM . . . . . . . . . . . .  7
     2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . .  7
     2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . .  8
   3 Performance and Diagnostic Metrics Destination Option Layout . .  8
     3.1 Destination Options Header . . . . . . . . . . . . . . . . .  8
     3.2 Performance and Diagnostic Metrics Destination Option  . . .  8
       3.2.1 PDM Layout . . . . . . . . . . . . . . . . . . . . . . .  8
       3.2.2 Base Unit for Time Measurement . . . . . . . . . . . . . 10
       3.2.3 Considerations of this time-differential
             representation . . . . . . . . . . . . . . . . . . . . . 11
         3.2.3.1 Limitations with this encoding method  . . . . . . . 11
         3.2.3.2 Loss of precision induced by timer value
                 truncation . . . . . . . . . . . . . . . . . . . . . 12
     3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 13
     3.4 Header Placement Using IPSec ESP Mode  . . . . . . . . . . . 13
       3.4.1 Using ESP Transport Mode . . . . . . . . . . . . . . . . 13
       3.4.2 Using ESP Tunnel Mode  . . . . . . . . . . . . . . . . . 14
     3.5 Implementation Considerations  . . . . . . . . . . . . . . . 15
       3.5.1 PDM Activation . . . . . . . . . . . . . . . . . . . . . 15
       3.5.2 PDM Timestamps . . . . . . . . . . . . . . . . . . . . . 15
     3.6 Dynamic Configuration Options  . . . . . . . . . . . . . . . 16
     3.6 5-tuple Aging  . . . . . . . . . . . . . . . . . . . . . . . 16
   4 Security Considerations  . . . . . . . . . . . . . . . . . . . . 16
     4.1. SYN Flood and Resource Consumption Attacks  . . . . . . . . 16
     4.2  Pervasive monitoring  . . . . . . . . . . . . . . . . . . . 17
     4.3 PDM as a Covert Channel  . . . . . . . . . . . . . . . . . . 17
     4.4 Timing Attacks . . . . . . . . . . . . . . . . . . . . . . . 18
   5 IANA Considerations  . . . . . . . . . . . . . . . . . . . . . . 18
   6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
     6.1 Normative References . . . . . . . . . . . . . . . . . . . . 19
     6.2 Informative References . . . . . . . . . . . . . . . . . . . 19
   Appendix A : Timing Time Differential Calculations . . . . . . . . 20
   Appendix B: Sample Packet Flows  . . . . . . . . . . . . . . . . . 21
     B.1 PDM Flow - Simple Client Server  . . . . . . . . . . . . . . 21
       B.1.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . 21
       B.1.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . 22
       B.1.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . 23
       B.1.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . 24
       B.1.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . 25
 

Elkins                 Expires September 14, 2017               [Page 3]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

     B.2 Other Flows  . . . . . . . . . . . . . . . . . . . . . . . . 25
       B.2.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . 25
       B.2.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . 26
       B.2.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . 27
   Appendix C: Potential Overhead Considerations  . . . . . . . . . . 29
   Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . . . 30
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 30

1  Background

   To assess performance problems,  measurements based on optional
   sequence numbers and timing may be embedded in each packet.  Such
   measurements may be interpreted in real-time or after the fact. 

   As defined in RFC2460 [RFC2460], destination options are carried by
   the IPv6 Destination Options extension header.  Destination options
   include optional information that need be examined only by the IPv6
   node given as the destination address in the IPv6 header, not by
   routers or other "middle boxes".  This document specifies a new
   destination option, the Performance and Diagnostic Metrics (PDM)
   destination option.  This document specifies the layout, field
   limits, calculations, and usage of the PDM in measurement.

1.1 Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

1.2 End User Quality of Service (QoS)

   The timing values in the PDM embedded in the packet will be used to
   estimate QoS as experienced by an end user device.

   For many applications, the key user performance indicator is response
   time.  When the end user is an individual, he is generally
   indifferent to what is happening along the network; what he really
   cares about is how long it takes to get a response back.  But this is
   not just a matter of individuals' personal convenience.  In many
   cases, rapid response is critical to the business being conducted.

   When the end user is a device (e.g. with the Internet of Things),
   what matters is the speed with which requested data can be
   transferred -- specifically, whether the requested data can be
   transferred in time to accomplish the desired actions.  This can be
   important when the relevant external conditions are subject to rapid
   change.
 

Elkins                 Expires September 14, 2017               [Page 4]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Low, reliable and acceptable response times are not just "nice to
   have".  On many networks, the impact can be financial hardship or can
   endanger human life.  In some cities, the emergency police contact
   system operates over IP; law enforcement, at all levels, use IP
   networks; transactions on our stock exchanges are settled using IP
   networks.  The critical nature of such activities to our daily lives
   and financial well-being demand a simple solution to support response
   time measurements.   

1.3 Need for a Packet Sequence Number (PSN)

   While performing network diagnostics of an end-to-end connection, it
   often becomes necessary to isolate the factors along the network path
   responsible for problems.  Diagnostic data may be collected at
   multiple places along the path (if possible), or at the source and
   destination.  Then, in post-collection processing, the diagnostic
   data corresponding to each packet at different observation points
   must be matched for proper measurements. A sequence number in each
   packet provides sufficient basis for the matching process.  If need
   be, the timing fields may be used along with the sequence number to
   ensure uniqueness.

   This method of data collection along the path is of special use to
   determine where packet loss or packet corruption is happening.    

   The packet sequence number needs to be unique in the context of the
   session (5-tuple).  See section 2 for a definition of 5-tuple.

1.4 Rationale for defined solution

   The current IPv6 specification does not provide timing nor a similar
   field in the IPv6 main header or in any extension header. So, we
   define the IPv6 Performance and Diagnostic Metrics destination option
   (PDM).

   Advantages include: 

   1. Real measure of actual transactions. 

   2. Independence from transport layer protocols. 

   3. Ability to span organizational boundaries with consistent
   instrumentation.

   4. No time synchronization needed between session partners 

   5. Ability to handle all transport protocols (TCP, UDP, SCTP, etc) in
   a uniform way 
 

Elkins                 Expires September 14, 2017               [Page 5]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   The PDM provides the ability to determine quickly if the (latency)
   problem is in the network or in the server (application).  That is,
   it is a fast way to do triage.

   One of the important functions of PDM is to allow you to do quickly
   dispatch the right set of diagnosticians.  Within network or server
   latency, there may be many components.  The job of the diagnostician
   is to rule each one out until the culprit is found.

   How PDM fits into this diagnostic picture is that PDM will quickly
   tell you how to escalate.  PDM will point to either the network area
   or the server area.   Within the server latency, PDM does not tell
   you if the bottleneck is in the IP stack or the application or buffer
   allocation. Within the network latency, PDM does not tell you which
   of the network segments or middle boxes is at fault. 

   What PDM will tell you is whether the problem is in the network or
   the server. In our experience, there is often a different group which
   is involved to troubleshoot the problem depending on the nature of
   the problem.   That is, the problem may be escalated to the
   application developers or the team that deals with the routers and
   infrastructure.  Both the network group and the application group
   have quite a few specialized tools at their disposal to further
   investigate their own areas.   What is missing is the first step,
   which PDM provides.

   In our experience, valuable time is often lost at this first stage of
   triage.  PDM is expected to reduce this time substantially. 

1.5 PDM Works in Collaboration with Other Headers

   The purpose of the PDM is not to supplant all the variables present
   in all other headers but to provide data which is not available or
   very difficult to get.   The way PDM would be used is by a technician
   (or tool) looking at a packet capture.   Within the packet capture,
   they would have available to them the layer 2 header, IP header (v6
   or v4), TCP, UCP, ICMP, SCTP or other headers.   All information
   would be looked at together to make sense of the packet flow.   The
   technician or processing tool could analyze, report or ignore the
   data from PDM, as necessary.

   For an example of how PDM can help with TCP retransmit problems,
   please look at section 8.

 

Elkins                 Expires September 14, 2017               [Page 6]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

1.6 IPv6 Transition Technologies

   In the path to full implementation of IPv6, transition technologies
   such as translation or tunneling may be employed.   The PDM header is
   not expected to work in such scenarios.  It is likely that an IPv6
   packet containing PDM will be dropped if using IPv6 transition
   technologies.

2 Measurement Information Derived from PDM

   Each packet contains information about the sender and receiver. In IP
   protocol,  the identifying information is called a "5-tuple".  

   The 5-tuple consists of:

      SADDR : IP address of the sender
      SPORT : Port for sender
      DADDR : IP address of the destination
      DPORT : Port for destination
      PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.)

   The PDM contains the following base fields:

      PSNTP    : Packet Sequence Number This Packet
      PSNLR    : Packet Sequence Number Last Received
      DELTATLR : Delta Time Last Received
      DELTATLS : Delta Time Last Sent

   Other fields for storing time scaling factors are also in the PDM and
   will be described in section 3.

   This information, combined with the 5-tuple, allows the measurement
   of the following metrics:

      1.  Round-trip delay
      2.  Server delay

2.1 Round-Trip Delay

   Round-trip *Network* delay is the delay for packet transfer from a
   source host to a destination host and then back to the source host.
   This measurement has been defined, and the advantages and
   disadvantages discussed in "A Round-trip Delay Metric for IPPM"
   [RFC2681].

 

Elkins                 Expires September 14, 2017               [Page 7]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

2.2 Server Delay

   Server delay is the interval between when a packet is received by a
   device and the first corresponding packet is sent back in response. 
   This may be "Server Processing Time".  It may also be a delay caused
   by acknowledgements.  Server processing time includes the time taken
   by the combination of the stack and application to return the
   response. The stack delay may be related to network performance.   If
   this aggregate time is seen as a problem, and there is a need to make
   a clear distinction between application processing time and stack
   delay, including that caused by the network, then more client based
   measurements are needed.

3 Performance and Diagnostic Metrics Destination Option Layout

3.1 Destination Options Header

   The IPv6 Destination Options Header is used to carry optional
   information that needs to be examined only by a packet's destination
   node(s). The Destination Options Header is identified by a Next
   Header value of 60 in the immediately preceding header and is defined
   in RFC2460 [RFC2460].  The IPv6 Performance and Diagnostic Metrics
   Destination Option (PDM) is an implementation of the Destination
   Options Header.  The PDM does not require time synchronization.

3.2 Performance and Diagnostic Metrics Destination Option

3.2.1 PDM Layout

   The IPv6 Performance and Diagnostic Metrics Destination Option (PDM)
   contains the following fields:

      SCALEDTLR: Scale for Delta Time Last Received
      SCALEDTLS: Scale for Delta Time Last Sent
      PSNTP    : Packet Sequence Number This Packet
      PSNLR    : Packet Sequence Number Last Received
      DELTATLR : Delta Time Last Received
      DELTATLS : Delta Time Last Sent

 

Elkins                 Expires September 14, 2017               [Page 8]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   The PDM destination option is encoded in type-length-value (TLV) 
   format as follows:

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Option Type  | Option Length |    ScaleDTLR  |     ScaleDTLS |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   PSN This Packet             |  PSN Last Received            |
      |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Delta Time Last Received    |  Delta Time Last Sent         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Option Type

   TBD = 0xXX (TBD)  [To be assigned by IANA] [RFC2780]

   Option Length

   8-bit unsigned integer. Length of the option, in octets, excluding
   the Option Type and Option Length fields. This field MUST be set to
   16.

    Scale Delta Time Last Received (SCALEDTLR)

   8-bit unsigned integer.  This is the scaling value for the Delta Time
   Last Received (DELTATLR) field.  The possible values are from 0-255. 
   See Section 4 for further discussion on Timing Considerations and
   formatting of the scaling values.

   Scale Delta Time Last Sent (SCALEDTLS)

   8-bit signed integer.  This is the scaling value for the Delta Time
   Last Sent (DELTATLS) field.  The possible values are from 0 to 255.

   Packet Sequence Number This Packet (PSNTP)

   16-bit unsigned integer.  This field will wrap. It is intended for
   use while analyzing packet traces.

   Initialized at a random number and incremented monotonically for each
   packet of the session flow of the 5-tuple.  The random number
   initialization is intended to make it harder to spoof and insert such
   packets.

 

Elkins                 Expires September 14, 2017               [Page 9]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Operating systems MUST implement a separate packet sequence number
   counter per 5-tuple.  

   Packet Sequence Number Last Received (PSNLR)

   16-bit unsigned integer.  This is the PSNTP of the packet last
   received on the 5-tuple.

   Delta Time Last Received (DELTATLR)

   A 16-bit unsigned integer field.  The value is set according to the
   scale in SCALEDTLR.

   Delta Time Last Received = (Send time packet 2 - Receive time packet
   1)

   Delta Time Last Sent (DELTATLS)

   A 16-bit unsigned integer field.   The value is set according to the
   scale in SCALEDTLS.

   Delta Time Last Sent = (Receive time packet 2 - Send time packet 1)

   Option Type

   In keeping with RFC2460[RFC2460], the two high order bits of the
   Option Type field are encoded to indicate specific processing of the
   option; for the PDM destination option, these two bits MUST be set to
   00.

   The third high order bit of the Option Type specifies whether or not
   the Option Data of that option can change en-route to the packet's
   final destination.

   In the PDM, the value of the third high order bit MUST be 0.  

3.2.2 Base Unit for Time Measurement

   A time differential is always a whole number in a CPU; it is the unit
   specification -- hours, seconds, nanoseconds -- that determine what
   the numeric value means. For PDM, we establish the base time unit as
   1 attosecond (asec). This allows for a common unit and scaling of the
   time differential among all IP stacks and hardware implementations.

   Note that we are trying to provide the ability to measure both time
 

Elkins                 Expires September 14, 2017              [Page 10]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   differentials that are extremely small, and time differentials in a
   DTN-type environment where the delays may be very great. To store a
   time differential in just 16 bits that must range in this way will
   require some scaling of the time differential value.

   One issue is the conversion from the native time base in the CPU
   hardware of whatever device is in use to some number of attoseconds.
   It might seem this will be an astronomical number, but the conversion
   is straightforward. It involves multiplication by an appropriate
   power of 10 to change the value into a number of attoseconds. Then,
   to scale the value so that it fits into DELTATLR or DELTATLS, the
   value is shifted by of a number of bits, retaining the 16 high-order
   or most significant bits. The number of bits shifted becomes the
   scaling factor, stored as SCALEDTLR or SCALEDTLS, respectively. For a
   full description of this process, including examples, please see
   Appendix A.

3.2.3 Considerations of this time-differential representation

   There are a few considerations to be taken into account with this
   representation of a time differential. The first is whether there are
   any limitations on the maximum or minimum time differential that can
   be expressed using method of a delta value and a scaling factor. The
   second is the amount of imprecision introduced by this method.

3.2.3.1 Limitations with this encoding method

   The DELTATLS and DELTATLR fields store only the 16 most-significant
   bits of the time differential value. Thus the range, excluding the
   scaling factor, is from 0 to 65535, or a maximum of 2**16-1.   This
   method is further described in [TRAM-TCPM].

   The actual magnitude of the time differential is determined by the
   scaling factor. SCALEDTLR and SCALEDTLS are 8-bit unsigned integers,
   so the scaling factor ranges from 0 to 255. The smallest number that
   can be represented would have a value of 1 in the delta field and a
   value of 0 in the associated scale field. This is the representation
   for 1 attosecond. Clearly this allows PDM to measure extremely small
   time differentials.

   On the other end of the scale, the maximum delta value is 65535, or
   FFFF in hexadecimal. If the maximum scale value of 255 is used, the
   time differential represented is 65535*2**255, which is over 3*10**55
   years, essentially, forever. So there appears to be no real
   limitation to the time differential that can be represented.

 

Elkins                 Expires September 14, 2017              [Page 11]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

3.2.3.2 Loss of precision induced by timer value truncation

   As PDM specifies the DELTATLR and DELTATLS values as 16-bit unsigned
   integers, any time the precision is greater than those 16 bits, there
   will be truncation of the trailing bits, with an accompanying loss of
   precision in the value.

   Any time differential value smaller than 65536 asec can be stored
   exactly in DELTATLR or DELTATLS, because the representation of this
   value requires at most 16 bits.

   Since the time differential values in PDM are measured in
   attoseconds, the range of values that would be truncated to the same
   encoded value is 2**(Scale)-1 asec.

   For example, the smallest time differential that would be truncated
   to fit into a delta field is

      1 0000 0000 0000 0000 = 65536 asec

   This value would be encoded as a delta value of 8000 (hexadecimal)
   with a scaling factor of 1. The value

      1 0000 0000 0000 0001 = 65537 asec

   would also be encoded as a delta value of 8000 with a scaling factor
   of 1. This actually is the largest value that would be truncated to
   that same encoded value. When the scale value is 1, the value range
   is calculated as 2**1 - 1, or 1 asec, which you can see is the
   difference between these minimum and maximum values.

   The scaling factor is defined as the number of low-order bits
   truncated to reduce the size of the resulting value so it fits into a
   16-bit delta field. If, for example, you had to truncate 12 bits, the
   loss of precision would depend on the bits you truncated. The range
   of these values would be

      0000 0000 0000 = 0 asec
   to
      1111 1111 1111 = 4095 asec

   So the minimum loss of precision would be 0 asec, where the delta
   value exactly represents the time differential, and the maximum loss
   of precision would be 4095 asec. As stated above, the scaling factor
   of 12 means the maximum loss of precision is 2**12-1 asec, or 4095
   asec.

   Compare this loss of precision to the actual time differential. The
 

Elkins                 Expires September 14, 2017              [Page 12]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   range of actual time differential values that would incur this loss
   of precision is from

   1000 0000 0000 0000 0000 0000 0000 = 2**27 asec or 134217728 asec
      to
   1111 1111 1111 1111 1111 1111 1111 = 2**28-1 asec or 268435455 asec

   Granted, these are small values, but the point is, any value between
   these two values will have a maximum loss of precision of 4095 asec,
   or about 0.00305% for the first value, as encoded, and at most
   0.001526% for the second. These maximum-loss percentages are
   consistent for all scaling values.

3.3 Header Placement

   The PDM Destination Option is placed as defined in RFC2460 [RFC2460].
   There may be a choice of where to place the Destination Options
   header. If using ESP mode, please see section 3.4 of this document
   for placement of the PDM Destination Options header.

   For each IPv6 packet header, the PDM MUST NOT appear more than once.
   However, an encapsulated packet MAY contain a separate PDM associated
   with each encapsulated IPv6 header.

3.4 Header Placement Using IPSec ESP Mode

   IPSec Encapsulating Security Payload (ESP) is defined in [RFC4303]
   and is widely used.  Section 3.1.1 of [RFC4303] discusses placement
   of Destination Options Headers. 

   The placement of PDM is different depending on if ESP is used in
   tunnel or transport mode. 

3.4.1 Using ESP Transport Mode 

   Below is the diagram from [RFC4303] discussing placement of headers.
   Note that Destination Options MAY be placed before or after ESP or
   both.  If using PDM in ESP transport mode, PDM MUST be placed after
   the ESP header so as not to leak information. 

 

Elkins                 Expires September 14, 2017              [Page 13]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

                          BEFORE APPLYING ESP 
             --------------------------------------- 
       IPv6  |             | ext hdrs |     |      | 
             | orig IP hdr |if present| TCP | Data | 
             --------------------------------------- 

                          AFTER APPLYING ESP 
             --------------------------------------------------------- 
       IPv6  | orig |hop-by-hop,dest*,|   |dest|   |    | ESP   | ESP| 
             |IP hdr|routing,fragment.|ESP|opt*|TCP|Data|Trailer| ICV| 
             --------------------------------------------------------- 
                                          |<--- encryption ---->| 
                                      |<------ integrity ------>| 

            * = if present, could be before ESP, after ESP, or both 

3.4.2 Using ESP Tunnel Mode 

   Below is the diagram from [RFC4303] discussing placement of headers. 

   Note that Destination Options MAY be placed before or after ESP or
   both in both the outer set of IP headers and the inner set of IP
   headers.

   In ESP tunnel mode, PDM MAY be placed before or after the ESP header
   or both. 

                          BEFORE APPLYING ESP 

          --------------------------------------- 
    IPv6  |             | ext hdrs |     |      | 
          | orig IP hdr |if present| TCP | Data | 
          --------------------------------------- 

                        AFTER APPLYING ESP 

          ------------------------------------------------------------ 
    IPv6  | new* |new ext |   | orig*|orig ext |   |    | ESP   | ESP| 
          |IP hdr| hdrs*  |ESP|IP hdr| hdrs *  |TCP|Data|Trailer| ICV| 
          ------------------------------------------------------------ 
                              |<--------- encryption ---------->| 
                          |<------------ integrity ------------>| 

          * = if present, construction of outer IP hdr/extensions and 
              modification of inner IP hdr/extensions is discussed in 
              the Security Architecture document. 

 

Elkins                 Expires September 14, 2017              [Page 14]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   As a completely new IP packet will be made, it means that PDM
   information for that packet does not contain any information from the
   inner packet, i.e. the PDM information will NOT be based on the
   transport layer (TCP, UDP, etc) ports etc in the inner header, but
   will be specific to the ESP flow. 

   If PDM information for the inner packet is desired, the original host
   sending the inner packet needs to put PDM header in the tunneled
   packet, and then the PDM information will be specific for that
   stream.

3.5 Implementation Considerations

3.5.1 PDM Activation

   The PDM destination options extension header MUST be explicitly
   turned on by each stack on a host node by administrative action. The
   default value of PDM is off.

   PDM MUST NOT be turned on merely if a packet is received with a PDM
   header. The received packet could be spoofed by another device.

3.5.2 PDM Timestamps

   The PDM timestamps are intended to isolate wire time from server or
   host time, but may necessarily attribute some host processing time to
   network latency.

   RFC2330 [RFC2330] "Framework for IP Performance Metrics" describes
   two notions of wire time in section 10.2.  These notions are only
   defined in terms of an Internet host H observing an Internet link L
   at a particular location:

   +    For a given IP packet P, the 'wire arrival time' of P at H on L
   is the first time T at which any bit of P has appeared at H's
   observational position on L.

   +    For a given IP packet P, the 'wire exit time' of P at H on L is
   the first time T at which all the bits of P have appeared at H's
   observational position on L.

   This specification does not define the exact H's observing position
   on L. That is left for the deployment setups to define. However, the
   position where PDM timestamps are taken SHOULD be as close to the
   physical network interface as possible.  Not all implementations will
   be able to achieve the ideal level of measurement.

 

Elkins                 Expires September 14, 2017              [Page 15]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

3.6 Dynamic Configuration Options

   If implemented, each operating system MUST have a default
   configuration parameter, e.g. diag_header_sys_default_value=yes/no.
   The operating system MAY also have a dynamic configuration option to
   change the configuration setting as needed.

   If the PDM destination options extension header is used, then it MAY
   be turned on for all packets flowing through the host, applied to an
   upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP
   address only.  These are at the discretion of the implementation.

3.6 5-tuple Aging

   Within the operating system, metrics must be kept on a 5-tuple basis.

   The question comes of when to stop keeping data or restarting the
   numbering for a 5-tuple.  For example, in the case of TCP, at some
   point, the connection will terminate.  Keeping data in control blocks
   forever, will have unfortunate consequences for the operating system.

   So, the recommendation is to use a known aging parameter such as Max
   Segment Lifetime (MSL) as defined in Transmission Control Protocol
   [RFC0793] to reuse or drop the control block.  The choice of aging
   parameter is left up to the implementation.

4 Security Considerations

   PDM may introduce some new security weaknesses.

4.1. SYN Flood and Resource Consumption Attacks

   PDM needs to calculate the deltas for time and keep track of the
   sequence numbers. This means that control blocks must be kept at the
   end hosts per 5-tuple.   Any time a control block is kept, an
   attacker can try to mis-use the control blocks such that there is a
   compromise of the end host.

   PDM is used only at the end hosts and the control blocks are only
   kept at the end host and not at routers or middle boxes.   Remember,
   PDM is an implementation of the Destination Option extension header.
   

   A "SYN flood" type of attack succeeds because a TCP SYN packet is
   small but it causes the end host to start creating a place holder for
   the session such that quite a bit of control block and other storage
   is used.   This is an asynchronous type of attack in that a small
   amount of work by the attacker creates a large amount of work by the
 

Elkins                 Expires September 14, 2017              [Page 16]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   resource attacked.

   For PDM, the amount of data to be kept is quite small. That is, the
   control block is quite lightweight.  Concerns about SYN Flood and
   other type of resource consumption attacks (memory, processing power,
   etc) can be alleviated by having a limit on the number of control
   block entries.

   We recommend that implementation of PDM SHOULD have a limit on the
   number of control block entries.    

4.2  Pervasive monitoring

   Since PDM passes in the clear, a concern arises as to whether the
   data can be used to fingerprint the system or somehow obtain
   information about the contents of the payload.  

   Let us discuss fingerprinting of the end host first. It is possible
   that seeing the pattern of deltas or the absolute values could give
   some information as to the speed of the end host - that is, if it is
   a very fast system or an older, slow device.   This may be useful to
   the attacker.  However, if the attacker has access to PDM, the
   attacker also has access to the entire packet and could make such a
   deduction based merely on the time frames elapsed between packets
   WITHOUT PDM.  

   As far as deducing the content of the payload, it appears to us that
   PDM is quite unhelpful in this regard.

4.3 PDM as a Covert Channel

   PDM provides a set of fields in the packet which could be used to
   leak data.   But, there is no real reason to suspect that PDM would
   be chosen rather than another part of the payload or another
   Extension Header.

   A firewall or another device could sanity check the fields within the
   PDM but randomly assigned sequence numbers and delta times might be
   expected to vary widely.   The biggest problem though is how an
   attacker would get access to PDM in the first place to leak data. 
   The attacker would have to either compromise the end host or have Man
   in the Middle (MitM).  It is possible that either one could change
   the fields.   But, then the other end host would get sequence numbers
   and deltas that don't make any sense.   Presumably, one is using PDM
   and doing packet tracing for diagnostic purposes, so the changes
   would be obvious.    It is conceivable that someone could compromise
   an end host and make it start sending packets with PDM without the
 

Elkins                 Expires September 14, 2017              [Page 17]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   knowledge of the host.  But, again, the bigger problem is the
   compromise of the end host.   Once that is done, the attacker
   probably has better ways to leak data.

   Having said that, an implementation SHOULD stop using PDM if it gets
   some number of "nonsensical" sequence numbers.  

4.4 Timing Attacks 

   The fact that PDM can help in the separation of node processing time
   from network latency brings value to performance monitoring.  Yet, it
   is this very characteristic of PDM which may be misused to make
   certain new type of timing attacks against protocols and
   implementations possible. 

   Depending on the nature of the cryptographic protocol used, it may be
   possible to leak the long term credentials of the device.  For
   example, if an attacker is able to create an attack which causes the
   enterprise to turn on PDM to diagnose the attack, then the attacker
   might use PDM during that debugging time to launch a timing attack
   against the long term keying material used by the cryptographic
   protocol.

   An implementation may want to be sure that PDM is enabled only for
   certain ip addresses, or only for some ports.  Additionally, we
   recommend that the implementation SHOULD require an explicit restart
   of monitoring after a certain timeperiod (for example for 1 hour), to
   make sure that PDM is not accidently left on after debugging has been
   done etc. 

   Even so, if using PDM, we introduce the concept of user "Consent to
   be Measured" as a pre-requisite for using PDM.  Consent is common in
   enterprises and with some subscription services. So, if with PDM, we
   recommend that the user SHOULD consent to its use.

5 IANA Considerations

   This draft requests an Option Type assignment in the Destination
   Options and Hop-by-Hop Options sub-registry of Internet Protocol
   Version 6 (IPv6) Parameters [ref to RFCs and URL below].

   http://www.iana.org/assignments/ipv6-parameters/ipv6-
   parameters.xhtml#ipv6-parameters-2

 

Elkins                 Expires September 14, 2017              [Page 18]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Hex Value      Binary Value      Description             Reference 
                  act chg rest
   -------------------------------------------------------------------  
   TBD             TBD            Performance and          [This draft]
                                  Diagnostic Metrics 
                                  (PDM)          

6 References

6.1 Normative References

   [RFC0793]  Postel, J., "Transmission Control Protocol", STD 7, RFC
   793, September 1981.

   [RFC1122]  Braden, R., "Requirements for Internet Hosts --
   Communication Layers", RFC 1122, October 1989.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
   Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2460]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
   (IPv6) Specification", RFC 2460, December 1998.

   [RFC2681]  Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
   Delay Metric for IPPM", RFC 2681, September 1999.

   [RFC2780]  Bradner, S. and  V. Paxson, "IANA Allocation Guidelines
   For Values In the Internet Protocol and Related Headers", BCP 37, RFC
   2780, March 2000.

   [RFC4303] Kent, S, "IP Encapsulating Security Payload (ESP)",  RFC
   4303, December 2005.

6.2 Informative References

   [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
   "Framework for IP Performance Metrics", RFC 2330, May 1998.

   [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP
   Timestamp Option-01", Internet Draft,  July 2013. [Work in Progress]

 

Elkins                 Expires September 14, 2017              [Page 19]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

Appendix A : Timing Time Differential Calculations

   The time counter in a CPU is a binary whole number, representing a
   number of milliseconds (msec), microseconds (usec) or even
   picoseconds (psec). Representing one of these values as attoseconds
   (asec) means multiplying by 10 raised to some exponent. Refer to this
   table of equalities:

         Base value      = # of sec      = # of asec       1000s of asec
         ---------------   -------------   -------------   -------------
         1 second          1 sec           10**18 asec     1000**6 asec 
         1 millisecond     10**-3  sec     10**15 asec     1000**5 asec 
         1 microsecond     10**-6  sec     10**12 asec     1000**4 asec 
         1 nanosecond      10**-9  sec     10**9  asec     1000**3 asec 
         1 picosecond      10**-12 sec     10**6  asec     1000**2 asec 
         1 femtosecond     10**-15 sec     10**3  asec     1000**1 asec 

   For example, if you have a time differential expressed in
   microseconds, since each microsecond is 10**12 asec, you would
   multiply your time value by 10**12 to obtain the number of
   attoseconds. If you time differential is expressed in nanoseconds,
   you would multiply by 10**9 to get the number of attoseconds.

   The result is a binary value that will need to be shortened by a
   number of bits so it will fit into the 16-bit PDM DELTA field.

   The next step is to divide by 2 until the value is contained in just
   16 significant bits. The exponent of the value in the last column of
   of the table is useful here; the initial scaling factor is that
   exponent multiplied by 10. This is the minimum number of low-order
   bits to be shifted-out or discarded. It represents dividing the time
   value by 1024 raised to that exponent.

   The resulting value may still be too large to fit into 16 bits, but
   can be normalized by shifting out more bits (dividing by 2) until the
   value fits into the 16-bit DELTA field. The number of extra bits
   shifted out is then added to the scaling factor. The scaling factor,
   the total number of low-order bits dropped, is the SCALEDTL value.

   For example: say an application has these start and finish timer
   values (hexadecimal values, in microseconds): 

         Finish:      27C849234 usec    (02:57:58.997556) 
         -Start:      27C83F696 usec    (02:57:58.957718) 
         ==========   =========         =============== 
         Difference   9B9E usec         00.039838 sec or 39838 usec 

   To convert this differential value to binary attoseconds, multiply
 

Elkins                 Expires September 14, 2017              [Page 20]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   the number of microseconds by 10**12. Divide by 1024**4, or simply
   discard 40 bits from the right. The result is 36232, or 8D88 in hex,
   with a scaling factor or SCALEDTL value of 40. 

   For another example, presume the time differential is larger, say
   32.311072 seconds, which is 32311072 usec. Each microsecond is 10**12
   asec, so multiply by 10**12, giving the hexadecimal value
   1C067FCCAE8120000. Using the initial scaling factor of 40, drop the
   last 10 characters (40 bits) from that string, giving 1C067FC. This
   will not fit into a DELTA field, as it is 25 bits long. Shifting the
   value to the right another 9 bits results in a DELTA value of E033,
   with a resulting scaling factor of 49.

   When the time differential value is a small number, regardless of the
   time unit, the exponent trick given above is not useful in
   determining the proper scaling value. For example, if the time
   differential is 3 seconds and you want to convert that directly, you
   would follow this path:

         3 seconds = 3*10**18 asec (decimal)
                   = 29A2241AF62C0000 asec (hexadecimal)

   If you just truncate the last 60 bits, you end up with a delta value
   of 2 and a scaling factor of 60, when what you really wanted was a
   delta value with more significant digits. The most precision with
   which you can store this value in 16 bits is A688, with a scaling
   factor of 46.

Appendix B: Sample Packet Flows

B.1 PDM Flow - Simple Client Server 

   Following is a sample simple flow for the PDM with one packet sent
   from Host A and one packet received by Host B.  The PDM does not
   require time synchronization between Host A and Host B.  The
   calculations to derive meaningful metrics for network diagnostics are
   shown below each packet sent or received.

B.1.1 Step 1

   Packet 1 is sent from Host A to Host B.  The time for Host A is set
   initially to 10:00AM.

   The time and packet sequence number are saved by the sender
   internally.  The packet sequence number and delta times are sent in
   the packet.

 

Elkins                 Expires September 14, 2017              [Page 21]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

      Packet 1

                          +----------+             +----------+
                          |          |             |          |
                          |   Host   | ----------> |   Host   |
                          |    A     |             |    B     |
                          |          |             |          |
                          +----------+             +----------+

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:     25 
   PSNLR    : Packet Sequence Number Last Received:   - 
   DELTATLR : Delta Time Last Received:               - 
   SCALEDTLR: Scale of Delta Time Last Received:      0
   DELTATLS : Delta Time Last Sent:                   -
   SCALEDTLS: Scale of Delta Time Last Sent:          0 

   Internally, within the sender, Host A, it must keep:

   Packet Sequence Number of the last packet sent:     25
   Time the last packet was sent:                10:00:00

   Note, the initial PSNTP from Host A starts at a random number.  In
   this case, 25.  The time in these examples is shown in seconds for
   the sake of simplicity.

B.1.2 Step 2

   Packet 1 is received at Host B.  Its time is set to one hour later
   than Host A.  In this case, 11:00AM

   Internally, within the receiver, Host B, it must note:

   Packet Sequence Number of the last packet received:    25
   Time the last packet was received                 :    11:00:03

   Note, this timestamp is in Host B time.  It has nothing whatsoever to
   do with Host A time.  The Packet Sequence Number of the last packet
   received will become PSNLR which will be sent out in the packet sent
   by Host B in the next step.  The time last received will be used to
   calculate the DELTALR value to be sent out in the packet sent by Host
   B in the next step.

 

Elkins                 Expires September 14, 2017              [Page 22]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

B.1.3 Step 3

   Packet 2 is sent by Host B to Host A.  Note, the initial packet
   sequence number (PSNTP) from Host B starts at a random number.  In
   this case, 12.   Before sending the packet, Host B does a calculation
   of deltas.  Since Host B knows when it is sending the packet, and it
   knows when it received the previous packet, it can do the following
   calculation:

   Sending time : packet 2 - receive time : packet 1

   We will call the result of this calculation: Delta Time Last Received
   (DELTATLR)

   Note, both sending time and receive time are saved internally in Host
   B.  They do not travel in the packet. Only the Delta is in the
   packet.

   Assume that within Host B is the following:

   Packet Sequence Number of the last packet received:     25
   Time the last packet was received:                      11:00:03
   Packet Sequence Number of this packet:                  12
   Time this packet is being sent:                         11:00:07

   We can now calculate a delta value to be sent out in the packet.
   DELTATLR becomes:

   4 seconds = 11:00:07 - 11:00:03  = 3782DACE9D900000 asec

   This is the derived metric: Server Delay.   The time and scaling
   factor must be converted; in this case, the time differential is
   DE0B, and the scaling factor is 2E, or 46 in decimal.  Then, these
   values, along with the packet sequence numbers will be sent to Host A
   as follows:

      Packet 2

                          +----------+             +----------+
                          |          |             |          |
                          |   Host   | <---------- |   Host   |
                          |    A     |             |    B     |
                          |          |             |          |
                          +----------+             +----------+

 

Elkins                 Expires September 14, 2017              [Page 23]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:    12
   PSNLR    : Packet Sequence Number Last Received:  25
   DELTATLR : Delta Time Last Received:              DE0B (4 seconds)
   SCALEDTLR: Scale of Delta Time Last Received:     2E (46 decimal)
   DELTATLS : Delta Time Last Sent:                   -
   SCALEDTLS: Scale of Delta Time Last Sent:          0

   The metric left to be calculated is the Round-Trip Delay. This will
   be calculated by Host A when it receives Packet 2.

B.1.4 Step 4

   Packet 2 is received at Host A.  Remember, its time is set to one
   hour earlier than Host B. Internally, it must note:

      Packet Sequence Number of the last packet received:    12
      Time the last packet was received                 :    10:00:12

   Note, this timestamp is in Host A time.  It has nothing whatsoever to
   do with Host B time.

   So, now, Host A can calculate total end-to-end time. That is:

   End-to-End Time = Time Last Received - Time Last Sent

   For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was
   received by Host A at 10:00:12 so:

   End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT
   delay combined).    This time may also be called total Overall Round-
   Trip Time (RTT) which includes Network RTT and Host Response Time. 

   This derived metric we will call Delta Time Last Sent (DELTATLS)

   We can now also calculate round trip delay.  The formula is:

   Round trip delay = (Delta Time Last Sent - Delta Time Last Received)

   Or:

   Round trip delay = 12 - 4 or 8

   Now, the only problem is that at this point all metrics are in Host A
   only and not exposed in a packet. To do that, we need a third packet.

   Note:  this simple example assumes one send and one receive.   That
 

Elkins                 Expires September 14, 2017              [Page 24]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   is done only for purposes of explaining the function of the PDM.  In
   cases where there are multiple packets returned, one would take the
   time in the last packet in the sequence.   The calculations of such
   timings and intelligent processing is the function of post-processing
   of the data. 

B.1.5 Step 5

   Packet 3 is sent from Host A to Host B.

                          +----------+             +----------+
                          |          |             |          |
                          |   Host   | ----------> |   Host   |
                          |    A     |             |    B     |
                          |          |             |          |
                          +----------+             +----------+

   PDM Contents:

   PSNTP    : Packet Sequence Number This Packet:    26 
   PSNLR    : Packet Sequence Number Last Received:  12 
   DELTATLR : Delta Time Last Received:               0 
   SCALEDTLS: Scale of Delta Time Last Received       0
   DELTATLS : Delta Time Last Sent:                A688 (scaled value)
   SCALEDTLR: Scale of Delta Time Last Received:     30 (48 decimal) 

   To calculate Two-Way Delay, any packet capture device may look at
   these packets and do what is necessary.

B.2 Other Flows

   What we have discussed so far is a simple flow with one packet sent
   and one returned.   Let's look at how PDM may be useful in other
   types of flows.

B.2.1 PDM Flow - One Way Traffic

   The flow on a particular session may not be a send-receive paradigm.
   Let us consider some other situations.   In the case of a one-way
   flow, one might see the following:

   Note: The time is expressed in generic units for simplicity.  That
   is, these values do not represent a number of attoseconds,
   microseconds or any other real units of time.

 

Elkins                 Expires September 14, 2017              [Page 25]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Packet   Sender      PSN            PSN        Delta Time  Delta Time
                     This Packet    Last Recvd    Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0            0   
   2        Server       2              0              0            5 
   3        Server       3              0              0           12 
   4        Server       4              0              0           20

   What does this mean and how is it useful?

   In a one-way flow, only the Delta Time Last Sent will be seen as
   used.  Recall, Delta Time Last Sent is the difference between the
   send of one packet from a device and the next.  This is a measure of
   throughput for the sender - according to the sender's point of view.
   That is, it is a measure of how fast is the application itself (with
   stack time included) able to send packets.  

   How might this be useful?  If one is having a performance issue at
   the client and sees that packet 2, for example, is sent after 5 time
   units from the server but takes 10 times that long to arrive at the
   destination,  then one may safely conclude that there are delays in
   the path other than at the server which may be causing the delivery
   issue of that packet.  Such delays may include the network links,
   middle-boxes, etc.

   Now, true one-way traffic is quite rare.   What people often mean by
   "one-way" traffic is an application such as FTP where a group of
   packets (for example, a TCP window size worth) is sent, then the
   sender waits for acknowledgment.  This type of flow would actually
   fall into the "multiple-send" traffic model.

B.2.2 PDM Flow - Multiple Send Traffic

   Assume that two packets are sent for each ACK from the server.  For
   example, a TCP flow will do this, per RFC1122 [RFC1122] Section-
   4.2.3.

   Packet   Sender      PSN            PSN       Delta Time  Delta Time 
                    This Packet    Last Recvd    Last Recvd  Last Sent
   =====================================================================
   1        Server       1              0              0           0    
   2        Server       2              0              0           5
   3        Client       1              2             20           0
   4        Server       3              1             10          15 

   How might this be used?

 

Elkins                 Expires September 14, 2017              [Page 26]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Notice that in packet 3, the client has a value of Delta Time Last
   received of 20.   Recall that Delta Time Last Received is the Send
   time of packet 3 - receive time of packet 2.   So, what does one know
   now?   In this case, Delta Time Last Received is the processing time
   for the Client to send the next packet.    

   How to interpret this depends on what is actually being sent. 
   Remember, PDM is not being used in isolation, but to supplement the
   fields found in other headers.   Let's take some examples:

   1.  Client is sending a standalone TCP ACK.   One would find this by
   looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back the ACK.   This may or may not
   be interesting.

   2.  Client is sending data with the packet.  Again, one would find
   this by looking at the payload length in the IPv6 header and the TCP
   Acknowledgement field in the TCP header.   So, in this case, the
   client is taking 20 units to send back data.   This may represent
   "User Think Time".   Again, this may or may not be interesting, in
   isolation.   But, if there is a performance problem receiving data at
   the server, then taken in conjunction with RTT or other packet timing
   information, this information may be quite interesting.

   Of course, one also needs to look at the PSN Last Received field to
   make sure of the interpretation of this data.   That is,  to make
   sure that the Delta Last Received corresponds to the packet of
   interest.

   The benefits of PDM are that we have such information available in a
   uniform manner for all applications and all protocols without
   extensive changes required to applications.

B.2.3 PDM Flow - Multiple Send with Errors

   Let us now look at a case of how PDM may be able to help in a case of
   TCP retransmission and add to the information that is sent in the TCP
   header.

   Assume that three packets are sent with each send from the server.

 

Elkins                 Expires September 14, 2017              [Page 27]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   From the server, this is what is seen.

   Pkt Sender    PSN        PSN     Delta Time Delta Time TCP     Data
               This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server      1        0           0          0      123     100
   2   Server      2        0           0          5      223     100
   3   Server      3        0           0          5      333     100

   The client, however, does not receive all the packets.  From the
   client, this is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN    Delta Time Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server     1         0          0          0        123   100
   2   Server     3         0          0          5        333   100

   Let's assume that the server now retransmits the packet.  (Obviously,
   a duplicate acknowledgment sequence for fast retransmit or a
   retransmit timeout would occur.  To illustrate the point, these
   packets are being left out.)

   So, then if a TCP retransmission is done, then from the client, this
   is what is seen for the packets sent from the server.

   Pkt Sender    PSN        PSN    Delta Time  Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd   LastSent   SEQ    Bytes
   =====================================================================
   1   Server    4          0           0          30    223      100

   The server has resent the old packet 2 with TCP sequence number of
   223.   The retransmitted packet now has a PSN This Packet value of 4.

   The Delta Last Sent is 30 - the time between sending the packet with
   PSN of 3 and this current packet.

   Let's say that packet 4 is lost again.  Then, after some amount of
   time (RTO) then the packet with TCP sequence number of 223 is resent.

   From the client, this is what is seen for the packets sent from the
   server.

   Pkt Sender    PSN        PSN    Delta Time Delta Time TCP    Data
              This Pkt  LastRecvd  LastRecvd  LastSent   SEQ    Bytes
   =====================================================================
   1   Server    5          0          0        60       223     100
 

Elkins                 Expires September 14, 2017              [Page 28]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   If now, this packet arrives at the destination, one has a very good
   idea that packets exist which are being sent from the server as
   retransmissions and not arriving at the client.   This is because the
   PSN of the resent packet from the server is 5 rather than 4.  If we
   had used TCP sequence number alone, we would never have seen this
   situation.  The TCP sequence number in all situations is 223.

   This situation would be experienced by the user of the application
   (the human being actually sitting somewhere) as a "hangs" or long
   delay between packets.  On large networks, to diagnose problems such
   as these where packets are lost somewhere on the network,  one has to
   take multiple traces to find out exactly where.

   The first thing is to start with doing a trace at the client and the
   server.  So, we can see if the server sent a particular packet and
   the client received it.  If the client did not receive it, then we
   start tracking back to trace points at the router right after the
   server and the router right before the client.  Did they get these
   packets which the server has sent?   This is a time consuming
   activity.

   With PDM,  we can speed up the diagnostic time because we may be able
   to use only the trace taken at the client to see what the server is
   sending.   

Appendix C: Potential Overhead Considerations

   One might wonder as to the potential overhead of PDM.  First, PDM is
   entirely optional.   That is, a site may choose to implement PDM or
   not as they wish.   If they are happy with the costs of PDM vs. the
   benefits, then the choice should be theirs.

   Below is a table outlining the potential overhead in terms of
   additional time to deliver the response to the end user for various
   assumed RTTs.

   Bytes         RTT         Bytes       Bytes      New      Overhead
   in Packet                Per Millisec in PDM     RTT        
   =====================================================================
   1000       1000 milli         1       16     1016.000  16.000 milli
   1000        100 milli        10       16      101.600   1.600 milli
   1000         10 milli       100       16       10.160    .160 milli
   1000          1 milli      1000       16        1.016    .016 milli

   Below are some examples of actual RTTs for packets traversing large
   enterprise networks.   The first example is for packets going to
   multiple business partners.
 

Elkins                 Expires September 14, 2017              [Page 29]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

    Bytes         RTT        Bytes       Bytes      New     Overhead
   in Packet                Per Millisec in PDM     RTT
   =====================================================================
   1000       17 milli         58        16       17.360   .360 milli

   The second example is for packets at a large enterprise customer
   within a data center.  Notice that the scale is now in microseconds
   rather than milliseconds.

    Bytes       RTT           Bytes      Bytes      New      Overhead
   in Packet                Per Microsec in PDM     RTT    
   =====================================================================
   1000       20 micro        50         16       20.320   .320 micro

Acknowledgments

   The authors would like to thank Keven Haining, Al Morton, Brian
   Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick
   Troth for their comments and assistance.  We would also like to thank
   Tero Kivinen and Jouni Korhonen for their detailed and perceptive
   reviews.

Authors' Addresses

   Nalini Elkins
   Inside Products, Inc.
   36A Upper Circle
   Carmel Valley, CA 93924
   United States
   Phone: +1 831 659 8360
   Email: nalini.elkins@insidethestack.com
   http://www.insidethestack.com

   Robert M. Hamilton
   Chemical Abstracts Service
   A Division of the American Chemical Society
   2540 Olentangy River Road
   Columbus, Ohio  43202
   United States
   Phone: +1 614 447 3600 x2517
   Email: rhamilton@cas.org
   http://www.cas.org

 

Elkins                 Expires September 14, 2017              [Page 30]
INTERNET DRAFT        ietf-ippm-6man-pdm-option-09        March 13, 2017

   Michael S. Ackermann
   Blue Cross Blue Shield of Michigan
   P.O. Box 2888
   Detroit, Michigan 48231
   United States
   Phone: +1 310 460 4080
   Email: mackermann@bcbsm.com
   http://www.bcbsm.com

Elkins                 Expires September 14, 2017              [Page 31]