Round-Trip Packet Loss Metrics
draft-ietf-ippm-rt-loss-05

[Ballot comment]
- Section 4.1.  Name: Type-P-Round-trip-Loss
I double checked this name with RFC 2680
1. It should be Type-P-Round-trip-Packet-Loss throughout the document
2. I opened this errata on RFC 2680 http://www.rfc-editor.org/errata_search.php?eid=3186

I'll trust the IPPM community on this one.
No need to discuss further

Regards, Benoit.

[Ballot discuss]
I have some clarifying questions for the point 1. and 2. that will determine whether these are real DISCUSS's or not.
A 10 minutes discussion with Al would help a lot

1. Addressed.


2.
Section 4.3
      o  the Dst sent a Type-P packet back to the Src as immediately as possible, and

Why is this even useful to mention "as immediately as possible"?
I mean: if you have to use round-trip packet loss (instead of one-way packet loss), it's because you're not able to install a "responder" application on the target device. Therefore, you have no control at all on that target device. And you are forced to use a protocol such as ICMP. So, why is this even useful to say "as immediately as possible" if you have no control on that target device?
The sentence "the Dst sent a Type-P packet back to the Src as immediately as possible" only makes sense in the case of one-way delay metric.

I have the same issue with your new proposed text (discussed with Adrian)
  o  the Dst sent a Type-P packet back to the Src as quickly as
      possible (certainly less than Tmax, and fast enough for the
      intended purpose), and

I have the same issue with your new proposed text in section 4.4 (discussed with Adrian, AFAIK)
  We add the following guidance regarding the responder process to
  "send a Type-P packet back to the Src as quickly as possible".

  A response that was not generated within Tmax is inadequate for any
  realistic test, and the Src will discard such responses.  A responder
  that serves typical round-trip loss testing (which is relevant to
  higher-layer application performance) SHOULD produce a response in 1
  second or less.  A responder that is unable to satisfy this
  requirement SHOULD log the fact so that an operator can adjust the
  load and priorities as necessary.  Analysis of responder time-stamps
  [RFC5357] that finds responses are not generated in a timely fashion
  SHOULD result in operator notification, and the operator SHOULD
  suspend tests to the responder since it may be overloaded.
  Additional measurement considerations are described in Section 8,
  below.

For example, "A responder that is unable to satisfy this requirement SHOULD log the fact so that an operator can adjust the load and priorities as necessary. "
I've been doing IP SLA measurements for years with Cisco boxes, and I would only use round trip delay and loss metrics when I can't touch the target device. And here you're asking the target device to do a task for you in case of round trip loss...
Note: the default configuration for SLA measurement is to put a responder on the target device, and to measure in both directions the one way delay, the loss, and jitter.

Or maybe, the metric in this draft can only be used with the TWAMP protocol, which I believe requires some configuration on the target device?
However, it appears it's not a requirement as TWAMP is mentioned as one example in

    8.  Measurement Considerations and Calibration

  Prior to conducting this measurement, the participating hosts MUST be
  configured to send and receive test packets of the chosen Type-P.
  Standard measurement protocols are capable of this task [RFC5357],
  but any reliable method is sufficient.

Next question: why do mention "but any reliable method is sufficient.". It means that that metric can't be used with ICMP?

Anyway, it needs some clarifications.

3. Addressed.

[Ballot discuss]
I have some clarifying questions for the point 1. and 2. that will determine whether these are real DISCUSS's or not.
A 10 minutes discussion with Al would help a lot

1. Addressed.


2.
Section 4.3
      o  the Dst sent a Type-P packet back to the Src as immediately as possible, and

Why is this even useful to mention "as immediately as possible"?
I mean: if you have to use round-trip packet loss (instead of one-way packet loss), it's because you're not able to install a "responder" application on the target device. Therefore, you have no control at all on that target device. And you are forced to use a protocol such as ICMP. So, why is this even useful to say "as immediately as possible" if you have no control on that target device?
The sentence "the Dst sent a Type-P packet back to the Src as immediately as possible" only makes sense in the case of one-way delay metric.

I have the same issue with your new proposed text (discussed with Adrian)
  o  the Dst sent a Type-P packet back to the Src as quickly as
      possible (certainly less than Tmax, and fast enough for the
      intended purpose), and

I have the same issue with your new proposed text in section 4.4 (discussed with Adrian, AFAIK)
  We add the following guidance regarding the responder process to
  "send a Type-P packet back to the Src as quickly as possible".

  A response that was not generated within Tmax is inadequate for any
  realistic test, and the Src will discard such responses.  A responder
  that serves typical round-trip loss testing (which is relevant to
  higher-layer application performance) SHOULD produce a response in 1
  second or less.  A responder that is unable to satisfy this
  requirement SHOULD log the fact so that an operator can adjust the
  load and priorities as necessary.  Analysis of responder time-stamps
  [RFC5357] that finds responses are not generated in a timely fashion
  SHOULD result in operator notification, and the operator SHOULD
  suspend tests to the responder since it may be overloaded.
  Additional measurement considerations are described in Section 8,
  below.

For example, "A responder that is unable to satisfy this requirement SHOULD log the fact so that an operator can adjust the load and priorities as necessary. "
I've been doing IP SLA measurements for years with Cisco boxes, and I would only use round trip delay and loss metrics when I can't touch the target device. And here you're asking the target device to do a task for you in case of round trip loss...
Note: the default configuration for SLA measurement is to put a responder on the target device, and to measure in both directions the one way delay, the loss, and jitter.

Or maybe, the metric in this draft can only be used with the TWAMP protocol, which I believe requires some configuration on the target device?
However, it appears it's not a requirement as TWAMP is mentioned as one example in

    8.  Measurement Considerations and Calibration

  Prior to conducting this measurement, the participating hosts MUST be
  configured to send and receive test packets of the chosen Type-P.
  Standard measurement protocols are capable of this task [RFC5357],
  but any reliable method is sufficient.

Next question: why do mention "but any reliable method is sufficient.". It means that that metric can't be used with ICMP?

Anyway, it needs some clarifications.

3.
In section 4.3

  Following the precedent of[RFC2681], we make the simplifying
  assertion:

  Type-P-Round-trip-Loss(Src->Dst) = Type-P-Round-trip-Loss(Dst->Src)

While I could agree that Type-P-Round-trip(Src->Dst) = Type-P-Round-trip(Dst->Src), at some conditions, I disagree with the assertion that if you loose 50% packets, you can conclude that you lost 25% in each direction.

[Ballot comment]
- Section 4.1.  Name: Type-P-Round-trip-Loss
I double checked this name with RFC 2680
1. It should be Type-P-Round-trip-Packet-Loss throughout the document
2. I opened this errata on RFC 2680 http://www.rfc-editor.org/errata_search.php?eid=3186

[Ballot comment]
  Please consider the comments raised by the Gen-ART Review by
  Ben Campbell on 10-Apr-2012.  The review can be found here:
  http://www.ietf.org/mail-archive/web/gen-art/current/msg07340.html

[Ballot comment]
I had a discuss to check that Sandy Murphy's secdir review
comments had been taken  into account. I asked and wasn't
told they hadn't been, so I've cleared.

[Ballot comment]

I had a discuss to check that Sandy Murphy's secdir review
comments had been  into account. I asked and wasn't told
they hadn't been.

[Ballot discuss]
I have some clarifying questions for the point 1. and 2. that will determine whether these are real DISCUSS's or not.
A 10 minutes discussion with Al would help a lot

1.
Looking at the list of documents at http://datatracker.ietf.org/wg/ippm/, I see that the first set of metrics where IP or IPPM related
http://tools.ietf.org/html/rfc2680 ->  A One-way Packet Loss Metric for IPPM
http://tools.ietf.org/html/rfc2681 -> A Round-trip Delay Metric for IPPM
...

then I see that that IP (or IPPM to be more precise) is not included any longer
http://datatracker.ietf.org/doc/rfc4737/ -> Packet Reordering Metrics
http://datatracker.ietf.org/doc/rfc5560/ -> A One-Way Packet Duplication Metric

I would be interested to understand the change, and it might help with my next question.
When reading this draft, I was wondering:
1. if this metric is only for packets? I'm pretty sure it's the case, specifically, when I see the section 3.4 that speaks about packet loss.
So should the title be "Round Trip Packet Loss Metrics", or even, to be fully in line with RFC2680, ""Round Trip Packet Loss Metrics for IPPM"?
2. if the metric was not only for packet, but for application data (the abstract mentions "Many user applications"), then what would be the link with PMOL, RFC6390? Note: I believe that TWAMP doesn't deal with application data, but could be easily extended. A solution such as the Cisco IP SLA (http://tools.ietf.org/html/draft-cisco-sla-protocol-00) could do it.



2.
Section 4.3
      o  the Dst sent a Type-P packet back to the Src as immediately as possible, and

Why is this even useful to mention "as immediately as possible"?
I mean: if you have to use round-trip packet loss (instead of one-way packet loss), it's because you're not able to install a "responder" application on the target device. Therefore, you have no control at all on that target device. And you are forced to use a protocol such as ICMP. So, why is this even useful to say "as immediately as possible" if you have no control on that target device?
The sentence "the Dst sent a Type-P packet back to the Src as immediately as possible" only makes sense in the case of one-way delay metric.

I have the same issue with your new proposed text (discussed with Adrian)
  o  the Dst sent a Type-P packet back to the Src as quickly as
      possible (certainly less than Tmax, and fast enough for the
      intended purpose), and

I have the same issue with your new proposed text in section 4.4 (discussed with Adrian, AFAIK)
  We add the following guidance regarding the responder process to
  "send a Type-P packet back to the Src as quickly as possible".

  A response that was not generated within Tmax is inadequate for any
  realistic test, and the Src will discard such responses.  A responder
  that serves typical round-trip loss testing (which is relevant to
  higher-layer application performance) SHOULD produce a response in 1
  second or less.  A responder that is unable to satisfy this
  requirement SHOULD log the fact so that an operator can adjust the
  load and priorities as necessary.  Analysis of responder time-stamps
  [RFC5357] that finds responses are not generated in a timely fashion
  SHOULD result in operator notification, and the operator SHOULD
  suspend tests to the responder since it may be overloaded.
  Additional measurement considerations are described in Section 8,
  below.

For example, "A responder that is unable to satisfy this requirement SHOULD log the fact so that an operator can adjust the load and priorities as necessary. "
I've been doing IP SLA measurements for years with Cisco boxes, and I would only use round trip delay and loss metrics when I can't touch the target device. And here you're asking the target device to do a task for you in case of round trip loss...
Note: the default configuration for SLA measurement is to put a responder on the target device, and to measure in both directions the one way delay, the loss, and jitter.

Or maybe, the metric in this draft can only be used with the TWAMP protocol, which I believe requires some configuration on the target device?
However, it appears it's not a requirement as TWAMP is mentioned as one example in

    8.  Measurement Considerations and Calibration

  Prior to conducting this measurement, the participating hosts MUST be
  configured to send and receive test packets of the chosen Type-P.
  Standard measurement protocols are capable of this task [RFC5357],
  but any reliable method is sufficient.

Next question: why do mention "but any reliable method is sufficient.". It means that that metric can't be used with ICMP?

Anyway, it needs some clarifications.

3.
In section 4.3

  Following the precedent of[RFC2681], we make the simplifying
  assertion:

  Type-P-Round-trip-Loss(Src->Dst) = Type-P-Round-trip-Loss(Dst->Src)

While I could agree that Type-P-Round-trip(Src->Dst) = Type-P-Round-trip(Dst->Src), at some conditions, I disagree with the assertion that if you loose 50% packets, you can conclude that you lost 25% in each direction.

[Ballot comment]
- section 1, introduction
s/round-trip loss/ round-trip packet loss

Note: I'm sure there are multiple instances of this one.

- section 1, introduction
  Also, the
  specifications of the One-way Loss metric [RFC2680] and the Round-
  trip Delay metric [RFC2681] are frequently referenced and modified to
  match the round-trip circumstances addressed here

s/One-way Loss metric [RFC2680]/A One-way Packet Loss Metric for IPPM [RFC2680]
s/Round-trip Delay metric [RFC2681] /A Round-trip Delay Metric for IPPM [RFC2681]


- Section 3.3.  Metric Definition

  This section is specific to each metric.

What does this section add? Maybe it's part of a template?

- Section 4.1.  Name: Type-P-Round-trip-Loss
I double checked this name with RFC 2680
1. It should be Type-P-Round-trip-Packet-Loss throughout the document
2. I opened this errata on RFC 2680 http://www.rfc-editor.org/errata_search.php?eid=3186

- Section 7
  As discussed above, packet reordering is always a possibility.  In
  addition to the severe delay variation that usually accompanies it,
  reordering on the Src->Dst path will cause a mis-alignment of
  sequence numbers applied at the reflector when compared to the sender
  numbers.  Measurement implementations SHOULD address this possible
  outcome.

"reflector" is a term specific to TWAMP, and not found in RFC2330

[Ballot discuss]

  In the last paragraph of Section 5, the document says: " ... (or other
  process, the details of which MUST be specified if used)."

  Specified how? Is an RFC required?  Is a standards-track RFC required?
  This document already mentions the lack of an IANA registry.  Will an
  IANA registry be needed to help locate these specifications.

[Ballot comment]

  Please consider the comments raised by the Gen-ART Review by
  Ben Campbell on 10-Apr-2012.  The review can be found here:
  http://www.ietf.org/mail-archive/web/gen-art/current/msg07340.html

[Ballot discuss]

This discuss is just to check that Sandy Murphy's secdir review
comments have been taken into account. I've seen a bunch
of discussion of Adrian's comments but am not clear if Sandy's
have gotten lost or not, or needed changes or not. So just
checking that.

[Ballot discuss]
The Routing Diretorate review by Dan Frost missed the IETF Last Call
period by two days, but I would like the following points from his
review to be considered as part of this Discuss. Other smaller issues
are recorded as Comments.

> General observation: It's not clear to me what the IPPM WG strategy
> is for security considerations sections in metric definition
> documents.  For example, the security section of this document more or
> less repeats the one in (for example) RFC 2681, which itself
> duplicates verbatim the one in RFC 2680, and the issues discussed are
> general ones with measurement protocols rather than specific ones with
> the metric that is the subject of the document.  There's probably a
> better way to organize this.


> 3. Section 9.3 mentions the use of cryptographic hashing "to
> discourage the kind of interference mentioned above"; while this
> would mitigate the second form of interference, it wouldn't help
> with the first.

I would add that "discourage" might not be an appropriate word.

[Ballot comment]
Other comments coming from Dan Frost's review

> 1. Although it's probably obvious to most readers, it would be helpful
> to provide a brief informal definition of "round-trip loss" early in
> the introduction.  A mention of the venerable "ping" procedure would
> also not be amiss.

> 2. Most of the text seems to assume an "active" or test-based
> measurement approach, but Section 9.2 refers to passive measurement.
> It would be helpful to discuss the applicability of the latter
> approach.

> Nits:
>
> 1. The phrase "as immediately as possible" that appears a couple of
> times in the text (and that seems to originate in RFC 5357) is a bit
> unfortunate.  "Immediately" or "as quickly as possible" are better.
>
> 2. Section 5.4, second paragraph: s/affects/effects/
>
> 3. Section 8, second paragraph: s/Two key features ... is described/
>  Two key features ... are described/
>
> 4. Section 9.3, first paragraph:
> OLD
>  it is possible to change the processing of the packets (e.g.
>  increasing or decreasing delay) that may distort the measured
>  performance.
> NEW
>  it is possible to change the processing of the packets (e.g.
>  increasing or decreasing delay) in a way that may distort the
>  measured performance.
> END

State changed to In Last Call from Last Call Requested

The following Last Call Announcement was sent out:

From: The IESG

To: IETF-Announce

CC:

Reply-To: ietf@ietf.org

Subject: Last Call:  (Round-trip Loss Metrics) to Proposed Standard





The IESG has received a request from the IP Performance Metrics WG (ippm)

to consider the following document:

- 'Round-trip Loss Metrics'

  as a Proposed Standard



The IESG plans to make a decision in the next few weeks, and solicits

final comments on this action. Please send substantive comments to the

ietf@ietf.org mailing lists by 2012-03-19. Exceptionally, comments may be

sent to iesg@ietf.org instead. In either case, please retain the

beginning of the Subject line to allow automated sorting.



Abstract





  Many user applications (and the transport protocols that make them

  possible) require two-way communications.  To assess this capability,

  and to achieve test system simplicity, round-trip loss measurements

  are frequently conducted in practice.  The Two-Way Active Measurement

  Protocol specified in RFC 5357 establishes a round-trip loss

  measurement capability for the Internet.  However, there is currently

  no metric specified according to the RFC 2330 framework.



  This memo adds round-trip loss to the set of IP Performance Metrics

  (IPPM).



There is a normative down-ref to RFC 2330, which has been used in this

way by the IPPM working group before, and is present in the IESG's

down-ref registry.



The file can be obtained via

http://datatracker.ietf.org/doc/draft-ietf-ippm-rt-loss/



IESG discussion can be tracked via

http://datatracker.ietf.org/doc/draft-ietf-ippm-rt-loss/ballot/





No IPR declarations have been submitted directly on this I-D.

    Round-trip Loss Metrics
    draft-ietf-ippm-rt-loss-02

  (1.a) Who is the Document Shepherd for this document? Has the
        Document Shepherd personally reviewed this version of the
        document and, in particular, does he or she believe this
        version is ready for forwarding to the IESG for publication?

Document sheperd is Henk Uijterwaal.

  (1.b) Has the document had adequate review both from key WG members
        and from key non-WG members?

Yes, the document was reviewed by some members of the working group.

  (1.c) Does the Document Shepherd have concerns that the document
        needs more review from a particular or broader perspective,

No.

  (1.d) Does the Document Shepherd have any specific concerns or
        issues with this document that the Responsible Area Director
        and/or the IESG should be aware of?

No.

  (1.e) How solid is the WG consensus behind this document? Does it
        represent the strong concurrence of a few individuals, with
        others being silent, or does the WG as a whole understand and
        agree with it?

Good, this is an extension of well known earlier work (RFC 2681)
aimed to fill in a small gap in the set of metrics.  It builds heavily
on earlier work.

  (1.f) Has anyone threatened an appeal or otherwise indicated extreme
        discontent?

No.

  (1.g) Has the Document Shepherd personally verified that the
        document satisfies all ID nits?

The usual downref to the framework document exists here as well. 

Section 3.3 is essentially empty, but it is there for consistency with
other documents.

  (1.h) Has the document split its references into normative and
        informative?

Yes
        Are there normative references to documents that
        are not ready for advancement or are otherwise in an unclear
        state?

No

  (1.i) Has the Document Shepherd verified that the document IANA
        consideration section exists and is consistent with the body
        of the document?

Yes, it is essentially empty, so please remove it upon publication.

  (1.j) Has the Document Shepherd verified that sections of the
        document that are written in a formal language, such as XML
        code, BNF rules, MIB definitions, etc., validate correctly in
        an automated checker?

N/A

  (1.k) The IESG approval announcement includes a Document
        Announcement Write-Up. Please provide such a Document
        Announcement Write-Up? Recent examples can be found in the
        "Action" announcements for approved documents. The approval
        announcement contains the following sections:

    Technical Summary

  Many user applications (and the transport protocols that make them
  possible) require two-way communications.  To assess this capability,
  and to achieve test system simplicity, round-trip loss measurements
  are frequently conducted in practice.  The Two-Way Active Measurement
  Protocol specified in RFC 5357 establishes a round-trip loss
  measurement capability for the Internet.  However, there is currently
  no metric specified according to the RFC 2330 framework.

  This memo adds round-trip loss to the set of IP Performance Metrics
  (IPPM).


    Working Group Summary
The normal WG process was followed and the document has been discussed for
several years.  The document as it is now, reflects WG consensus, with nothing
special worth noticing.

    Document Quality
Good