RTP Control Protocol (RTCP) Extended Report (XR) Block for Packet Delay Variation Metric Reporting
draft-ietf-xrblock-rtcp-xr-pdv-08

[Ballot comment]
I had a discussion with Dan Romascanu, and we settled on:
- RFC 6390 template is required for new perf metric definition
- RFC 6390 template is a nice-to-have when we refer to an existing perf metric
Nice-to-have because the performance metric reference doesn't always include all the required information about: measurement points, measurement timing, use and applications, reporting model, etc... but focus only on the "Method of Measurement or Calculation"

So basically, at this point, I will clear my DISCUSS-DISCUSS

Somehow, I've been trying to solve a growing problem, and potentially bigger problem:

Where does the list of performance metric definitions come from at
the IETF? We have multiple sources:
- IPPM for IP performance metrics.
- RTCP for RTP performance metrics:
- PMOL: Performance Metrics at Other Layers, with
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example
  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03.
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different parts
of the IETF, without consistency.
And even finding the performance metrics specified in the IETF is not an easy task.

I've convinced that the community will have solve this problem, and I will propose a meeting during the next IETF to try to come up with a solution. This meeting should include the PMOL directorate, the XRBLOCK chairs, and the IPPM chairs.
If someone not in the mentioned list wants to participate, let me know privately.

Regards, Benoit.

[Ballot discuss]
  The Performance Metrics Framework [RFC6390] provides guidance on the
  definition and specification of performance metrics.  The RTP
  Monitoring Architectures [MONARCH] provides guideline for reporting
  block format using RTCP XR.  The XR Block described in this document
  are in accordance with the guidelines in [RFC6390] and [MONARCH].

"in accordance with RFC6390": Are you defining the metric according http://tools.ietf.org/html/rfc6390#section-5.4.4?
I don't think that's correct!
I see that you referenced [G.1020] and [Y.1540], and that's fine.
However, there are some other aspects of http://tools.ietf.org/html/rfc6390#section-5.4.4, specific to RTP, that should be filled in. For example: Measurement Point(s) with Potential Measurement Domain, Measurement Timing, etc..

My entire point is more a DISCUSS-DISCUSS, for both draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
Sorry to pick on these two drafts, but we need to have an IESG performance metrics discussion.
Where does the list of performance metric definitions come from at the IETF?
We have multiple sources:
- IPPM for IP performance metrics
- RTCP for RTP performance metrics:
  Definitions in the document themselves or potentially referencing some other SDOs
  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
  bits 014-011
            0: MAPDV2, Clause 6.2.3.2 of [G.1020],
            1: 2-point PDV, Clause 6.2.4 of [Y.1540].
- PMOL: Performance Metrics at Other Layers, with
  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
- IPFIX will one day or the other exports performance metrics.
  I see for example http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
  It's again a redefinition, and it should not be!

My concerns are that we start to define performance metrics in different parts of the IETF, without consistency.

We have defined RFC 6390 on "Guidelines for Considering New Performance Metric Development", which ask for specific definition
See http://tools.ietf.org/html/rfc6390#section-5.4.4

I believe that the IETF should at least:
- define the performance metrics in a consistent way according to RFC6390.
- document those performance metrics in a single location

So my questions are:
- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?

[Ballot discuss]
In section 2, you should cite RFC 5234.

The ABNF in section 4 is a mess. First, the indents are different; that's legal, but hard to read. As far as specifics:

  rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF

Why is this not:

  rtcp-xr-attrib = "a=rtcp-xr:" [xr-format *(SP xr-format)] CRLF

?

  (defined in [RFC3611])

This line needs a ";" in front of it if it's a comment. And what is defined in 3611? I can't tell.

        pdvtype    = "pdv="    "0"      ; MAPDV2 ITU-T G.1020
                              / "1"      ; 2-point PDV ITU-T Y.1540
                            / 1*2DIGIT    ;Value 2~15 are valid and
                                          ;reserved for future use

That just has to be wrong. This matches:

pdv=0
1
5
14

It does not match:

pdv=1
pdv=5
pdv=14

Without the parens, concatenation takes precedence over alternation. I think you want this:

        pdvtype    = "pdv="    ("0"    ; MAPDV2 ITU-T G.1020
                              / "1"      ; 2-point PDV ITU-T Y.1540
                            / 1*2DIGIT) ;Value 2~15 are valid and
                                        ;reserved for future use

That gets you the right result.

        nspec      = "nthr=" fixpoint    ; negative PDV threshold (ms)
                    / "npc=" fixpoint    ; negative PDV percentile
        pspec      = "pthr=" fixpoint    ; positive PDV threshold (ms)
                    / "ppc=" fixpoint    ; positive PDV percentile

These are fine, but would be clearer with parens.

        DIGIT          = %x30-39
       
This can simply be imported from RFC 5234; no need to redefine it here.

[Ballot comment]
Just one non-blocking comment on the abstract:
The abstract should be readable on its own by someone who doesn't know what RFC XYZ is about and sees only the title and abstract.  Someone who doesn't know what RTP and RTCP are won't get any clue from the abstract.  Will you consider this, so people will immediately see what the document refers to?:

OLD
  This document defines an RTCP XR Report Block that allows the
  reporting of Packet Delay Variation metrics for a range of RTP
  applications.
NEW
  This document defines a Real-Time Control Protocol (RTCP)
  Extended Report (XR) block that allows the reporting of Packet
  Delay Variation metrics for a range of Real-time Transport
  Protocol (RTP) applications.

---
Also, I will note that the shepherd writeup is incorrect:

> (18) List any new IANA registries that require Expert Review for future
> allocations. Provide any public guidance that the IESG would find
> useful in selecting the IANA Experts for these new registries.
>
> N/A

Section 5.4 creates a new registry with Specification Required policy, which requires the appointment of a designated expert.  I presume the ADs have or will get the input they need to appoint one, but I wanted to point out the need, since the shepherd writeup didn't.

[Ballot comment]
Documents like this would benefit from a worked example showing a set of RTP packets as seen by the element doing the metric calculation, demonstrating where the inputs to the calculation come from. Please consider such a section for future documents of this type.

Should the Y.1540 reference be updated to the currently in force version (Mar2011)?

[Ballot comment]
Minor observation: something is missing in this text from the
"Document Quality" ballot text:

  and other new may show up.

Perhaps "interoperability issues" or "conformance issues"?

[Ballot discuss]
I have a simple editorial Discuss that is important enough to require
that it is fixed before publiscation.

---

Section 3

  Block Length: 16 bits

      The length of this report block in 32-bit words, minus one.  For
      the Packet Delay Variation Metrics block, the block length is
      equal to 4.

Nice! The figure shows...

        0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    BT=NPDV  | I |pdvtyp |Rsv|      block length=3          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

[Ballot comment]
I have a number of Comments that I hope you will look at before
publication.

---

I should have liked it if Section 1 had a small subsection giving some
form of requirement. I.e., although RFC6390 gives goodguidance, for this
document I would have liked to understand what it is that people intend
to do with this XR block.

Similarly, the applicability statement in 1.4is very open!

The concern (which I doubt is valid, so this is not a Discuss) is that
you are defining a protocol extension with no particular planned use,
but a general statement that someone might use it. We obviously do not
need to clutter our protocols with extensions that no-one actually uses.

---
                                                                             
Do you not think that there should be some mention of Y.1541 and G.1020
in the Introduction?

---

I think [MEASI] is used in a way that makes it a normative reference.

---

Section 3

  If the measurement interval is not received for this metric
  block, this metric block SHOULD be discarded.

Why is that a SHOULD not a MUST? Under what circumstances MAY the
metric block be retained? How would it be used?

---

Section 3.2

  Interval Metric flag (I): 2 bit

The value I=00 is not valid? What happens if it is received?

---

Section 3.2 calls the final bits "reserved" but the figure shows them as
"unused"

[Ballot comment]
1) s1.2: Is this a "MUST":

r/This draft defines a new Extended Report block that
  must be used in accordance with [RFC3550] and [RFC3611].
/This draft defines a new Extended Report block
  for use with [RFC3550] and [RFC3611].

2) s3.2, Packet Delay Variation Metric Type: Trying to figure out whether there some other way to interpret an enumerated ;)  Maybe just replace should be/is:

r/Packet Delay Variation Metric Type is of type enumerated and
      should be interpreted as Integer.
/Packet Delay Variation Metric Type is of type enumerated and
      is interpreted as Integer.

3) s3.2: Reserved: Curious why there's a SHOULD where almost every other time I can remember it's a MUST for setting reserved bits to zero:

In the absence of
such a definition, the bits in this field SHOULD be set to zero
and MUST be ignored by the receiver.

4) s3.2: Positive PDV Threshold/Peak (and others): Is the S11:4 format documented somewhere?

IANA has reviewed draft-ietf-xrblock-rtcp-xr-pdv-03 and has the following comments:

IANA understands that, upon approval of this document, there are three IANA
actions which must be completed.

First, in the RTP Control Protocol Extended Reports (RTCP XR) Block Type
Registry located at:

http://www.iana.org/assignments/rtcp-xr-block-types/rtcp-xr-block-types.xml

a new RTCP XR Block Type will be registered as follows:

BT: [ tbd ]
Name: Packet Delay Variation Metrics Block
Reference: [ RFC-to-be ]

Second, in the RTP Control Protocol Extended Reports (RTCP XR) Session
Description Protocol (SDP) Parameters Registry located at:

http://www.iana.org/assignments/rtcp-xr-sdp-parameters/rtcp-xr-sdp-parameters.xml

a new SDP parameter will be registered as follows:

Parameter: pkt-dly-var
Reference: [ RFC-to-be ]

Third, a new subregistry of the RTP Control Protocol Extended Reports
(RTCP XR) Block Type Registry located at:

http://www.iana.org/assignments/rtcp-xr-block-types/rtcp-xr-block-types.xml

will be created. The new registry will be called the "RTCP XR PDV block - PDV
type" subregistry. New values for the registry are created and maintained
through Specification Required as defined in RFC 5226. The values in the
registry range from 0 to 15 inclusive.

There are initial values for the registry as follows:

PDV Type Description Reference
------------ --------------------------------- -----------------
0 MAPDV2 ITU-T Rec. G.1020
1 2-point PDV ITU-T Rec. Y.1540
2-15 Available

IANA understands that these three actions are the only ones required
to be completed upon approval of this document.

Note: The actions requested in this document will not be completed
until the document has been approved for publication as an RFC.

UPDATED writeup from Dan Romascanu:

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard. Header says ' Intended status: Standards Track '

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This document defines an RTCP XR Report Block that allows the
  reporting of Packet Delay Variation metrics for a range of RTP
  applications.

Working Group Summary

The WG path of this document was reasonably short and efficient.
Many technical comments were made during the reviews and all were
resolved with consensus.

Document Quality

There are implementations conforming to Y.1541 and G.1020 but not
exact
the same as this I-D. It is expected that with the approval of this
document these implementation will be made conformant, and other new
may
show up.

Personnel

Dan Romascanu is the Document Shepherd. Gonzalo Camarillo
is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have performed a detailed review of the document and I consider it
ready.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

The document was already reviewed by Al Morton, but it would be useful
that another RFC 6390 review is performed, especially if significant
changes will happen as result of the AD review or IETF Last Call.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Yes.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures have been submitted directly on
draft-ietf-xrblock-rtcp-xr-pdv.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it?

The number of active participants in the Working Group is not very
high (around ten). Among the active participants there seems to be solid
consensus in support of this document.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

No issues excepting the ones related to the submission date and a
couple of references on documents that have issued more recent versions
since the publication of the I-D.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

Al Morton already reviewed a previous version of the document, but I
would recommend for another reviewer from the performance metrics
directorate to do an RFC 6390 review of the I-D.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

N/A

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

N/A

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

N/A

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

The document requires from IANA allocations of values in existing
registries which are clearly defined. It also requires establishing a
new registry for PDV types with clearly specified initial values and
'Specification Required' policy for adding new values.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

N/A

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

N/A

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (RTCP XR Report Block for Packet Delay Variation Metric Reporting) to Proposed Standard


The IESG has received a request from the Metric Blocks for use with
RTCP's Extended Report Framework WG (xrblock) to consider the following
document:
- 'RTCP XR Report Block for Packet Delay Variation Metric Reporting'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-07-23. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document defines an RTCP XR Report Block that allows the
  reporting of Packet Delay Variation metrics for a range of RTP
  applications.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard. Header says ' Intended status: Standards Track '

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This document defines an RTCP XR Report Block that allows the
  reporting of Packet Delay Variation metrics for a range of RTP
  applications.

Working Group Summary

The WG path of this document was reasonably short and efficient.
Many technical comments were made during the reviews and all were
resolved with consensus.

Document Quality

There are implementations conforming to Y.1541 and G.1020 but not
exact
the same as this I-D. It is expected that with the approval of this
document these implementation will be made conformant, and other new
may
show up.

Personnel

Dan Romascanu is the Document Shepherd. Gonzalo Camarillo
is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have performed a detailed review of the document and I consider it
ready.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed? 

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

The document was already reviewed by Al Morton, so I do not believe
what another RFC 6390 review is necessary unless significant changed
will happen as result of the AD review or IETF Last Call.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Yes.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures have been submitted directly on
draft-ietf-xrblock-rtcp-xr-pdv.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The number of active participants in the Working Group is not very
high (around ten). Among the active participants there seems to be solid
consensus in support of this document.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

No issues excepting the ones related to the submission date and a
couple of references on documents that have issued more recent versions
since the publication of the I-D.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

Al Morton already reviewed a previous version of the document, but I
would recommend for another reviewer from the performance metrics
directorate to do an RFC 6390 review of the I-D.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

N/A

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

N/A

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

N/A

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

The document requires from IANA allocations of values in existing
registries which are clearly defined. It also requires establishing a
new registry for PDV types with clearly specified initial values and
'Specification Required' policy for adding new values.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

N/A

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

N/A