Considerations for Selecting RTP Control Protocol (RTCP) Extended Report (XR) Metrics for the WebRTC Statistics API
draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-10

[Ballot comment]
Thanks to the authors and contributors on this document. I have two substantive
comments, and a handful of minor nits.

---------------------------------------------------------------------------

§5:

>  application can use these metrics to calculate the Mean Opinion Score
>  (MoS) values or Media Delivery Index (MDI) for their services.

"...calculate Estimated Mean Opinion Score (eMOS) values..."

(MOS scores necessarily require human subjective evaluation of audio under
carefully controlled circumstances -- any computer-generated metrics are
inherently estimations, and should not be called MOS scores.)

---------------------------------------------------------------------------

§5.3.1:

>  Error-resilience mechanisms, like RTP retransmission or FEC, are
>  optional in RTCWEB because the overhead of the repair bits adding to
>  the original streams.

This is a little misleading. draft-ietf-rtcweb-rtp-usage says:

  WebRTC endpoints MUST follow the recommendations for FEC use given in
  [I-D.ietf-rtcweb-fec].

And, in turn, draft-ietf-rtcweb-fec says:

  To support the functionality recommended above, implementations MUST
  be able to receive and make use of the relevant FEC formats for their
  supported audio codecs, and MUST indicate this support, as described
  in Section 4.  Use of these formats when sending, as mentioned above,
  is RECOMMENDED.

Rather than trying to reiterate this somewhat nuanced situation in *this*
document, I recommend removing the entire sentence and fixing the rest of the
paragraph. If you choose to keep it, please make sure it more clearly explains
the level of FEC support required of RTCWEB implementations.


===========================================================================

The remainder of my comments are grammatical or editorial nits that should be
fixed before progressing the document.

---------------------------------------------------------------------------

§3:

>  The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
>  exposes the basic metrics...

"expose"

>  However, these metrics provides...

"provide"


>  For example, it may be useful to distinguish between
>  packets lost and packets discarded due to late arrival, even though
>  they have the same impact on the multimedia quality, it helps in
>  identifying and diagnosing issues.

This is a run-on sentence. Suggest: "...due to late arrival. Even though..."

>  The WebRTC application extracts the statistic from the browser by
>  querying the getStats() API [W3C.WD-webrtc-20161124], but the browser
>  currently only reports the local variables i.e., the statistics
>  related to the outgoing RTP media streams and the incoming RTP media
>  streams.

I don't think the "currently" in this sentence is actually true any longer; and
I expect it to get increasingly less true as time goes on. Consider rephrasing.

>  [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of
>  any RTCP XRs and since their usage is optional.

This is not a sentence. Maybe remove "since"?

---------------------------------------------------------------------------

§4:

>  For example, an application may choose
>  to poll the stack for statistics every 1 second, in this case the
>  underlying stack local will return the current snapshot of the local
>  statistics (for incoming and outgoing media streams).

This is a comma splice. Consider: "...every second. In this case..."

>  However it may
>  return the same remote statistics as before for the remote
>  statistics

Add a comma after "However".

---------------------------------------------------------------------------

§5:

>  Since following metrics are all defined in RTCP XR which is not
>  mandated in WebRTC, all of them are local.

"Since the following..."
      ^^^

"...RTCP XR, which is..."
          ^

>  However, if RTCP XR is
>  supported by negotiation between two browsers, following metrics can
>  also be generated remotely and be sent to local by RTCP XR packets.

"...two browsers, the following..."
                  ^^^

>  Following metrics are classified into 3 categories: network impact

"The following metrics..."
^^^


>  viewpoint of application, e.g., bit rate, frames rate or jitter

"...frame rate..."

>  WebRTC
>  application can use these metrics to calculate the Mean Opinion Score

"applications"


---------------------------------------------------------------------------

§5.1.1:

>  lost and duplicated packets.  Lost packets counts are useful for

"Lost packet counts..."

---------------------------------------------------------------------------

§5.1.3:

>  the two run length vectors to identify congestion-related losses,
>  i.e., a router queue became overloaded causing delays and then

Replace "i.e." ("that is") with "e.g." ("for example").

---------------------------------------------------------------------------

§5.2.1:

>  The metric reports the cumulative size of the packets discarded in
>  the interval, it is complementary to number of discarded packets.

"...the interval. It is complementary..."

---------------------------------------------------------------------------

§5.2.2:

>  For
>  audio streams, a single RTP packet may contain one or multiple audio
>  frames, each of which has a fixed length.

I don't think this is generally true. Even in the case of WebRTC, I believe the
default mode of operation for Opus is VBR, which results in frame sizes that
vary from frame to frame.

>  The metrics in this category includes: number of discarded key

"...category include:..."

---------------------------------------------------------------------------


§5.3.1:

>  The un-repaired packets count and repaired loss count defined in

"...unrepaired packet count..."

>  packets to lost packets.  Including this kind of metrics helps the

Choose either "...these kinds of metrics..." or "...this kind of metric..."

---------------------------------------------------------------------------

§5.3.2:

>  related losses, i.e., a router queue became overloaded causing delays

Replace "i.e." with "e.g."

---------------------------------------------------------------------------

§6:

>  identifiers relevant to RTP media in WebRTC.  These group of
>  identifiers are defined on a ReportGroup corresponding to an
>  Synchronization source (SSRC).  In practice the application need to

"This group of identifiers..."
^^^^

"...corresponding to a synchronization source..."
                    ^

"...the application needs to..."
                    ^^^^^

>  For XR metrics, it depends on
>  two factors: 1) if it measured at the endpoint, 2) if it reported by
>  the endpoint in an XR report.

"...if it is measured..."
          ^^

"...if it is reported..."
          ^^

---------------------------------------------------------------------------

§11.2:

>  [W3C.WD-webrtc-20161124]
>            Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
>            Communication Between Browsers", World Wide Web Consortium
>            WD WD-webrtc-20161124, November 2016,
>            .

The authorship of this document is incorrect (neither Sporny nor Longley are
authors on WebRTC).

The URL points to an outdated version of the specification. Please update to
https://www.w3.org/TR/2017/CR-webrtc-20171102/


>  [W3C.WD-webrtc-stats-20161214]
>            Alvestrand, H. and V. Singh, "Identifiers for
>            WebRTC's Statistics API", World Wide Web Consortium
>            WD WD-webrtc-stats-20161214, December 2016,
>            .

Please replace "'" with an apostrophe.

Please update the URL to https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/

[Ballot comment]
Thanks to the authors and contributors on this document. I have two substantive
comments, and a handful of minor nits.

---------------------------------------------------------------------------

§5.3.1:

>  application can use these metrics to calculate the Mean Opinion Score
>  (MoS) values or Media Delivery Index (MDI) for their services.

"...calculate Estimated Mean Opinion Score (eMOS) values..."

(MOS scores necessarily require human subjective evaluation of audio under
carefully controlled circumstances -- any computer-generated metrics are
inherently estimations, and should not be called MOS scores.)

---------------------------------------------------------------------------

§5.3.1:

>  Error-resilience mechanisms, like RTP retransmission or FEC, are
>  optional in RTCWEB because the overhead of the repair bits adding to
>  the original streams.

This is a little misleading. draft-ietf-rtcweb-rtp-usage says:

  WebRTC endpoints MUST follow the recommendations for FEC use given in
  [I-D.ietf-rtcweb-fec].

And, in turn, draft-ietf-rtcweb-fec says:

  To support the functionality recommended above, implementations MUST
  be able to receive and make use of the relevant FEC formats for their
  supported audio codecs, and MUST indicate this support, as described
  in Section 4.  Use of these formats when sending, as mentioned above,
  is RECOMMENDED.

Rather than trying to reiterate this somewhat nuanced situation in *this*
document, I recommend removing the entire sentence and fixing the rest of the
paragraph. If you choose to keep it, please make sure it more clearly explains
the level of FEC support required of RTCWEB implementations.


===========================================================================

The remainder of my comments are grammatical or editorial nits that should be
fixed before progressing the document.

---------------------------------------------------------------------------

§3:

>  The RTCP Sender Reports (SRs) and Receiver Reports (RRs) [RFC3550]
>  exposes the basic metrics...

"expose"

>  However, these metrics provides...

"provide"


>  For example, it may be useful to distinguish between
>  packets lost and packets discarded due to late arrival, even though
>  they have the same impact on the multimedia quality, it helps in
>  identifying and diagnosing issues.

This is a run-on sentence. Suggest: "...due to late arrival. Even though..."

>  The WebRTC application extracts the statistic from the browser by
>  querying the getStats() API [W3C.WD-webrtc-20161124], but the browser
>  currently only reports the local variables i.e., the statistics
>  related to the outgoing RTP media streams and the incoming RTP media
>  streams.

I don't think the "currently" in this sentence is actually true any longer; and
I expect it to get increasingly less true as time goes on. Consider rephrasing.

>  [I-D.ietf-rtcweb-rtp-usage] does not mandate the use of
>  any RTCP XRs and since their usage is optional.

This is not a sentence. Maybe remove "since"?

---------------------------------------------------------------------------

§4:

>  For example, an application may choose
>  to poll the stack for statistics every 1 second, in this case the
>  underlying stack local will return the current snapshot of the local
>  statistics (for incoming and outgoing media streams).

This is a comma splice. Consider: "...every second. In this case..."

>  However it may
>  return the same remote statistics as before for the remote
>  statistics

Add a comma after "However".

---------------------------------------------------------------------------

§5:

>  Since following metrics are all defined in RTCP XR which is not
>  mandated in WebRTC, all of them are local.

"Since the following..."
      ^^^

"...RTCP XR, which is..."
          ^

>  However, if RTCP XR is
>  supported by negotiation between two browsers, following metrics can
>  also be generated remotely and be sent to local by RTCP XR packets.

"...two browsers, the following..."
                  ^^^

>  Following metrics are classified into 3 categories: network impact

"The following metrics..."
^^^


>  viewpoint of application, e.g., bit rate, frames rate or jitter

"...frame rate..."

>  WebRTC
>  application can use these metrics to calculate the Mean Opinion Score

"applications"


---------------------------------------------------------------------------

§5.1.1:

>  lost and duplicated packets.  Lost packets counts are useful for

"Lost packet counts..."

---------------------------------------------------------------------------

§5.1.3:

>  the two run length vectors to identify congestion-related losses,
>  i.e., a router queue became overloaded causing delays and then

Replace "i.e." ("that is") with "e.g." ("for example").

---------------------------------------------------------------------------

§5.2.1:

>  The metric reports the cumulative size of the packets discarded in
>  the interval, it is complementary to number of discarded packets.

"...the interval. It is complementary..."

---------------------------------------------------------------------------

§5.2.2:

>  For
>  audio streams, a single RTP packet may contain one or multiple audio
>  frames, each of which has a fixed length.

I don't think this is generally true. Even in the case of WebRTC, I believe the
default mode of operation for Opus is VBR, which results in frame sizes that
vary from frame to frame.

>  The metrics in this category includes: number of discarded key

"...category include:..."

---------------------------------------------------------------------------


§5.3.1:

>  The un-repaired packets count and repaired loss count defined in

"...unrepaired packet count..."

>  packets to lost packets.  Including this kind of metrics helps the

Choose either "...these kinds of metrics..." or "...this kind of metric..."

---------------------------------------------------------------------------

§5.3.2:

>  related losses, i.e., a router queue became overloaded causing delays

Replace "i.e." with "e.g."

---------------------------------------------------------------------------

§6:

>  identifiers relevant to RTP media in WebRTC.  These group of
>  identifiers are defined on a ReportGroup corresponding to an
>  Synchronization source (SSRC).  In practice the application need to

"This group of identifiers..."
^^^^

"...corresponding to a synchronization source..."
                    ^

"...the application needs to..."
                    ^^^^^

>  For XR metrics, it depends on
>  two factors: 1) if it measured at the endpoint, 2) if it reported by
>  the endpoint in an XR report.

"...if it is measured..."
          ^^

"...if it is reported..."
          ^^

---------------------------------------------------------------------------

§11.2:

>  [W3C.WD-webrtc-20161124]
>            Sporny, M. and D. Longley, "WebRTC 1.0: Real-time
>            Communication Between Browsers", World Wide Web Consortium
>            WD WD-webrtc-20161124, November 2016,
>            .

The authorship of this document is incorrect (neither Sporny nor Longley are
authors on WebRTC).

The URL points to an outdated version of the specification. Please update to
https://www.w3.org/TR/2017/CR-webrtc-20171102/


>  [W3C.WD-webrtc-stats-20161214]
>            Alvestrand, H. and V. Singh, "Identifiers for
>            WebRTC's Statistics API", World Wide Web Consortium
>            WD WD-webrtc-stats-20161214, December 2016,
>            .

Please replace "'" with an apostrophe.

Please update the URL to https://www.w3.org/TR/2018/WD-webrtc-stats-20180519/

[Ballot comment]
Thanks for producing this document.

I did have one comment (and I apologize in advance that this is a particular hot button for me).

I understand that you can do things with these metrics that you can't do with previous metrics, but I wish there was a clearer description of "if you want to understand X, you should implement this metric" earlier in the document. The impression the reader is left with, is "if you're doing RTCWeb, you should implement this metric", but I'm guessing that at least some of these metrics would be useful for people who weren't doing RTCWeb, but need to measure something that people who are doing RTCWeb also need to measure, and if that's true, a short section that helped them figure that out would be helpful.

[Ballot discuss]
For the AD: Please note that the intended status in the datatracker is "Proposed Standard" while the doc itself says "Informational". After reading the doc, I would find informational correct. However, please clarify what the intended status is supposed to be!

[Ballot comment]
Thanks for the well-written document. A few comments mostly on references:

- Maybe also provide an (informative) reference to draft-ietf-rtcweb-overview-19 on the first occurrence of WebRTC...?

- Is ReportGroup defined in another RFC? If so, please provide reference in ther terminology section.

- Maybe provide (informative) references for Mean Opinion Score (MoS) and Media Delivery Index (MDI), as well as "Media Loss Rate (MLR) of MDI" maybe.

- It seems to me that at least [W3C.WD-webrtc-stats-20161214] should be a normative reference.

- I wonder if it would make sense to refer to some of the IPPM RFCs that define these or similar metrics?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-08, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2018-02-23):

From: The IESG
To: IETF-Announce
CC: ben@nostrum.com, shida.at.ietf@gmail.com, xrblock-chairs@ietf.org, draft-ietf-xrblock-rtcweb-rtcp-xr-metrics@ietf.org, Shida Schubert , xrblock@ietf.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Considerations for Selecting RTCP Extended Report (XR) Metrics for the WebRTC Statistics API) to Proposed Standard


The IESG has received a request from the Metric Blocks for use with RTCP's
Extended Report Framework WG (xrblock) to consider the following document: -
'Considerations for Selecting RTCP Extended Report (XR) Metrics for the
  WebRTC Statistics API'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2018-02-23. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document describes monitoring features related to media streams
  in Web real-time communication (WebRTC).  It provides a list of RTCP
  Sender Report, Receiver Report and Extended Report metrics, which may
  need to be supported by RTP implementations in some diverse
  environments.  It lists a set of identifiers for the WebRTC's
  statistics API.  These identifiers are a set of RTCP SR, RR, and XR
  metrics related to the transport of multimedia flows.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcweb-rtcp-xr-metrics/ballot/


No IPR declarations have been submitted directly on this I-D.


The document contains these normative downward references.
See RFC 3967 for additional information:
    rfc3792: Survey of IPv4 Addresses in Currently Deployed IETF Security Area Standards Track and Experimental Documents (Informational - IETF stream)

This is my AD Evaluation of draft-ietf-xrblock-rtcweb-rtcp-xr-metrics-06. I would like to resolve at least the substantive comments prior to IETF LC.
—————————
Substantive:

- General: If I understand correctly, this document lists and describes XR metrics that a WebRTC application might choose to negotiate and expose via the WebRTC API. It does not define any new XR metrics. That brings up two questions:

1) Why is this specific to WebRTC? For the most parts, all the arguments here would apply to any sort of RTP endpoint.

2) Why is this standards track? This sort of material is typically informational, or occasionally a BCP. There are a small number of 2119 keywords, but I’m not sure they are needed or appropriate. (Specifics below.) My initial instinct is that this should be informational.

- General: This needs an IANA considerations section, even if it just contains the statement to the effect of “This document makes no requests for IANA”.

- 1, last paragraph: “ The document also creates a registry containing identifiers from the metrics reported in the RTCP Sender, Receiver, and Extended Reports.”

It doesn’t actually do that. I understand from the shepherd’s report that this was intentionally removed.

— “All identifiers proposed in this document are RECOMMENDED to be implemented by an endpoint.  An endpoint MAY choose not to expose an identifier if it does not implement the corresponding RTCP Report. “

Does the “RECOMMENDED” apply to all endpoints or just WebRTC endpoints? If the latter, doesn’t that requirement belong (or need to update) some requirement from an RTCWEB (maybe rtp-usage) document, or even the API itself? (I suspect that this draft does not have standing to state this normatively.) What is meant by “MAY choose not to expose”. Is that talking about via the WebRTC API? If so, isn’t that up to the API definition? (That is, it shouldn’t be normative here.)

-2: If this document keeps the normative keywords, please use the updated boilerplate from RFC 8174. (I note that there are at least some uncapitalized “may”s that do not seem normative.)

- 5.2.2, last paragraph: “The following metrics can also be considered…”
Be considered by whom? Implementors? API designers?

- 6, first paragraph: “In practice the application MUST be able to query the statistic identifiers on both an incoming (remote) and outgoing (local) media stream.”
What does the MUST requirement apply to? The WebRTC API? This seems more like a statement of fact that “ the application needs to be able…”

-8: "Therefore encryption procedures, such as the ones suggested for a Secure RTCP (SRTCP), need to be used.”
The text should describe the reasons encryption is needed. Also, is this a new normative requirement, or a restatement of an existing requirement?

Editorial:

-1, paragraph 1:" If sufficient information (metrics or statistics) are provided to the applications, it can attempt to improve the media quality. “

s / “are provided” / “is provided”
s / applications / application

-3, 2nd paragraph: It would help to clarify that the references to section 5 and 6 are to this document, not RFC 3611.

— 3rd paragraph: "At the moment…”
Please clarify that is at the time of writing (not reading)

-5, 2nd paragraph: “ Application impact metrics mainly collect the information in the viewpoint of application … “
s / in / from
Also, please expand “FEC” on first mention.

- 5.1.1, first paragraph: “ Duplicate packets may be a result of network delays, which causes the sender to retransmit the original packets.”
s/ :delays, which” / “delays that”

— last paragraph: "For those RTCWEB services with jitter buffer…”
s/buffer/buffers

- 5.1.2, first paragraph:
First sentence is a comma splice.
— “some transitory nature of the impairments”
Should this say “the transitory nature of some impairments”?
— “ Distributed burst provides a higher subjective quality”
“Burst distribution …”?
— Last paragraph: “ Hence, if WebRTC application”
Missing article (or should application be plural?)

- 5.2.1, title: Should “Discard” be “Discarded”?

-7: Please clarify that this is at the time of this writing.

1. Summary

The document shepherd is Shida Schubert. The responsible Area Director is Ben Campbell.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

- Proposed Standard
- The title page header indicates that it is a standard-track.
- The document defines a process/mechanism for standard defined in XRBLOCK WG to be referenced/used by WebRTC WG in W3C.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This document describes monitoring features related to media streams in Web real-time communication (WebRTC). It provides a list of RTCP Sender Report, Receiver Report and Extended Report metrics, which may need to be supported by RTP implementations in some diverse environments.
For each of the metrics proposed, the document provides rational along with consideration when using them.

Working Group Summary:
A lot of effort went into how to ensure proposed metrics are referenced and used by WebRTC WG in W3C. There were no objection or questions about the usefulness of the metrics proposed but more so about ensuring
implementors are aware and take full advantage of the metrics.

Initially a creating a registry was agreed to be the right way forward but after much debates, in order for better exposure to the implementing community, it was agreed that metrics defined in IETF are to be added to the
specification within W3C and for this document to define initial proposed metrics along with how to add/expose future metrics useful to the WebRTC WG and surrounding community.

The defined metrics here are already part of the W3C documents and the metrics proposed to be used by WebRTC are implemented and used in the wild.

Personnel:
The document shepherd is Shida Schubert.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd has read the draft and is confident that this document is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

We had contributors from OPS area and W3C folks beside the usual reviewers in XRBLOCK contributing to development of this document.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

Irrelevant as no IPR disclosures have been filed.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The WG as a whole understand and agree with the current standing of specification.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summaries the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

There was nothing except for the format of the document that raised extensive discussions. Current form of this document is the format that the Work group agreed will provide the most value to the impelemtors.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/  and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

** The document seems to lack an IANA Considerations section. (See Section
2.2 of https://www.ietf.org/id-info/checklist  for how to handle the case
when there are no actions for IANA.)

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

N/A

(13) Have all references within this document been identified as either normative or informative?

YES

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No problem here.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

N/A

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

N/A

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

No nits error