Evaluating Congestion Control for Interactive Real-time Media
draft-ietf-rmcat-eval-criteria-14

Note: This ballot was opened for revision 11 and is now closed.

(Mirja Kühlewind) Yes

Deborah Brungard No Objection

Alissa Cooper No Objection

Comment (2020-03-03 for -12)
"Sample video test sequences are available at: [xiph-seq] and
   [HEVC-seq].  The following two video streams are the recommended
   minimum for testing: Foreman and FourPeople."

Are there minimum recommended ones at [xiph-seq]? Both of the ones listed appear to be at [HEVC-seq].

Roman Danyliw (was Discuss) No Objection

Comment (2020-03-18 for -13)
No email
send info
Thanks for addressing my DISCUSS point.

----

-- Section 4.4.  Consider adding a citation for the “Bilbert-Elliot” model

Benjamin Kaduk No Objection

Comment (2020-03-05 for -12)
Section 1

   media flow's throughput.  Furthermore, the proposed algorithms are
   expected to operate within the envelope of the circuit breakers
   defined in RFC8083 [RFC8083].

The "proposed algorithms" are the congestion-control schemes, not the
evaluation procedures, right?

Section 3.1

   If the congestion control implements, retransmissions or FEC, the

nit: no comma (first one)

Section 4.4

It's too bad that we don't have more specific guidance to give on loss
generation modeling.

Section 4.5.1

   path.  Due to this, if a packet becomes overly delayed, the packets
   after it on that flow are also delayed.  This is especially true for

Doesn't this imply that the real PDV data poitns are not independent and
that we should expect simple PDF modeling to produce inaccurate or
unphysical results?

(Suresh Krishnan) No Objection

Barry Leiba No Objection

Comment (2020-03-04 for -12)
I agree with the comments about Section 3.1 by Alexey and Adam.

With respect to Éric’s comment about Section 2, the verb is the last word, “apply”.  Only, the subject in “terminology”, which takes a singular verb, so it should be “applies”.

(Alexey Melnikov) No Objection

Comment (2020-03-03 for -12)
Thank you for this document.

I understand that section 3.1 is not serving the main purpose of this document, but I don't think it is implementable as written. In particular:

3.1.  RTP Log Format

   Having a common log format simplifies running analyses across and
   comparing different measurements.  The log file should be tab or
   comma separated containing the following details:

           Send or receive timestamp (unix)
           RTP payload type
           SSRC
           RTP sequence no
           RTP timestamp
           marker bit
           payload size

Is this sufficient inambiguous to be useful for interoperability? I.e. does each field has a single textual format? How is "marker bit" encoded?
Is each line CRLF or LF terminated?

Alvaro Retana No Objection

(Adam Roach) No Objection

Comment (2020-03-04 for -12)
Thanks for the work performed on this document. I agree with Alexey that Section 3.1 needs to either be fleshed out or removed, as the current specification is insufficiently specified to create interoperable analysis tools. You might look to RFC 6873 for an example of the degree of precision that is called for here.

Éric Vyncke No Objection

Comment (2020-03-02 for -12)
Thank you for the work put into this document. It is indeed important to be able to compare apples with apples.

I have a generic comment, is the case of multi-path / multi-home considered in this document ? 

Nevertheless, please find below some non-blocking COMMENTs (and I would appreciate a response from the authors) and NITS.

I hope that this helps to improve the document,

Regards,

-éric

== COMMENTS ==

-- Section 3 --
Please add references to PCAP format and expand the acronym.

-- Section 3.1 --
AFAIK, Unix timestamps have a 1-second accuracy but this document requires an accurancy of 200 msec. Is it expected that Unix timestamp are enough ?


== NITS ==

-- Section 2 --
The sentence "The terminology defined ..." is not a sentence as there is no verb :-)

Magnus Westerlund No Objection