Operational Considerations for Streaming Media
draft-ietf-mops-streaming-opcons-12

[Ballot comment]
Thanks to Michael Scharf for his excellent TSVART review. I look forward to the dialogue about his comments, and in particular would like to concur that 1 second seems like an awful high bound for "ultra-low latency."

- Please update the references to RFC793 and RFC4960 with their bis equivalents, which are both in the RFC editor queue.

- (6.3) While QUIC endpoints can unilaterally adopt congestion controls, this is of course absolutely the case in TCP and SCTP as well, so it's odd to single it out in this section.

- (6.2) implies that reducing bufferbloat is pretty new idea. As I'm sure the authors are well aware, delay-based congestion control goes back to the early 90s and TCP Vegas, though it's absolutely true that it's currently in a bit of a renaissance.

[Ballot comment]
Thanks for this document. It is a useful informative read.

I agree with Roman's DISCUSS, and even think some of his COMMENTS could have been DISCUSS items. (especially his note on the word "Unfortunately").

COMMENTS:

  Bittorrent favored peers
  who uploaded as much as they downloaded, so that new Bittorrent users
  had an incentive to significantly increase their upstream bandwidth
  utilization.

Bram Cohen, who wrote the protocol and main implementation, has commented many times that this "tit for tat" scheme was abandoned early on as it was causing more problems than useful prioritization.

[Ballot comment]
Thanks for this document, I found it interesting and easy to read. All of my comments other than the last one are minor proofreading or style points.

1. In the Introduction, “a continuous media” makes me sad because of the disagreement in number. I get that this may reflect common usage but it still made me wince. The obvious way to make it conventionally grammatical would be “a continuous media stream” but I suppose that would make the definition recursive. :-( So, I don’t have a good solution to offer but maybe you do; if so that’d be nice.

2. Introduction, “match to client's consumption rate” should be “match the client's consumption rate”.

3. In Section 3.6, “Bittorrent favored peers
  who uploaded as much as they downloaded, so that new Bittorrent users
  had an incentive to significantly increase their upstream bandwidth
  utilization.”

I think you don’t mean “so that”, but just “so” — the implied causal arrow is reversed by the “that”, you mean something like “therefore”, right?

4. In 3.7 you seem to have a stray close brace? “}”

5. In 3.7 you misspelled Craig Labovitz’s name as “Labowitz”. The "v" spelling is correct.

6. Section 5.5.3, s/detction/detection/

7. Section 6.1, s/tansport/transport/

8. Section 6.1, “approaches like the above generally
  experiences”, should be “generally experience” (agreement in number).

9. Section 7, “prevent media steam manipulation” should be “stream”.

10. Section 7.2 has

  The choice of whether to involve intermediaries sometimes requires
  careful consideration.  As an example, when ABR manifests were
  commonly sent unencrypted some networks would modify manifests during
  peak hours by removing high-bitrate renditions in order to prevent
  players from choosing those renditions, thus reducing the overall
  bandwidth consumed for delivering these media streams and thereby
  improving the network load and the user experience for their
  customers.  Now that ubiquitous encryption typically prevents this
  kind of modification, in order to maintain the same level of network
  health and user experience across networks whose users would have
  benefitted from this intervention a media streaming operator
  sometimes needs to choose between adding intermediaries who are
  authorized to change the manifests or adding significant extra
  complexity to their service.

It wasn’t immediately obvious to me as a reader what recourse (other than intermediaries) the media streaming operator would have, regardless of their willingness to add extra complexity.

- Are you implicitly referencing something you’ve touched on elsewhere in the document (an xref would be nice if so),

- or are such “extra complexity” solutions known and published (again, a ref would be nice),

- or known but proprietary special sauce (maybe there’s nothing to be done, although a few words indicating this might be suitable),

- or are you just speculating on the basis that as the saying goes, with enough thrust, even a brick can fly?

[Ballot comment]
Thank you for this document -- I found it both interesting and informative, and covered a topic which I know almost nothing about.

I'd suggest that the authors look at and respond to Linda Dunbar's very useful OpsDir review: https://datatracker.ietf.org/doc/review-ietf-mops-streaming-opcons-10-opsdir-lc-dunbar-2022-05-06/

[Ballot comment]
Thanks for working on this document. It has some details that is very useful to read. Thanks to Michael Scharf for his TSVART review.

As instructed by my fellow AD to review this document as it is instead of waiting for updates, I am doing it but I would like to see the TSVART review addressed. The TSVART review has some very good comments.

Overall, by looking at the abstract of this document and the content of it, it feels like it goes beyond "operational networking issues" or I felt to see in some place what is the issue it conveys (I suggested an update see my comments below).

Below I have some comments which I believe when addressed will improve the document.


# Comments

## Abstract : the abstract should also say that this document is about both networking and transport issues, as it does in the introduction.

## Section 1 :

  - "Streaming media" - does this covers both live streaming and VoD? this would be useful to convey in the beginning of this document.

  - it says "the server's transmission rate must (loosely or tightly) match to client's consumption rate in order to provide uninterrupted playback.", this might not be true if caching is involved. I realize the term "server" is a bit ambiguous here as it does not differ between a origin server and caching server at this point.

  - it says "the client's consumption rate is limited not only by bandwidth availability,but also media availability. The client cannot fetch media that is not available from a server yet." I would suggest to s/media/media segment(s). The media description file may be available but media segments might not, specially for live streaming media. and if the media is not available there is no consumption rate.

  - Minor comment - the beginning sentence in this section is way too long to read. I would suggest it to be split into several sentences.

## Section 2:

  - reference CVNI : can't read hence can't verify, seems like behind paywall, the URL leads to https://www.cisco.com/c/en/us/solutions/service-provider/index.html

  - I thought (from the introduction section) that unreliable media is out of scope of this document then I see some details will be provided later sections. I would not categorize unreliable RTP as a part of streaming media, even though streaming over RTP is a reality or did I misunderstood something?

## Section 3.1: I think it will be useful to add that

      - applications that use the video encoder can ask for lower bitrate encoding as the cost of video quality.

      - usually the video encoding does not take the available bandwidth into account.

      - for VoD case, the switching between the video bitrate depends on the available pre-encoded content at a particular bitrate.

## Section 3.2: AFAIK, the media player actually can detect the buffer under-run and can ask the server to switch the bitrate up by selecting a higher bitrate encoded media segment. I have not seen any mention of that in this section, is this part of mitigation that is mentioned? if not then this section is a bit incomplete capture the whole mechanism I am afraid.

## Section 3.7: I am confused about section 3.7 not sure what exactly to get out of this section, specially what is the impact on network operation related to streaming media.

## Section 4: the categorization of latency requirement will need a reference if not the categorization is done and agreed in the WG.

## Section 5.4: if streaming of advertising and modulation quality uses same technology as media streaming, then do we need to this long section for the Ad? In the later part this section talks about ad fraud and mitigation for that which I find completely application/player specific or integrated with application. what is there for network OPS that is important in the context of this document?

## Section 5.5.3: it says "5G and LTE likewise can easily see rate variation by a factor of 2 or more over a span of seconds as users move around". Can we have pointer to show some data supporting this, like we have the Micro? is the because of the 5G and LTE technology itself or is this about network planning issues?

## What about active measurements? it would be very useful to review them and say why they are not relevant or not relevant for steaming media as part of section 5.

## I would suggest to define the terms like media/content "producer" and media/content "provider", or refer to where they are defined to be used in this document. It is not that understandable from the text in this document. also state the difference between media provider and content provider as both has been used in this document.

## Section 6: I don't mind the transport protocol commentary :-), however, I don't really see how the provided commentary actually achieves what the hanging paragraphs in this section wanted to achieve. To me, it is possible that media can be fetched from a server or a intermediary, and the media player metrics collected to sense the observed media quality can be send to some other nodes in the network. This means unless the media distributor and media provider are same it will be hard to get a proper picture about the contention and fairness (if that is really necessary and yes this adds to the point I made about defining media provider and producer terms). The media application will have to rely on the underlying transport protocol(s) for some sort of fairness among the flows sharing same bottleneck, thats it. Yes, transport protocols will evolve and the ware image and congestion control might change but the fundamentals of transport services perhaps won't. To that extend I am kind of with the INTDIR an TSVART reviewer's view. 


## It seems like in the document only media consumed by human is considered, there are other streaming cases where the media can be consumed by machines. If later is out of scope then this need to be implicitly stated in this document.


## There has been more than one place where "we" is used. Even though I like to read in active form of narratives in documents, here the use of "we" felt a bit odd.

[Ballot comment]
# GEN AD review of draft-ietf-mops-streaming-opcons-10

CC @larseggert

## Comments

### Section 6.3, paragraph 0
```
  6.3.  QUIC and Its Behavior
```
I'm surprised this section doesn't refer to draft-ietf-quic-manageability? That
document discusses some of the same (and related) issues.

### Section 6.3, paragraph 1
```
    Although QUIC provides an alternative to the TCP and UDP transport
    protocols, QUIC is itself encapsulated in UDP.  As noted elsewhere in
    Section 7.1, the QUIC protocol encrypts almost all of its transport
    parameters, and all of its payload, so any intermediaries that
    network operators may be using to troubleshoot HTTP streaming media
    performance issues, perform analytics, or even intercept exchanges in
    current applications will not work for QUIC-based applications
    without making changes to their networks.  Section 7 describes the
    implications of media encryption in more detail.
```
"Transport parameter" is a QUIC-specific term to describe connection-level
settings that are negotiated during the handshake. I think this paragraph - and
others that mention transport parameters in this document - mean to talk about
the broader concept of "connection metadata". I would suggest not overloading
the QUIC "transport parameter" term in this way.

### Inclusive language

Found terminology that should be reviewed for inclusivity; see
https://www.rfc-editor.org/part2/#inclusive_language for background and more
guidance:

* Terms `his` and `he`; alternatives might be `they`, `them`, `their`
* Term `traditional`; alternatives might be `classic`, `classical`, `common`,
  `conventional`, `customary`, `fixed`, `habitual`, `historic`,
  `long-established`, `popular`, `prescribed`, `regular`, `rooted`,
  `time-honored`, `universal`, `widely used`, `widespread`

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Typos

#### Section 3.2, paragraph 1
```
-    the contraints on bandwidth at various points in the network.  This
-    analysis is necessary because media servers may react to bandwith
+    the constraints on bandwidth at various points in the network.  This
+          +
+    analysis is necessary because media servers may react to bandwidth
+                                                                  +
```

#### Section 3.2, paragraph 7
```
-      transport-level loss is occuring, but
+      transport-level loss is occurring, but
+                                    +
```

#### Section 5.4, paragraph 3
```
-    (herafter referred to as "ads").
+    (hereafter referred to as "ads").
+        +
```

#### Section 5.5.3, paragraph 1
```
-    hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detction have
+    hop (Wi-Fi, 5G, or LTE), new problems in bandwidth detection have
+                                                          +
```

#### Section 6.1, paragraph 3
```
-    In contrast to adaptive segmented delivery over a reliable tansport
+    In contrast to adaptive segmented delivery over a reliable transport
+                                                                +
```

#### Section 6.3, paragraph 5
```
-    application until the lost packet is retransmitted, allowing in-order
-                                      ^    ^^^^^^ ^^
+    application until a retransmission of the lost packet has been received, allowing in-order
+                      ++++++++++++++++++++                ^^ +++++  ^^ ^
```

#### Section 6.3, paragraph 7
```
-    As noted in Section 6.2, there is an increasing interest in transport
-                                                                ^^ -----
-    protocol behaviors that respond to delay measurements, instead of
-    ^ ----  --- ^^
-    responding to packet loss.  These behaviors may deliver improved user
-                                      --- ^^
+    As noted in Section 6.2, there is an increasing interest in congestion
+                                                                ++++++ ^^
+    control algorithms that respond to delay measurements, instead of
+    ^^^^    ^^  ++++
+    responding to packet loss.  These algorithms may deliver improved user
+                                      ^^  ++++
```

#### Section 6.3, paragraph 7
```
-    agility, and [RFC9002] defines a basic algorithm with transport
-                                                    ---------------
-    behavior that is roughly similar to TCP NewReno [RFC6582].  However,
-  ---------
+    agility, and [RFC9002] defines a basic congestion control algorithm
+                                          +++++++++++++++++++
```

#### Section 7.2, paragraph 9
```
-    benefitted from this intervention a media streaming operator
-          -
```

#### Section 7.3, paragraph 4
```
-    monitoring of IETF protocols to continue for the forseeable future.
+    monitoring of IETF protocols to continue for the foreseeable future.
+                                                        +
```

### Duplicate references

Duplicate informative references to:
`https://datatracker.ietf.org/meeting/interim-2020-mops-01/materials/slides-interim-2020-mops-01-sessa-april-15-2020-mops-interim-an-update-on-streaming-video-alliance`.

### Outdated references

Reference `[RFC2001]` to `RFC2001`, which was obsoleted by `RFC2581` (this may
be on purpose).

### Grammar/style

#### Section 1, paragraph 12
```
Discussion This document is in the Github repository at: https://github.com/
                                  ^^^^^^
```
The official name of this software platform is spelled with a capital "H".

#### Section 3.6, paragraph 1
```
"throttle" these transfers in order to to mitigate the load that these hosts
                                    ^^^^^
```
Possible typo: you repeated a word.

#### Section 3.6, paragraph 3
```
a [Mishra] reported that after the CoViD-19 pandemic broke out in early 2020
                                  ^^^^^^^^
```
Did you mean "COVID-19" or the alternative spelling "Covid-19" (= coronavirus)?

#### Section 3.6, paragraph 4
```
ffic up 36% over an average day (pre COVID-19)}. We note that other operators
                                ^^^^^^^^^^^^
```
The word "pre-COVID-19" is spelled with a hyphen.

#### Section 4.1, paragraph 2
```
o, and some tradeoffs may be necessary relative to what is feasible in a hig
                            ^^^^^^^^^^^^^^^^^^
```
In this context, the adverb seems more correct than the adjective "necessary".

#### Section 4.1, paragraph 5
```
rations. However, shorter segments means more frequent intra-coded frames an
                                  ^^^^^
```
It seems that the correct verb form here is "mean".

#### Section 4.2, paragraph 3
```
ecessary to support low-latency live streaming. This level of latency can typ
                                ^^^^^^^^^^^^^^
```
This expression is normally spelled as one or with a hyphen.

#### Section 5.3, paragraph 1
```
oth content and ads) are able to be accessed quickly. The less targeted, the
                                ^^^^^^^^^^^
```
Avoid the passive voice after "to be able to".

#### Section 6.3, paragraph 12
```
ransport parameters itself, with the exception of a few invariant header fie
                            ^^^^^^^^^^^^^^^^^^^^^
```
Consider using "except" or "except for".

#### Section 7.2, paragraph 5
```
Reading and References The Media Operations community maintains a list of ref
                                ^^^^^^^^^^
```
An apostrophe may be missing.

#### Section 7.2, paragraph 8
```
C., Zimmermann, R., and A. Bentaleb et al, "A Survey on Bitrate Adaptation Sc
                                    ^^^^^
```
A period is misplaced or missing. (Also in other references.)

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

[Ballot comment]
# Internet AD comments for {draft-ietf-mops-streaming-opcons-10}
CC @ekline

## Comments

### S3.4

* It seems like the final paragraph is attempting to describe a real-world
  experience.  It would be great if there were a citation for this (but
  understandable if nothing had been made public).

### S3.6

* RFC 5594 does not appear to contain the word "throttle".  It might not
  be great to inject this term here?  5594 instead seems to use phrases
  describing approaches to "congestion management".

### S4.4

* Seems like the introduction of the term "manifest" without any
  definition, explanation, or reference.  Is there some explanatory text
  or reference that might be included?

[Ballot discuss]
** Section 5.4. 

  Ads may be inserted either with Client Side Ad Insertion (CSAI) or
  Server Side Ad Insertion (SSAI).  In CSAI, the ABR manifest will
  generally include links to an external ad server for some segments of
  the media stream, while in SSAI the server will remain the same
  during advertisements, but will include media segments that contain
  the advertising.  In SSAI, the media segments may or may not be
  sourced from an external ad server like with CSAI.
 
        …

  As a
  mitigation for concerns driven by those incidents, some SSPs have
  required the use of players with features like reporting of ad
  delivery, or providing information that can be used for user
  tracking.  Some of these and other measures have raised privacy
  concerns for end users.

Thanks for starting the discussion about privacy.  The framing doesn’t seem completely accurate.  Whether there is ad fraud or not, user data of some kind is being sent off to ad exchanges (it’s the basis of the bidding process), and network level tracking is being facilitated through connects to CSAIs.  Please provide some editorial construct to suggest that practically any kind of targeted ads are going to entail some trade in privacy, and explain the risks specifically or with a reference.

[Ballot comment]
** Thanks for distilling a diverse ecosystem into a single document.  To make navigating this ecosystem easier, please provide definitions for terms like “media provider”, “streaming media provider”, “intermediary”, “intermediate streaming operators”, and “content provider”; and show a notional architecture between them.  If some of these terms or elements are interchangeable, please make it clear.  I’ll call out some of the places where I got confused below.

** Section 3.5.  Is there are reference that can be provided to support the assertion that better codecs (which save bandwidth) can’t offset increased video consumption.

** Section 4.  Is the definition of “ultra low-latency” used here the same as “ultra low latency caches” in Section 3.4?

** Section 4.2.  What is a “premium service to the delivery of live video”?  Is that “premium” in the sense of my ISP and associated provisioning? Or in the sense of my relationship with the content provider?

** Section 4.2.  This section lists HLS, DASH and a few other enabling technologies.  Are the technologies for ultra low-latency applicable?

** Section 4.3. 

  This level
  of latency is often considered adequate for content like news or pre-
  recorded content. 

What is the difference between the “pre-recorded content” described here and the “on-demand” described in Section 4.4?

** Section 5.4

  In general this is a rapidly developing space with many
  considerations, and media streaming operators engaged in advertising
  may need to research these and other concerns to find solutions that

Who are “media streaming operators”?  Are those different than “media providers”

** Section 6.1.
  … we have trusted UDP-based
  applications to limit their impact on other users

Who is the “we”?  Can this “trust” please be clarified.  Same comment for Section 6.2.

** Section 6.1.

  Although it is
  possible to saturate a path between a DNS client and DNS server with
  DNS requests, in practice, that was rare enough that DNS included few
  mechanisms to resolve contention between DNS users and other users
  (whether they are also using DNS, or using other application
  protocols that share the same pathways).

Can this observation please be restated?  How is a DNS server to resolve contention for non-DNS traffic?

** Section 7.
      Media encrypted at the application layer, typically using some
      sort of Digital Rights Management (DRM) system, and typically
      remaining encrypted "at rest", when senders and receivers store
      it.

Is this proposed scheme where the media remains encrypted even when at rest in fact just describing object encryption/security – that is, the security properties of the encrypted media hold and are independent of the communication mechanism transporting them?

** Section 7.

  The use of strong encryption does provide confidentiality for
  encrypted streaming media, from the sender to either an intermediary
  or the ultimate media consumer, and this does prevent Deep Packet
  Inspection by any intermediary that does not possess credentials
  allowing decryption. 

What is an intermediary here?  Is it any on-path observer?

** Section 7.1.

  An intermediary that can identify an
  encrypted media stream without decrypting it, may be able to
  "fingerprint" the encrypted media stream of known content, and then
  match the targeted media stream against the fingerprints of known
  content.  This protection can be lessened if a media provider is
  repeatedly encrypting the same content. 

-- I had trouble following the setup.  I assumed the text was trying to abstractly describe the methodology in [CODASPY17].  Recommend:

NEW
An on-path observer that can identify that encrypted traffic contains a media stream, could “fingerprint” this encrypted media steam, and then compare it against “fingerprints” of known content.

-- Per the last sentence, “[t]this protection …”, what protection is being lessened here?

** Section 7.1

  If traffic analysis is successful at identifying encrypted content
  and associating it with specific users, this breaks privacy as
  certainly as examining decrypted traffic.

Please be more precise on the security properties being lost rather than summarizing it has “privacy”

** Section 7.2.

  If a content provider does not actively work to avoid interception by
  intermediaries, the effect will be indistinguishable from
  "impersonation attacks", and endpoints cannot be assumed of any level
  of privacy.

Please be precise on what security properties are in play from “impersonation attacks” and loss of “privacy”.  In the situation described above, it seems to be both confidentiality (e.g., the intermediary can see the entirety of all communication between the end-point and the media provider) and authenticity (e.g., no party can trust the content received in fact came from the expected party)

** Section 7.2

      Server And Network assisted DASH [MPEG-DASH-SAND] - this
      specification introduces explicit messaging between DASH clients
      and network elements or between various network elements

Are “network elements” the same as “intermediaries?”

** Section 7.2
  The choice of whether to involve intermediaries sometimes requires
  careful consideration.  As an example, when ABR manifests were
  commonly sent unencrypted some networks would modify manifests during
  peak hours by removing high-bitrate renditions in order to prevent
  players from choosing those renditions, thus reducing the overall
  bandwidth consumed for delivering these media streams and thereby
  improving the network load and the user experience for their
  customers. 

-- Please be clear on who is deciding to involve the intermediaries. Is it the media provider?

-- It isn’t clear who is operating these intermediaries – is it the media provider or the connectivity provider of the end user?  I took to be the latter.  Assuming that’s right, how is this a design choice by the media providers? Did the media providers explicitly choose to use an unencrypted protocol to support these network operators; or did the network operators exploit this design choice without coordination?

** Section 7.2

Now that ubiquitous encryption typically prevents this
  kind of modification, in order to maintain the same level of network
  health and user experience across networks whose users would have
  benefitted from this intervention a media streaming operator
  sometimes needs to choose between adding intermediaries who are
  authorized to change the manifests or adding significant extra
  complexity to their service.

There is an implicit architectural assumptions being made that I’m not following.  This text suggests that the media streaming operators are operating the intermediaries.  Are these intermediaries in a network controlled by the media streaming operator?  Before, intermediaries were modifying manifests, and now due to encryption more intermediaries are needed?

** Section 7.3.

Unfortunately, as noted in [RFC7258], there is no way to prevent
  pervasive monitoring by an "attacker", while allowing monitoring by a
  more benign entity who "only" wants to use DPI to examine HTTP
  requests and responses in order to provide a better user experience.

s/Unfortunately//.  Keeping "unfortunately" is a value judgement.  Some users might be quite pleased by this development.

** Section 7.3.  What is an “Intermediary streaming operator”?  How is that different than a “media provider” or “media streaming provider”?

Editorial

** Section 1.  Consider if you can simplify the first sentence, “This document examines …”.  It is dense.

** Section 1.  Typo. s/availability,but/availability, but/

** Section 3.2.  Typo. s/contraints/constraints/

** Section 3.2. s/bandwith/bandwidth/

** Section 3.2.  Typo. s/occuring/occurring/

** Section 3.4  Per the paragraph starting with “Caching and pre-loading …”, consider if this can be simplified.  It’s exactly one sentence long with multiple “especially …” clauses. 

** Section 3.4.  Editorial

  And as with other parts of the ecosystem, new technology brings new
  challenges.  For example, with the emergence of ultra-low-latency
…
Consider if the first sentence is needed.  This who paragraph seems to be related to caches  Perhaps just start this paragraph with “With the emergence of …”

** Section 3.5. Consider defining “mobile data.”

** Section 4.1.

  This introduces new challenges relative to less-
  restricted levels of latency requirements because this latency is the
  same scale as commonly observed end-to-end network latency variation
  (for example, due to effects such as bufferbloat ([CoDel]), Wi-Fi
  error correction, or packet reordering). 

This sentence doesn’t parse.

** Section 4.1.

  These effects can make it
  difficult to achieve this level of latency for the general case, and
  may require tradeoffs in relatively frequent user-visible media
  artifacts.

The wording isn’t right here –- “… may require tradeoffs ...”  Does this text mean “... may require accepting relatively frequent user-visible media artifacts.”

** Section 5.5.3.  Typo. s/detction/detection/

** Section 6.1.  Per “In recent times”, please restate this as this phrase is not precise and is unlikely to age well.

** Section 6.1.  Typo. s/tansport/transport/

** Section 6.3.  Per “… without melting the Internet”, consider a less colloquial expression.

** Section 6.3.  Per “As noted in [RFC8312], both the CUBIC congestion controller and its predecessor …”, consider re-writing this sentence for readability.  The double “and both were ...” clauses makes this text very dense.

** Section 7.3.  The opening paragraph of this section appears to be a restatement of the opening paragraph of Section 7.1.

[Ballot comment]

Hi,

Thank you for this document and thanks Linda for the OPS DIR review.  I find these sorts of documents to be informative and interesting, and I found this document to be interesting and pleasant to read - so thank you for that.

I only have a couple of trivial comments:

(1) The introduction made it clear that "download and play" is out of scope, but it was less clear to me from the introduction as to whether RTC was meant to be in scope or not.

(2)
5.1.  Overview
"ad workflows are introduced." 
- "ad" is being used before it is later defined in 5.4.

Regards,
Rob

[Ballot comment]

Hi,

Thank you for this document and thanks Linda for the OPS DIR review.  I find these sorts of documents to be informative and interesting, and I found this document to be interesting and pleasant to read - so thank you for that.

I only have a couple of trivial comments:

(1) The introduction made it clear that "download and play" is out of scope, but it was less clear to me from the introduction as to whether RTC was meant to be in scope or not.

(2)
5.1.  Overview
"ad workflows are introduced." 
- "ad" is being used before it is later defined in 5.4.

Regards,
Rob


I don't have any significant commne

Closed request for Last Call review by OPSDIR with state 'Overtaken by Events': Discovered the request too late. Please make new request with a new deadline date

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-mops-streaming-opcons-10, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

Michelle Thangtamsatid
IANA Services Specialist

The following Last Call announcement was sent out (ends 2022-05-06):

From: The IESG
To: IETF-Announce
CC: draft-ietf-mops-streaming-opcons@ietf.org, evyncke@cisco.com, mops-chairs@ietf.org, mops@ietf.org, sanjay.mishra@verizon.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Operational Considerations for Streaming Media) to Informational RFC


The IESG has received a request from the Media OPerationS WG (mops) to
consider the following document: - 'Operational Considerations for Streaming
Media'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2022-05-06. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document provides an overview of operational networking issues
  that pertain to quality of experience when streaming video and other
  high-bitrate media over the Internet.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-mops-streaming-opcons/



No IPR declarations have been submitted directly on this I-D.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational. This document does not specify any protocol(s) and only contains terminology and descriptions.  Yes,  the "type of RFC" as Informational is noted in the title page header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This document is focused on operational - networking - issues in dealing with  quality of experience for streamed video and other high-bitrate media over the Internet.

The document identifies video as a significant part of the traffic on the Internet and while this video traffic can be handled transparently as generic application-level traffic but that the network design decisions can impact application-level performance and consequently quality of video delivery.


Working Group Summary:

Was there anything in WG process that is worth noting? For example, was there controversy about particular points or were there decisions where the consensus was particularly rough?

This draft is very straight forward and is intended as Informational. There were no controversies for the Draft as it progressed through the working group last call. In addition to the mailing list
discussions and other input occurred in the GitHub repository (https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons)

Document Quality:

Are there existing implementations of the protocol? Have a significant number of vendors indicated their plan to implement the specification? Are there any reviewers that merit special mention as having done a thorough review, e.g., one that resulted in important changes or a conclusion that the document had no substantive issues? If there was a MIB Doctor, YANG Doctor, Media Type or other expert review, what was its course (briefly)? In the case of a Media Type review, on what date was the request posted?

Not applicable. This document does not proposes any new protocol but simply provides guidelines - Informational - for better understanding how the streamed media flows through the network.


Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Document shepherd  is Sanjay Mishra and the responsible area director is Eric Vyncke.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

Document is reviewed by Sanjay Mishra and that the quality is document as written is good and can proceed for IESG's review.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

The document is well reviewed within the working group and I have no concerns on the depth and breadth of those reviews. 

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No broader review is necessary for this document.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns. There have been no controversial discussions that arose out of the reviews.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

There is a strong consensus from the WG as a whole.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No threats.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

None. Idnits tool flags no issues.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
Not applicable.

(13) Have all references within this document been identified as either normative or informative?

All references are Informative.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No. There are no normative references.

(15) Are there downward normative references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

Not applicable. This document has no IANA changes.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

Not applicable. The document specifies no actions for IANA.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

Not applicable. None of the document is written in a formal language.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

Not applicable.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational. This document does not specify any protocol(s) and only contains terminology and descriptions.  Yes,  the "type of RFC" as Informational is noted in the title page header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This document is focused on operational - networking - issues in dealing with  quality of experience for streamed video and other high-bitrate media over the Internet.

The document identifies video as a significant part of the traffic on the Internet and while this video traffic can be handled transparently as generic application-level traffic but that the network design decisions can impact application-level performance and consequently quality of video delivery.


Working Group Summary:

Was there anything in WG process that is worth noting? For example, was there controversy about particular points or were there decisions where the consensus was particularly rough?

This draft is very straight forward and is intended as Informational. There were no controversies for the Draft as it progressed through the working group last call. In addition to the mailing list
discussions and other input occurred in the GitHub repository (https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons)

Document Quality:

Are there existing implementations of the protocol? Have a significant number of vendors indicated their plan to implement the specification? Are there any reviewers that merit special mention as having done a thorough review, e.g., one that resulted in important changes or a conclusion that the document had no substantive issues? If there was a MIB Doctor, YANG Doctor, Media Type or other expert review, what was its course (briefly)? In the case of a Media Type review, on what date was the request posted?

Not applicable. This document does not proposes any new protocol but simply provides guidelines - Informational - for better understanding how the streamed media flows through the network.


Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Document shepherd  is Sanjay Mishra and the responsible area director is Eric Vyncke.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

Document is reviewed by Sanjay Mishra and that the quality is document as written is good and can proceed for IESG's review.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

The document is well reviewed within the working group and I have no concerns on the depth and breadth of those reviews. 

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No broader review is necessary for this document.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns. There have been no controversial discussions that arose out of the reviews.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

There is a strong consensus from the WG as a whole.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No threats.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

None. Idnits tool flags no issues.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
Not applicable.

(13) Have all references within this document been identified as either normative or informative?

All references are Informative.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No. There are no normative references.

(15) Are there downward normative references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

Not applicable. This document has no IANA changes.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

Not applicable. The document specifies no actions for IANA.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

Not applicable. None of the document is written in a formal language.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

Not applicable.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational and is also noted in the title page header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This document is focused on operational - networking - issues in dealing with  quality of experience for streamed video and other high-bitrate media over the Internet.

The document identifies video as a significant part of the traffic on the Internet and while this video traffic can be handled transparently as generic application-level traffic but that the network design decisions can impact application-level performance and consequently quality of video delivery.


Working Group Summary:

Was there anything in WG process that is worth noting? For example, was there controversy about particular points or were there decisions where the consensus was particularly rough?

This draft is very straight forward and is intended as Informational. There were no controversies for the Draft as it progressed through the working group last call. In addition to the mailing list
discussions and other input occurred in the GitHub repository (https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons)

Document Quality:

Are there existing implementations of the protocol? Have a significant number of vendors indicated their plan to implement the specification? Are there any reviewers that merit special mention as having done a thorough review, e.g., one that resulted in important changes or a conclusion that the document had no substantive issues? If there was a MIB Doctor, YANG Doctor, Media Type or other expert review, what was its course (briefly)? In the case of a Media Type review, on what date was the request posted?

Not applicable. This document does not proposes any new protocol but simply provides guidelines - Informational - for better understanding how the streamed media flows through the network.


Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Document shepherd  is Sanjay Mishra and the responsible area director is Eric Vyncke.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

Document is reviewed by Sanjay Mishra and that the quality is document as written is good and can proceed for IESG's review.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

The document is well reviewed within the working group and I have no concerns on the depth and breadth of those reviews. 

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No broader review is necessary for this document.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns. There have been no controversial discussions that arose out of the reviews.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

There is a strong consensus from the WG as a whole.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No threats.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

None. Idnits tool flags no issues.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
Not applicable.

(13) Have all references within this document been identified as either normative or informative?

All references are Informative.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No. There are no normative references.

(15) Are there downward normative references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

Not applicable. This document has no IANA changes.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

Not applicable. The document specifies no actions for IANA.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

Not applicable. None of the document is written in a formal language.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

Not applicable.