ISP Dual Queue Networking Deployment Recommendations
draft-livingood-low-latency-deployment-06

Independent Stream                                          J. Livingood
Internet-Draft                                                   Comcast
Intended status: Informational                           14 October 2024
Expires: 17 April 2025

          ISP Dual Queue Networking Deployment Recommendations
               draft-livingood-low-latency-deployment-06

Abstract

   The IETF's Transport Area Working Group (TSVWG) has finalized
   experimental RFCs for Low Latency, Low Loss, Scalable Throughput
   (L4S) and new Non-Queue-Building (NQB) per hop behavior.  These
   documents describe a new architecture and protocol for deploying low
   latency networking.  Since deployment decisions are left to
   implementers, this document explores the potential implications of
   those decisions and makes recommendations that can help drive
   adoption and acceptance of L4S and NQB.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 17 April 2025.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.

Livingood                 Expires 17 April 2025                 [Page 1]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  A New Understanding of Application Needs  . . . . . . . . . .   3
   3.  New Thinking on Low Latency Packet Processing . . . . . . . .   4
   4.  Key Low Latency Networking Concepts . . . . . . . . . . . . .   5
     4.1.  Prioritization: Same Best Effort Priority for All
           Traffic . . . . . . . . . . . . . . . . . . . . . . . . .   5
     4.2.  Throughput: Shared Across All Traffic . . . . . . . . . .   5
     4.3.  Access-Agnostic: Can Work on All Types of Network
           Technology  . . . . . . . . . . . . . . . . . . . . . . .   5
   5.  Recommended Deployment Decisions  . . . . . . . . . . . . . .   6
     5.1.  Applications Mark Traffic . . . . . . . . . . . . . . . .   6
     5.2.  Any Application Provider Can Mark Traffic - No Advance
           Permission  . . . . . . . . . . . . . . . . . . . . . . .   7
     5.3.  Support a Variety of End User Equipment . . . . . . . . .   7
     5.4.  Consider Queue Protection . . . . . . . . . . . . . . . .   7
     5.5.  Avoid Internal Remarking of DSCP Values if Possible . . .   8
     5.6.  In Home Wi-Fi LAN Configuration . . . . . . . . . . . . .   8
     5.7.  Use Permissive Upstream NQB Queue Admission . . . . . . .   8
     5.8.  Pass Markings to Customer-Administered CPE  . . . . . . .   9
   6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   9.  Regulatory Considerations . . . . . . . . . . . . . . . . . .  10
   10. Revision History  . . . . . . . . . . . . . . . . . . . . . .  10
   11. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . .  11
   12. Informative References  . . . . . . . . . . . . . . . . . . .  11
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  14

1.  Introduction

   The IETF's Transport Area Working Group (TSVWG) has finalized
   experimental RFCs for Low Latency, Low Loss, Scalable Throughput
   (L4S) and Non-Queue-Building (NQB) per hop behavior [RFC9330]
   [RFC9331] [RFC9332] [I-D.ietf-tsvwg-l4sops] [I-D.ietf-tsvwg-nqb]
   [I-D.ietf-tsvwg-dscp-considerations].  These documents do a good job
   of describing a new architecture and protocol for deploying low
   latency networking.  But as is normal for many such standards,
   especially those in experimental status, certain deployment decisions
   are ultimately left to implementers.

   This document explores the potential implications of key deployment
   decisions and makes recommendations for those decisions that may help
   drive adoption.  In particular, there are best practices based on
   prior experience as a network operator that should be considered and
   there are network neutrality types of considerations as well.  These
   technologies are benign on their own, but the way they are

Livingood                 Expires 17 April 2025                 [Page 2]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   operationally implemented can determine whether they are ultimately
   perceived positively and adopted by the broader Internet ecosystem.
   That is a key issue for low latency networking, because the more
   applications developers and edge platforms that adopt new packet
   marking for low latency traffic, then the greater the value to end
   users, so ensuring it is received well is key to driving strong
   initial adoption.

   It is worth stating though that these decisions are not embedded in
   or inherent to L4S and NQB per se, but are decisions that can change
   depending upon differing technical, regulatory, business or other
   requirements.  Even two network operators with the same type of
   access technology and in the same market area may choose to implement
   in different ways.  Nevertheless, this document suggests that certain
   specific deployment decisions can help maximize the value of low
   latency networking to both users and network operators.

   The IETF documents on L4S and NQB also made clear that nearly all
   modern application types - from video conferencing to web browsing -
   can benefit from low latency networking.  In addition, the design of
   the protocols also make clear that applications developers are best
   positioned to understand the needs of their applications and to, by
   extension, express any such low latency needs via appropriate L4S or
   NQB packet marking.  Furthermore, unlike with bandwidth priority on a
   highly/fully utilized link, low latency networking can better balance
   the needs of different types of best effort flows (with some caveats
   - see Section 3).

   For additional background on latency and why latency matters so much
   to the Internet, please read [BITAG].

2.  A New Understanding of Application Needs

   In the course of working to improve the responsiveness of network
   protocols, the IETF concluded with their L4S and NQB work that there
   were fundamentally two types of Internet traffic and that these two
   major traffic types could benefit from having separate network
   processing queues in order to improve the way the Internet works for
   all applications, and especially for interactive applications.

   One of the two major traffic types is Queue Building (QB) - things
   like file downloads and backups that are designed utilize as much
   network capacity as possible but with which users are usually not
   interacting with in real-time.  The other was Non-Queue-Building
   (NQB) - such as DNS lookups, voice interaction with artificial
   intelligence (AI) assistants, video conferencing, gaming, and so on.
   NQB flows tend to be ones where the end user is sensitive to any
   delays.

Livingood                 Expires 17 April 2025                 [Page 3]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   Thus, the IETF created specifications for how two different network
   processing queues.  The performance value of dual queue networking
   (simply "low latency networking" hereafter) has proven out in
   Comcast's dual queue networking field trial [Comcast].  That field
   trial lasted for over a year and was regularly reported on over time
   at several IETF TSVWG meetings [IETF-TSVWG-117] [IETF-TSVWG-118]
   [IETF-TSVWG-119] [IETF-TSVWG-120] and demonstrated that L4S and NQB
   can work deliver excellent responsiveness for a variety of
   applications - from video conferencing to cloud gaming, DNS and other
   applications.  It seems likely that this new capability will enable
   entirely new classes of applications to become possible, driving a
   wave of new Internet innovation, while also improving the
   applications people use today.

   +--------------------------------+---------------------------------+
   | May Benefit from L4S/NQB       | Classic Queue Building          |
   +--------------------------------+---------------------------------+
   | User interacting with screen   | Unattended background process   |
   +--------------------------------+---------------------------------+
   | Video conference, audio        | Cloud file backup, cloud file   |
   | conference, live event video   | restoration, game platform      |
   | stream, cloud document editing | update, operating system update |
   +--------------------------------+---------------------------------+

                   Table 1: Comparison of Traffic Types

3.  New Thinking on Low Latency Packet Processing

   The Introduction says that unlike with bandwidth priority on a
   highly/fully utilized link, low latency networking can better balance
   the needs of different types of best effort flows.  But this bears a
   bit of further discussion to understand more fully.

   L4S does *not* provide low latency in the same way as previous
   technologies like DiffServ Quality of Service (QoS).  That prior QoS
   approach used packet prioritization, where it was possible to assign
   a higher relative priority to certain application traffic, such as
   Voice over IP (VoIP) telephony.  This approach could provide
   consistent and relatively low latency by assigning high priority to a
   partition of the capacity of a link, and then policing the rate of
   packets using that partition.  This traditional approach to QoS is
   hierarchical in nature.

   That QoS approach is to some extent predicated on an idea that
   network capacity is very limited and that links are often highly
   utilized.  But in today's Internet, it is increasingly the case that
   there is an abundance of capacity to end users (e.g., symmetric 1
   Gbps), which makes such traditional QoS approaches ineffective in

Livingood                 Expires 17 April 2025                 [Page 4]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   delivering ever-lower latency.  This new low latency networking
   approach is not based on hierarchical QoS prioritization.  Rather, it
   is built upon conditional priority scheduling between its two queues
   that operate at best effort QoS priority.

4.  Key Low Latency Networking Concepts

4.1.  Prioritization: Same Best Effort Priority for All Traffic

   Low latency traffic to is not prioritized over other (best effort
   priority) "classic" Internet traffic.  That is the case over the ISP
   network and the broader internet, though it may not not necessarily
   be the case for a user's in-home Wi-Fi network due to the particulars
   of how the IEEE 802.11 wireless protocol [IEEE] functions at the
   current time - see [RFC8325]).  In addition, some user access points
   may prioritize certain traffic (such as gaming) and some traffic such
   as NQB may use the AC_VI Wi-Fi link layer queue [I-D.ietf-tsvwg-nqb].
   This best effort approach stands in contrast to prior differential
   quality of service (QoS) approaches or to what has been discussed for
   5G network slicing [CDT-NN] [van-Schewick-1A] [van-Schewick-1B]
   [van-Schewick-2] [van-Schewick-3].

4.2.  Throughput: Shared Across All Traffic

   Low latency networking flows do not get access to greater throughput
   than "classic" flows.  Thus, a user's total provisioned or permitted
   throughput on an ISP access network link is shared between both
   classic and low latency queues.

4.3.  Access-Agnostic: Can Work on All Types of Network Technology

   Ultimately, the emergence of low latency networking represents a
   fundamental new network capability that applications can choose to
   utilize as their needs dictate.  It reflects a new ground truth about
   two fundamentally different types of application traffic and
   demonstrates that networks continue to evolve in exciting ways.  This
   new network capability can be implemented in a variety of network
   technologies.  For example in access network technologies this could
   be implemented in DOCSIS [LLD], 5G [Ericsson], PON [CTI], and many
   other types of networks.  Anywhere that a network bottleneck could
   occur may benefit from this technology.

Livingood                 Expires 17 April 2025                 [Page 5]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

5.  Recommended Deployment Decisions

   Like any network or system, a good deployment design and
   configuration matters and can be the difference between a well-
   functioning and accepted system and one that experiences problems and
   faces user and/or developer opposition.  In the context of deploying
   low latency networking in an ISP network, this document describes
   some recommendations that should help to ensure a deployment is
   resilient, well-accepted, and creates the environment for generating
   strong network effects.  Following these recommendations should help
   avoid barriers to adoption and help provide a strong foundation for
   growth.

5.1.  Applications Mark Traffic

   The best approach is for applications to mark traffic to indicate
   their preference for the low latency queue, not the network making
   such a decision on its own.  This is for several reasons:

   *  According to the end-to-end principle, this function is best
      delegated to the edge of the network as an architectural best
      practice (the edge being the application in this case).

   *  Application marking maintains the loose coupling between the
      application and network layers, eliminating the need for close
      coordination between networks and application developers.

   *  Application developers know best whether their application is
      compatible with low latency networking and which aspects of their
      traffic flows will or will not benefit.

   *  Only the application (not the network) knows whether a scalable
      congestion control algorithm congestion control is being used on
      the application server.  Thus, only the developer and server
      administrator know if they are correctly responding to Congestion
      Experienced (CE) markings for L4S (see Section 4.1 of [RFC9331]).

   *  Application traffic is almost entirely encrypted, which makes it
      very difficult for networks to accurately determine application
      protocols and to further infer which flows will benefit from low
      latency and which flows may be harmed because they need to build a
      queue.  It is likely that false positives [Lotus] and false
      negatives will occur if network-based inference is used; all of
      which can be avoided simply by relying solely on application
      marking.

Livingood                 Expires 17 April 2025                 [Page 6]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   *  The pace of innovation and iteration is necessarily faster-moving
      in the application edge at layer 7, rather than in the network at
      layer 3 (and below) - where there is greater standards stability
      and a lower rate of major changes.  As a result, the application
      layer is best suited to rapid experimentation and iteration.
      Network operators and equipment vendors trying to infer
      application needs will in comparison always be in a reactive mode,
      one step behind changes made in applications.

5.2.  Any Application Provider Can Mark Traffic - No Advance Permission

   Any application provider should be able to mark their traffic for the
   low latency queue, with no restrictions other than standards
   compliance or other reasonable and openly documented technical
   guidelines.  This maintains the loose cross-layer coupling that is a
   key tenet of the Internet's architecture by eliminating the need for
   application providers and networks to coordinate and creates an
   environment of so-called "permissionless innovation".

5.3.  Support a Variety of End User Equipment

   To create the best foundation for growth, both customer-owned and
   ISP-administered Customer Premises Equipment (CPE) should be
   supported (assuming that is typical and technically feasible in a
   particular type of access network), when applicable.  In mobile
   networks, where a user device connects directly to the network, this
   may be easier as it simply depends upon operating system software in
   that end user device.  In fixed networks, there is usually CPE that
   demarcates between the ISP network and the user's home LAN.  The
   software running on those CPE devices will also need to support low
   latency networking, as well as software in other ISP network devices
   where CPE connections are aggregated.  The more CPE devices that are
   enabled, the greater the pool of potential users, and thus the
   broader adoption would be, positively driving network effects.

5.4.  Consider Queue Protection

   The specifications in [I-D.ietf-tsvwg-nqb] describe a concept of
   Traffic Protection, also known as a Queue Protection Function
   [I-D.briscoe-docsis-q-protection] or simply Queue Protection.  The
   document says that Traffic Protection is optional and may not be
   needed in certain networks.  In the case of an ISP deploying low
   latency networking with two queues, an ISP should consider deploying
   such a network function to at least detect mismarking (if not
   necessarily to correct mismarking).  This may be implemented, for
   example, in end user CPE, last mile network equipment, and/or
   elsewhere in the ISP network - or closely monitors network statistics
   and user feedback for any indication of widespread low latency packet

Livingood                 Expires 17 April 2025                 [Page 7]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   mismarking by applications.

5.5.  Avoid Internal Remarking of DSCP Values if Possible

   If possible, for operational simplicity, a network should try to
   maintain the use of DSCP 45 on an end-to-end basis without remarking
   in their interior network hops.  This may not be possible in all
   networks, because some may already use DSCP 45 for some private
   internal reason.  In such cases, packets must obviously be remarked
   to and from DSCP 45 at the customer edge (CPE) and network ingress/
   egress to other networks.  But if DSCP 45 is not used internally, it
   is far simpler for network operations and troubleshooting to preserve
   that mark on an end to end basis.

5.6.  In Home Wi-Fi LAN Configuration

   As noted above with respect to prioritization of packets in the ISP
   network, all packets should be handled with the same best effort
   priority.  However, in a user's home Wi-Fi (wireless) local area
   network (WLAN), this is more complicated, because there is not a
   precise mapping between IETF packet marking and IEEE 802.11 marking,
   explored in [RFC8325].  In short, today's 802.11 specifications
   enable a Wi-Fi network to have multiple queues, using different "User
   Priority" and "Access Category" values.  At the current time, these
   queues are AC_BK (Background), AC_BE (Best Effort), AC_VI (Video),
   and AC_VO (Voice).

   As explored in [I-D.ietf-tsvwg-nqb], packets in the low latency queue
   may be marked for the best effort (AC_BE) or video (AC_VI) wireless
   queue.  For additional context, please refer to Section 8.1 of
   [I-D.ietf-tsvwg-nqb].  In some situations, such as a user-owned
   wireless access point or CPE, it may not be possible for the user to
   select which wireless queue is used.  In cases where the CPE is ISP-
   administered, selecting a specific wireless queue may be possible -
   though it is not yet clear what the best practice may be for this
   selection until ISPs and application developers have more experience
   with low latency networking.

5.7.  Use Permissive Upstream NQB Queue Admission

   Since the IETF's NQB specification is only recently completed, many
   applications that have been using other DSCP marks for their latency-
   sensitive flows have not yet shifted to adopt DSCP-45.  One example
   is the Microsoft Xbox platform [Microsoft], which is using DSCP-46.
   So in the relatively short-term, ISPs may find it beneficial to their
   customers to use a more permissive upstream NQB admission policy,
   allowing DSCP-40, 45, 46, and 56 admission into the low latency
   queue.  It may take a year or more after the NQB DSCP assignment is

Livingood                 Expires 17 April 2025                 [Page 8]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   made by IANA for developers to shift to DSCP-45, given other items in
   their development backlog and their software release schedule.

5.8.  Pass Markings to Customer-Administered CPE

   In some cases, dual queue networking and associated packet marking is
   supported up to the ISP's demarcation point - such as in a cable
   modem.  It is recommended that packet markings should pass from such
   a demarcation point to any attached customer-administered CPE, such
   as a router or wireless access point.  That enables a customer-
   administered router to implement dual queue networking, rather that
   it only being possible with ISP-administered CPE.

6.  Acknowledgements

   Thanks to Bob Briscoe, Mat Ford, Vidhi Goel, Eliot Lear, Sebastian
   Moeller, Sebnem Ozer, Jim Rampley, Dan Rice, Greg Skinner, Greg
   White, and Yiannis Yiakoumis for their review and feedback on this
   document.

7.  IANA Considerations

   RFC Editor: Please remove this section before publication.

   This memo includes no requests to or actions for IANA.

8.  Security Considerations

   The key security consideration pertains to Queue Protection.  As the
   current time, it is recommended that implementers utilize Queue
   Protection, to ensure that any traffic that is incorrectly marked for
   low latency can be detected and remarked for the classic queue.  The
   necessity of Queue Protection remains something of a debate, with
   some firmly believing it is necessary but others believing that it is
   not needed.  The latter view is that application developers have a
   natural incentive to correctly mark their traffic, because to do
   otherwise would worsen the quality of experience (QoE) for their
   users.  In that line of thinking, if a developer mismarks, they and/
   or their users will notice and they will fix that error.  However, it
   is also conceivable that malicious software could be operating on a
   user's device or home network and that malicious software could try
   to send some much traffic to the low latency queue that the queue or
   both queues become unusable.  This is quite similar to other
   "traditional" denial of servce (DoS) attacks, so it does not
   necessarily seems unique to low latency networking.  But due to the
   possibility of this occuring, and low latency networking being such a
   new approach, it seems prudent to implement Queue Protection.

Livingood                 Expires 17 April 2025                 [Page 9]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

9.  Regulatory Considerations

   Network Neutrality (a.k.a.  Net Neutrality) can mean a variety of
   things within a country, as well as between different countries,
   based on different opinions, market structures, business practices,
   laws, and regulations.  Generally speaking, In the context of the
   United States' market, it has come to mean that Internet Service
   Providers (ISPs) should not block, throttle, or deprioritize lawful
   application traffic, and should not engage in paid prioritization,
   among other things.  Net Neutrality concerns can sometimes affect the
   deployment of new technologies by ISPs, so they should carefully
   consider regulatory issues when making deployment decisions.

   As it is envisioned in the design of the IETF's new low latency
   networking protocols, the addition of a low latency packet processing
   queue at a network link is merely a second packet queue and does not
   mean that this queue is hierarchically prioritized or that it has
   more capacity.  As a result, low latency networking appears to pose
   no new Net Neutrality issues.  However, it is important for ISPs to
   keep these risks in mind as they make deployment design decisions.

   One key aspect of low latencty networking is that it operates, from
   the perspective of an ISP's deployment, is application-agnostic.  The
   ISP creates a second network queue on key network links, but does not
   decide on their own what applications can use this queue.  Rather,
   they add the queue and packet flows are sent to that queue based on
   packet marking set by application developers.  This approach is far
   superior to older approaches, which caused significant Net Neutrality
   risks [Lotus], that used middleboxes to attempt to infer applications
   based on observing packet flows on ISP network links.

10.  Revision History

   RFC Editor: Please remove this section before publication.

   v00: First draft

   v01: Incorporate comments from 1st version after IETF-115

   v02: Incorporate feedback from the TSVWG mailing list

   v03: Final feedback from TSVWG and prep for sending to ISE

   v04: Refresh expiration before major revision

   v05: Changes from Greg Skinner and Eliot Lear

   v06: More changes from Eliot Lear

Livingood                 Expires 17 April 2025                [Page 10]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

11.  Open Issues

   RFC Editor: Please remove this section before publication.

   - Open issues are being tracked in a GitHub repository for this
   document at https://github.com/jlivingood/IETF-L4S-Deployment/issues

12.  Informative References

   [RFC8325]  Szigeti, T., Henry, J., and F. Baker, "Mapping Diffserv to
              IEEE 802.11", RFC 8325, DOI 10.17487/RFC8325, February
              2018, <https://www.rfc-editor.org/info/rfc8325>.

   [RFC9330]  Briscoe, B., Ed., De Schepper, K., Bagnulo, M., and G.
              White, "Low Latency, Low Loss, and Scalable Throughput
              (L4S) Internet Service: Architecture", RFC 9330,
              DOI 10.17487/RFC9330, January 2023,
              <https://www.rfc-editor.org/info/rfc9330>.

   [RFC9331]  De Schepper, K. and B. Briscoe, Ed., "The Explicit
              Congestion Notification (ECN) Protocol for Low Latency,
              Low Loss, and Scalable Throughput (L4S)", RFC 9331,
              DOI 10.17487/RFC9331, January 2023,
              <https://www.rfc-editor.org/info/rfc9331>.

   [RFC9332]  De Schepper, K., Briscoe, B., Ed., and G. White, "Dual-
              Queue Coupled Active Queue Management (AQM) for Low
              Latency, Low Loss, and Scalable Throughput (L4S)",
              RFC 9332, DOI 10.17487/RFC9332, January 2023,
              <https://www.rfc-editor.org/info/rfc9332>.

   [I-D.ietf-tsvwg-l4sops]
              White, G., "Operational Guidance on Coexistence with
              Classic ECN during L4S Deployment", Work in Progress,
              Internet-Draft, draft-ietf-tsvwg-l4sops-06, 17 March 2024,
              <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-
              l4sops-06>.

   [I-D.ietf-tsvwg-nqb]
              White, G., Fossati, T., and R. Geib, "A Non-Queue-Building
              Per-Hop Behavior (NQB PHB) for Differentiated Services",
              Work in Progress, Internet-Draft, draft-ietf-tsvwg-nqb-25,
              31 August 2024, <https://datatracker.ietf.org/doc/html/
              draft-ietf-tsvwg-nqb-25>.

   [I-D.ietf-tsvwg-dscp-considerations]
              Custura, A., Fairhurst, G., and R. Secchi, "Considerations
              for Assigning a new Recommended DiffServ Codepoint

Livingood                 Expires 17 April 2025                [Page 11]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

              (DSCP)", Work in Progress, Internet-Draft, draft-ietf-
              tsvwg-dscp-considerations-13, 3 March 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-
              dscp-considerations-13>.

   [I-D.briscoe-docsis-q-protection]
              Briscoe, B. and G. White, "The DOCSIS(r) Queue Protection
              Algorithm to Preserve Low Latency", Work in Progress,
              Internet-Draft, draft-briscoe-docsis-q-protection-07, 23
              November 2023, <https://datatracker.ietf.org/doc/html/
              draft-briscoe-docsis-q-protection-07>.

   [BITAG]    Broadband Internet Technical Advisory Group, "Latency
              Explained", 10 January 2022,
              <https://bitag.org/documents/BITAG_latency_explained.pdf>.

   [Lotus]    Eckerseley, P., von Lohmann, F., and S. Schoen, "Packet
              Forgery By ISPs: A Report on the Comcast Affair", 28
              November 2007, <https://www.eff.org/wp/packet-forgery-
              isps-report-comcast-affair>.

   [IETF-114-Slides]
              White, G., "First L4S Interop Event @ IETF Hackathon", 25
              July 2022,
              <https://datatracker.ietf.org/meeting/114/materials/
              slides-114-tsvwg-update-on-l4s-work-in-ietf-114-hackathon-
              00.pdf>.

   [LLD]      White, G., Sundaresan, K., and B. Briscoe, "Low Latency
              DOCSIS: Technology Overview", February 2019,
              <https://cablela.bs/low-latency-docsis-technology-
              overview-february-2019>.

   [Ericsson] Willars, P., Wittenmark, E., Ronkainen, H., Johansson, I.,
              Strand, J., Ledl, D., and D. Schnieders, "Enabling time-
              critical applications over 5G with rate adaptation", May
              2021, <https://www.ericsson.com/49bc82/assets/local/
              reports-papers/white-papers/26052021-enabling-time-
              critical-applications-over-5g-with-rate-adaptation-
              whitepaper.pdf>.

   [CTI]      International Telecommunications Union - Telecommunication
              Standardization Sector (ITU-T), "Optical line termination
              capabilities for supporting cooperative dynamic bandwidth
              assignment", Series G: Transmission Systems and Media,
              Digital Systems and Networks Supplement 71, April 2021,
              <https://www.itu.int/rec/T-REC-G.Sup71-202104-I>.

Livingood                 Expires 17 April 2025                [Page 12]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   [IEEE]     IEEE Computer Society (IEEE), "Part 11: Wireless LAN
              Medium Access Control (MAC) and Physical Layer (PHY)
              Specifications", DOI 10.1109/IEEESTD.2021.9363693, IEEE
              Standard for Information Technology--Telecommunications
              and Information Exchange between Systems - Local and
              Metropolitan Area Networks--Specific
              Requirements 802.11-2020, 26 February 2021,
              <https://ieeexplore.ieee.org/document/9363693>.

   [Microsoft]
              Microsoft, "Quality of service (QoS) packet tagging on
              Xbox consoles", 19 August 2022,
              <https://learn.microsoft.com/en-
              us/gaming/gdk/_content/gc/networking/overviews/qos-packet-
              tagging>.

   [Comcast]  Comcast, "Comcast Kicks Off Industry's First Low Latency
              DOCSIS Field Trials", 16 June 2023,
              <https://corporate.comcast.com/stories/comcast-kicks-off-
              industrys-first-low-latency-docsis-field-trials>.

   [IETF-TSVWG-120]
              Livingood, J., "TSVWG Meeting at IETF-120", 26 July 2024,
              <https://datatracker.ietf.org/doc/slides-120-tsvwg-52-
              comcasts-l4s-nqb-field-trials/>.

   [IETF-TSVWG-119]
              Livingood, J., "TSVWG Meeting at IETF-119", 18 March 2024,
              <https://datatracker.ietf.org/doc/slides-119-tsvwg-sessa-
              41-comcasts-l4s-nqb-field-trials/>.

   [IETF-TSVWG-118]
              Livingood, J., "TSVWG Meeting at IETF-118", 18 November
              2023, <https://datatracker.ietf.org/doc/slides-118-tsvwg-
              sessa-61-l4s-experience/>.

   [IETF-TSVWG-117]
              Livingood, J., "TSVWG Meeting at IETF-117", 27 July 2023,
              <https://datatracker.ietf.org/doc/slides-118-tsvwg-sessa-
              61-l4s-experience/>.

   [CDT-NN]   Doty, N. and M. Knodel, "Slicing the Network: Maintaining
              Neutrality, Protecting Privacy, and Promoting Competition.
              A technical and policy overview with recommendations for
              operators and regulators.", April 2023,
              <https://arxiv.org/pdf/2308.05829>.

Livingood                 Expires 17 April 2025                [Page 13]
Internet-Draft  ISP Dual Queue Networking Deployment Rec    October 2024

   [van-Schewick-1A]
              van Schewick, B., Jordan, S., Open Technology Institute at
              New America, and Public Knowledge, "FCC Ex Parte In the
              matter of Safeguarding and Securing the Open Internet, WC
              Docket No. 23-320", 20 March 2024,
              <https://www.fcc.gov/ecfs/document/103120890811342/1>.

   [van-Schewick-1B]
              van Schewick, B., Jordan, S., Open Technology Institute at
              New America, and Public Knowledge, "Net Neutrality & Non-
              BIAS Data Services", 20 March 2024,
              <https://www.fcc.gov/ecfs/document/10323701322790/2>.

   [van-Schewick-2]
              van Schewick, B., "Net Neutrality & 5G Network Slicing", 3
              April 2024, <https://law.stanford.edu/wp-
              content/uploads/2024/08/van-Schewick-2024-5G-Network-
              Slicing-and-Net-Neutrality-Shetler-Steffen1.pdf>.

   [van-Schewick-3]
              van Schewick, B., "Network Slicing and Net Neutrality: No
              Throttling Rule", 18 April 2024,
              <https://law.stanford.edu/wp-content/uploads/2024/08/van-
              Schewick-2024-5G-Network-Slicing-and-No-Throttling-Rule-
              20240418.pdf>.

Author's Address

   Jason Livingood
   Comcast
   Philadelphia, PA
   United States of America
   Email: jason_livingood@comcast.com

Livingood                 Expires 17 April 2025                [Page 14]