Deterministic Networking (DetNet) Data Plane Framework
Note: This ballot was opened for revision 04 and is now closed.
Comment (2020-04-21 for -04)
(1) References: I don't think that RFC3209, RFC3473 or RFC4385 need to be Normative as they seem to be used mostly as examples. (2) [nits] s/reproduced from the [RFC8655],shows/reproduced from [RFC8655] shows s/[RFC4385],can be used/[RFC4385], can be used s/constraint on its size of/constraint on the size of s/RFC5575/draft-ietf-idr-rfc5575bis
Comment (2020-04-21 for -04)
My colleagues have covered the nits quite well.
Comment (2020-04-12 for -04)
Editorial nits mostly. The more interesting stuff first: Throughout the document, various terms are sometimes capitalized and sometimes not. A pass should be done to make this uniform throughout the document. This becomes especially confusing in Section 188.8.131.52. Examples: "App-flows". "Service sub-layer", "Forwarding sub-layer", "Packet". Section 2.2: * "CW" by itself doesn't appear anywhere in this document. Neither do "LSR", "MPLS-TE", or "PW". Section 3.1: * Given the sentence in 3.1, I would expect the "technology" and "encapsulation" sections to be at the same level in the Table Of Contents tree, but they aren't. That is, since "Data Plane Technology" is 3.1.1, I would expect "Encapsulation" to be 3.1.2. Is some adjustment in order here? Section 3.3: * This paragraph needs a bit of editing: -- OLD -- Metadata can be included implicit or explicit. Explicit means that a dedicated header field is used to include metadata in a DetNet packet. In case of implicit method a part of an already existing header field is used to encode the metadata. -- SUGGEST -- Metadata inclusion can be implicit or explicit. Explicit inclusions involve a dedicated header field that is used to include metadata in a DetNet packet. In the implicit method, part of an already existing header field is used to encode the metadata. Section 184.108.40.206: * "Use of a specific path for a flow." -- this isn't a sentence; suggest: "A flow can be routed over a specific, pre-computed path." Section 220.127.116.11: * "Use of multiple packet streams using multiple paths, for example 1+1 or 1:1 linear protection." -- again not a sentence; maybe start with "Service protection involves use of ..." Section 18.104.22.168: * "... are currently out of scope for this document." -- remove "currently"; these issues won't be in scope for this document later Section 22.214.171.124: * "An OAM scheme that monitors the paths detects the loss of path or traffic is required to initiate the switch." -- I can't parse this sentence. Maybe "detects" should be "to detect"? Section 3.6.3: * "the sum of the reservations should be the sum of all the individual reservations" -- there may be a term of art in play here, but that phrase seems self-redundant; what else could the latter be but the former? Section 4.2.1: -- OLD -- The assignment of resources along the path depends on the technology and it includes assignment of specific links and coordination of the queuing and other traffic management capabilities such as policing and shaping. -- NEW -- The assignment of resources along the path depends on the technology and includes assignment of specific links, coordination of queueing, and other traffic management capabilities such as policing and shaping. And now the boring stuff, to save the RFC Editor some work: Section 1: * "Different application flows (e.g., Ethernet, IP, etc.), can be mapped on top of DetNet." -- remove the comma Section 3: * "Figure 1 reproduced from the [RFC8655],shows ..." -- remove "the" and replace the comma with a space * "The ordering (POF) uses sequence numbers ..." -- should probably be "The ordering function (POF) uses sequence numbers ..." Section 3.3: * "... can provide or carry metadata:" -- add "the following" after "carry" Section 3.4: * "... for example GRE, IPSec etc." -- comma after "IPSec" Section 3.5: * "... the d-CW [I-D.ietf-detnet-mpls], [RFC4385],can be used." -- add a space before "can" * "... no architectural constraint on its size of this structure ... -- s/its/the/ Section 126.96.36.199: * "... for a certain characteristic e.g. highest ..." -- s/characteristic e.g. highest/characteristic, e.g., highest/ Section 188.8.131.52: * "For DetNet this primarily ..." -- add a comma after "DetNet" Section 184.108.40.206: * "Network Coding, [nwcrg] not to ..." -- move the comma to after "[nwcrg]" * "DetNet could utilized Network coding ..." -- s/utilized/utilize/ (or simply "use") Section 3.6.2: * "Service protection allow DetNet services ..." -- s/allow/allows/ Section 220.127.116.11: * "Figure 3: Example Packet Flow in DetNet protected Network" -- capitalize "protected" * "copy of packet 1.2 etc." -- add a comma after "1.2" * "...entities shown in Figure 3 ." -- s/3 ./3./ * "... traffic at a time, could use the ..." -- remove comma Section 18.104.22.168: * "Many of the same concepts apply however rings ..." -- add a comma after "apply" Section 3.6.3: * "When aggregating DetNet flows the flows should ..." -- add a comma after the first "flows" * "... should be compatible i.e. the same ..." -- s/compatible i.e./compatible, i.e.,/ * "When an encapsulation is used the choice ..." -- add comma after "used" Section 3.6.5: * "L2 represent a generic ..." s/represent/represents/ Section 4.1.: * The second citation of RFC8655 in the first paragraph is probably not needed. Section 4.2: * "While management plane and control planes ..." -- maybe "While the management and control planes ..." * Add a comma after "For example" in the last paragraph. Section 4.2.1: * "... provisioned or requested one or more ..." -- add comma after "requested" Section 4.2.4: -- OLD -- An example control plane solution for MPLS can be found in [RFC3473] , [RFC6387] and [RFC7551]. -- NEW -- Example control plane solutions for MPLS can be found in [RFC3473], [RFC6387], and [RFC7551].
Comment (2020-04-24 for -05)
Sorry, I ran out of time to fully review this document, but one surprise what that the title and abstract only mentions the dataplane, but then the document also has a reasonably sized section on control plane considerations, and I didn't know whether it would be helpful for the abstract to also mention this fact.
Comment (2020-04-22 for -04)
Doh! I had collected a whole pile of editorial nits and comments, but see that others have already covered them - that'll larn me to procrastinate...
Deborah Brungard Former IESG member
Yes (for -04)
Alissa Cooper Former IESG member
No Objection (for -04)
Barry Leiba Former IESG member
No Objection (2020-04-14 for -04)
Thanks for this document. Just lots of editorial comments here: The RPC will need to do a lot of comma editing. I’m not going to call them all out here. — Section 1 — It describes the function and operation of the Packet Replication (PRF) Packet Elimination (PEF) and the Packet Ordering (POF) functions You need commas between the items in the list. I also find “the fuction of the functions” to be very odd, and suggest eliminating “functiin and” from the beginning of the sentence. Furthermore, it also describes the forwarding sub-layer. “Furthermore” and “also” together is redundant, and I suggest eliminating one of them. Different application flows (e.g., Ethernet, IP, etc.) can be mapped “e.g.” and “etc.” together is redundant, and I suggest “Different application flows, such as Ethernet or IP, can be mapped”. If you keep the parentheses, the comma needs to disappear. — Section 3 — The DetNet Architecture, [RFC8655], models the DetNet related data plane functions You don’t need the commas to set off the citation, as the brackets already do that. And “DetNet-related” needs a hyphen. Figure 1 reproduced from the [RFC8655],shows Make this, “Figure 1, reproduced from [RFC8655], shows” — Section 3.1.1 — currently defined for operation over packet switched (IP) networks or label switched (MPLS) networks. Hyphenate “packet-switched” and “label-switched”. — Section 3.1.2 — used and no sequence number is available, and in DetNet MPLS, DetNet specific information Hyphenate “DetNet-specific”. — Section 3.2 — An example of such metadata is the inclusion of a sequence number required by the PREOF function. “PREOF function” is redundant, as the “F” already means “function”. Also later in the document. — Section 3.4 — may be deployed, for example GRE, IPSec etc. “for example” and “etc.” together is redundant. — Section 22.214.171.124 — Reservation of resources can allocate resources to specific DetNet flows. This sounds a bit odd. Maybe, “Resources might be reserved in order to make them available for allocation to specific DetNet flows.” ? Misbehaving DetNet flows must be able to be detected and ensure that they do not compromise QoS of other flows. This sounds like it’s something that misbehaving DetNet flows have to do. It would be better this way: NEW It must be possible to detect misbehaving DetNet flows and to ensure that they do not compromise QoS of other flows. END — Section 126.96.36.199 — Explicit route computation can encompass a wide set of constraints and optimize the path for a certain characteristic e.g. highest bandwidth or lowest jitter. This would sound better with “and can optimize”. And “e.g.” needs commas bith before and after it, always. — Section 188.8.131.52 — DetNet could utilized Network coding Make it “utilize” (or, better, “use”). — Section 184.108.40.206 — traffic over each segment of the end to end path. Hyphenate “end-to-end” here. In this case there is no PRF function “PRF function” is redundant. — Section 220.127.116.11 — DetNet uses flow specific requirements (e.g., maximum number of out-of-order packets, maximum latency of the flow, etc.) for configuration of POF related buffers. Hyphenate “flow-specific” and “POF-related”, and eliminate either “e.g.” or “etc.” — Section 18.104.22.168 — Many of the same concepts apply however rings are There needs to be a comma before “however”. — Section 3.6.3 — How this is accomplished is data plane or control plane dependent. NEW How this is accomplished is data-plane or control-plane dependent. END When aggregating DetNet flows the flows should be compatible i.e. the same or very similar QoS and CoS characteristics. NEW When aggregating DetNet flows, the flows should be compatible, i.e., have the same or very similar QoS and CoS characteristics. END — Section 3.6.5 — MPLS nodes may interconnected by different sub-network “may be interconnected” Each of these sub-network technologies need to provide appropriate service The subject is “each”, which is singular, so “needs to provide”. — Section 4.2 — While management plane and control planes are traditionally considered separately, from the Data Plane perspective Don’t capitalize “data plane” here. — Section 4.2.1 — There are many techniques to achieve aggregation, for example in case of IP, it can be grouping of IP flows that share 6-tuple attributes or flow identifiers at the DetNet sub-layer. Very awkward sentence. Split and rephrase: NEW There are many techniques to achieve aggregation. For example, in the case of IP, IP flows that share 6-tuple attributes or flow identifiers at the DetNet sub-layer can be grouped. END — Section 5 — The primary considerations for the data plane is to maintain integrity of data and delivery of the associated DetNet service “consideration”, singular. through the use of existing mechanism such as policing and shaping Make it “an existing mechanism” or “existing mechanisms”.
Benjamin Kaduk Former IESG member
No Objection (2020-04-23 for -04)
Other than some remarks on the security considerations, I basically just have a bunch of editorial comments. I've tried to deduplicate with what was already reported by other ADs, but no doubt have kept a few things in that have already been mentioned. Section 1 The DetNet Architecture models the DetNet related data plane functions decomposed into two sub-layers: a service sub-layer and a forwarding sub-layer. The service sub-layer is used to provide nit: "models" needs two objects to act on/compare, so maybe "as being decomposed into". replicated in other forwarding technologies. Most of DetNet benefits can be gained by running over a data link layer that has not been specifically enhanced to support all TSN capabilities but for certain networks and traffic mixes delay and jitter performance may vary due to the forwarding sub-layer intrinsic properties. nit: I think the "certain networks and traffic mixes" are supposed to be contingent on "a data link layer that [is not a TSN]" but the current text does not quite match that. Perhaps "certain such networks" is the smallest fix, though it does conflate "data link layer" and "network" in an unfortunate manner. Section 3 connectivity function of the forwarding sub-layer. An example of this is Packet Replication, Elimination, and Ordering functions see Section 4.3. The ordering (POF) uses sequence numbers added to nit: the punctuation around the reference should be tweaked. The method of instantiating each of the layers is specific to the particular DetNet data plane method, and more than one approach may be applicable to a given bearer network type. What is a "bearer network"? Section 3.1.1 applying existing standardized headers and/or encapsulations. The Detnet forwarding sub-layer may provide capabilities leveraging that same header or encapsulation technology (e.g., DN IP or DN MPLS) or it may be achieved by other technologies (e.g., Figure 2). DetNet is nit: Figure 2 is not a technology; maybe "as shown in Figure 2". Section 3.1.2 number) in packets. For example, in DetNet IP, zero encapsulation is used and no sequence number is available, and in DetNet MPLS, DetNet specific information may be added explicitly to the packets in the format of S-label and d-CW [I-D.ietf-detnet-mpls] . nit: s/format of/form of/ Section 3.3 Number (for PREOF) for each DetNet flow. The DetNet Service sub- layer requires both; the DetNet forwarding sub-layer requires only Flow-ID. Metadata can also be used for OAM indications and nit(?): to say that "the service sublayer requires both" to me implies that it always and unconditionally requires both, but IIUC the PREOF functions are optional, and so for a given flow the service sublayer might not require the sequence number. Perhaps swapping it around to be structured more like "the flow-ID is used by both the service and forwarding sub-layers, but the sequence number is only used by the service layer" would help? Some MPLS examples of implicit metadata include the sequence number for use by the PREOF function, or even all the essential information being included into the DetNet over MPLS label stack (the DetNet Control Word and the DetNet Service label). I would have thought that the detnet elements in the label stack would be considered explicit metadata, but perhaps I'm just looking through the wrong lens. Section 3.4 One method of operating an IP DetNet data plane without encapsulation is to use "6-tuple" based flow identification, where "6-tuple" refers to information carried in IP and higher layer protocol headers. General background on the use of IP headers, and "6-tuples", to identify flows and support Quality of Service (QoS) can be found in [RFC3670]. [RFC7657] provides useful background on differentiated I did not see the explicit phrase "6-tuple" used in RFC 3670, so I don't think this text is quite right as written. (7657 does talk about it, but is not currently being referenced for that purpose.) Section 22.214.171.124 Is there a difference between "packet-by-packet distribution [of the same flow]" and "packet-by-packet load sharing"? Section 126.96.36.199 as those in use by IP and MPLS networks. At the Application layer a client of a DetNet service can use existing techniques to detect and monitor delay and loss. [Is there a good reference for these "existing techniques"? I recognize that we shouldn't go into extensive detail here, and absent a single consolidated reference, no-reference may be best.] Section 3.6.2 Service protection allow DetNet services to increase reliability and maintain a DetNet Service Assurance in the case of network congestion or network failure. Detnet relies on the underlying technology My understanding is that the whole point of DetNet was to provide reliability in the face of network congestion (e.g., by resource reservation). So is it really only the "network failure" case that benefits from service protection? Section 188.8.131.52 IP aggregation has both data plane and controller plane aspects. For the data plane, flows may be aggregated for treatment based on shared characteristics such as 6-tuple. Alternatively, an IP encapsulation In what way would 6-tuple be a "shared characteristic" for *different* flows? I thought 6-tuple was typically used as a unique flow identifier... (I see in Section 4.2.1 we refer to flows that "share 6-tuple attributes".) Section 4.1 However, from a practical and implementation standpoint, they are not equivalent at all. Some approaches are more scalable than others in terms of signaling load on the network. Some can take advantage of global tracking of resources in the DetNet domain for better overall network resource optimization. Some are more resilient than others if link, node, or management equipment failures occur. While a detailed analysis of the control plane alternatives is out of the scope of this document, the requirements from this document can be used as the basis of a later analysis of the alternatives. side note: using "some" so much requires the reader to know which are which ... that may be reasonable, but perhaps we want the reader's brainpower working on other things. Section 4.2 nit: RestConf and YANG probably are best considered together as a single "management mechanism" for these purposes. Section 4.2.2 DetNet application with the required service. A requirement for the DetNet Controller Plane will be the ability to assign a particular identified DetNet IP flow to a path through the DetNet domain that has been assigned the required nodal resources. This provides the nit: I don't think that "nodal" is quite the right word, here, as it refers to a property of the (network) graph more than the specific element colocated with the node in the graph. "per-node" seems like a better fit, to me. Section 4.3 domain requires explicit support. There may be capabilities that can be used, or extended, for example GMPLS end-to-end recovery [RFC4872] and GMPLS segment recovery [RFC4873]. nit: maybe this is "existing capabilities"? Section 5 I guess there's some standard security considerations for when flow aggregation is in play, namely that misbehavior from one component flow can affect sibling flows as collateral damage. Security considerations for DetNet are described in detail in [I-D.ietf-detnet-security]. General security considerations are The referenced document (now at -09) seems significantly improved from when I previously made an in-passing review of the -03 (https://mailarchive.ietf.org/arch/msg/detnet/jZLodXBmQa7ZFDbBIvDO_xCDD0E/), however, some of the issues I mentioned there remain, at least in part (e.g., mentioning the use of HMAC without a colocated discussion of the need for key distribution and having a taxonomy of threats titled "threat model"). I would like to know what the current maturity state of the detnet-security document is believed to be, so as to judge how appropriate it is for the data-plane-framework to refer to it in this manner. described in [RFC8655]. This section considers general security I think this is more like "Architecture-level DetNet" than "Generic", as the current formulation sounds like it should be referring to BCP 72 :) Security aspects which are unique to DetNet are those whose aim is to provide the specific quality of service aspects of DetNet, which are primarily to deliver data flows with extremely low packet loss rates and bounded end-to-end delivery latency. I can't find a way to read this other than "security aspects [...] whose aim is to provide the specific quality of service aspects", and I'm puzzling at why it's phrased in this way. Specifically, I thought that *providing* the QoS aspects is the core role of DetNet, and thus that the *security* aspects would be protecting those aspects in the face of (some class of) attackers. Would it be appropriate to replace this with something like the following? % Part of what makes DetNet unique is its ability to provide specific and % reliable quality of service (delivering data flows with extremely low % packet loss rates and bounded end-to-end delivery latency), and the % security-related aspects of protecting that quality of service are % similarly unique. This might also be a good place to note that correct operation of the PREOF functions is pretty critical, and failures there could lead to pretty catastrophic situations for the operational technology relying on them. I don't think there are ways for an attacker to try to attack them directly, but Murphy's Law can be a pretty effective attacker, too. The primary considerations for the data plane is to maintain integrity of data and delivery of the associated DetNet service traversing the DetNet network. Application flows can be protected I'd suggest leading in to this with "As for all communications protocols," to be clear that this paragraph is not trying to cover the bits that are "unique to DetNet" as much as remind the reader of general best practices. At the management and control level DetNet flows are identified on a per-flow basis, which may provide controller plane attackers with additional information about the data flows (when compared to controller planes that do not include per-flow identification). This is an inherent property of DetNet which has security implications that should be considered when determining if DetNet is a suitable technology for any given use case. I might also note that attackers with access to the controller plan already have pretty significant attacks available to them. To provide uninterrupted availability of the DetNet service, provisions can be made against DOS attacks and delay attacks. To protect against DOS attacks, excess traffic due to malicious or malfunctioning devices can be prevented or mitigated, for example through the use of existing mechanism such as policing and shaping applied at the input of a DetNet domain. To prevent DetNet packets In my head this would include things like disabling switch ports that are receiving traffic floods that might overload the switch CPU, in order to protect the DetNet traffic on other ports. I don't see much value in saying that in the document, though, since it's more of an operational practice thing. Section 9.1 RFC 3209 is cited only once, and it does not seem like a particularly normative one. Similarly, RFC 3473 is cited only as a "such as" example (though several times).