Deterministic Networking (DetNet) Data Plane: MPLS
Note: This ballot was opened for revision 11 and is now closed.
(Deborah Brungard) Yes
Roman Danyliw (was Discuss) No Objection
Thank you for addressing my DISCUSS and COMMENT feedback.
Martin Duke No Objection
Comment (2020-08-28 for -11)
I found the position of the OAM indicator to be somewhat confusing. 4.2 says it happens in the d-cw layer but there is no mentioning of it in the corresponding section, 4.2.1, instead buried in the S-label text. More importantly, it is correct that an OAM packet cannot test the PREOF functions because it does not have a d-cw, and therefore no sequence number? If so, that seems like a significant capability gap. 4.2.2. I would like to see some guidance for the POF and PEF as to how long it should store observed sequence numbers. This would appear to be one of the harder design decisions in the space, and some general principles would be helpful. For example, are there certain latency guarantees in the Detnet, so that after some time the sequence number can be safely discarded?
Benjamin Kaduk No Objection
Comment (2020-09-09 for -11)
There's nothing particularly earth-shattering in these remarks; just some things to double-check and potential minor improvements. (The combination of generic DetNet and MPLS security considerations cover just about everything that's going on here.) I also have a number of nit-level notes that I'll send to the authors separately. Section 2.2 I agree with the directorate reviewer that references for at least some of these terms would be useful. Section 3.1 I trust there's a reason for the figure to have "bottom of stack" at the top of the figure (and vice versa). The DetNet MPLS data plane representation is illustrated in Figure 1. The service sub-layer includes a DetNet control word (d-CW) and an identifying service label (S-Label). The DetNet control word (d-CW) conforms to the Generic PW MPLS Control Word (PWMCW) defined in [RFC4385]. An aggregation label (A-Label) is a special case of Just to check my understanding: we conform to the generic format from RFC 4385 but not the preferred one? Section 4.2.1 Do we want to say anything about why the RFC 4385 preferred-format's flags, fragmentation indicator, and length fields are not applicable to the DetNet usage? (I do not recall any restrictions that prevent DetNet flows from traversing Ethernet segments, one of the justifications given in RFC 4385 for the length field.) I would also suggest avoiding the "S/N" abbreviation (as colliding with "signal/noise"), and note that it is not listed at https://www.rfc-editor.org/materials/abbrev.expansion.txt . The sequence number length MUST be provisioned on a per Detnet service basis via configuration, i.e., via the controller plane described in [I-D.ietf-detnet-data-plane-framework]. (side note) I didn't find a great definition for "DetNet service" as a standalone term in RFC 8655, though it's clearly already in use there for this meaning. When the sequence number field length is 16 or 28 bits for a flow, the sequence number MUST be incremented by one for each new app-flow packet sent. When the field length is 16 bits, d-CW bits 4 to 15 Are there any considerations on the initial sequence number value? Randomization of the initial sequence number may be a generic best practice, as discussed in draft-gont-numeric-ids-sec-considerations (which I am AD sponsoring). Section 4.2.2 S-Label values MUST be provisioned per DetNet service via configuration, e.g., via the controller plane described in [I-D.ietf-detnet-data-plane-framework]. Note that S-Labels provide [...] MAY be allocated from the platform label space [RFC3031]. Because S-Labels are local to each node rather than being a global identifier within a domain, they must be advertised to their upstream DetNet service-aware peer nodes (e.g., a DetNet MPLS End System or a DetNet Relay or Edge Node) and interpreted in the context of their received I'm having a hard time reconciling "provisioned via configuration" with "advertised to their upstream peer nodes" -- can you help explain what is expected to happen here? An S-Label will normally be at the bottom of the label stack once the last F-Label is removed, immediately preceding the d-CW. To support service sub-layer level OAM, an OAM Associated Channel Header (ACH) [RFC4385] together with a Generic Associated Channel Label (GAL) [RFC5586] MAY be used in place of a d-CW. Does this preclude using an ACH in the absence of a GAL? In the case where platform labels are not used, zero or more F-Labels and optionally, the incoming interface, proceeding the S-Label MUST be used together with the S-Label to uniquely identify the DetNet service associated with a received packet. The incoming interface MAY also be used together with any present F-Label(s) and the S-Label to uniquely identify an incoming DetNet service, for example, in the case where PHP is used. [...] I'm not sure what the difference in meaning between these two sentences is supposed to be. Section 220.127.116.11 When multiple sets of outgoing F-Labels or interfaces are provisioned for a particular DetNet service, a copy of the outgoing packet, Would it be banal to note that this occurs as part of the PRF? S-label uniquely identifies the DetNet service. In the case where platform labels are not used, incoming F-Label(s) and, optionally, the incoming interface MUST also be provisioned for a DetNet service. This might be the third time we've mentioned this behavior; is it perhaps getting redundant? Section 4.3 OAM follows the procedures set out in [RFC5085] with the restriction that only Virtual Circuit Connectivity Verification (VCCV) type 1 is supported. Earlier we talked about OAM as just using the regular RFC 4385 ACH method (which is what VCCV type 1 corresponds to); why is it necessary to pull in the RFC 5085 procedures now (especially when we follow up two paragraphs down with "the reader is referred to [RFC5058] for a more detailed description")? Section 4.4.1 into and from an H-LSP. Both carried LSPs and H-LSPs may or may not use the TC field, i.e., L-LSPs or E-LSPs. Such nodes will need to Perhaps a RFC 3270 reference is in order here, to pick up L-LSPs and E-LSPs? We seem to not really mention it in this context until Section 4.6.1 otherwise. Additional details of the traffic control capabilities needed at a DetNet-aware node may be covered in the new service definitions mentioned above or in separate future documents. Controller plane Er, mentioned where? This seems to be the first instance of "service definition" in this document. Section 4.5 latency the impact on the DetNet application would be severe. To avoid the problem of a transient forwarding loop, changes to an LSP supporting DetNet MUST be loop-free. Just to check my understanding: it's possible to get this loop-free behavior with all the common control planes, e.g., RSVP-TE, MPLS-TP, MPLS SR, etc.? Section 5 and hybrid combinations of the two. The details of the controller plane solution required for the label distribution and the management of the label number space are out of scope of this document. There The details are out of scope, sure, but the requirements for what it needs to provide seem to be in scope. It looks like this is what is in the following sub-sections, which is good, but perhaps the transition to them could be written more explicitly. (I did not try to cross-reference the lists given here with the in-line requirements stated throughout the document, and trust the authors to have done so.) Section 5.1.1 Section 6 plane. The considerations raised related to MPLS networks in general in [RFC5920] are equally applicable to the the DetNet MPLS data plane. In line of Roman's remarks, I'd suggest calling out a few key pieces from the RFC 5920 security considerations, especially boundary filtering to protect against label spoofing. We might pull in the considerations from the relevant control word RFCs as well, and from RFC 4206 for H-LSPs. Some of the service label stuff is fairly analogous to the ongoing SFC work, but it's probably a stretch to say that we should reference their security considerations from here. There's a couple chunks of text that are essentially copied from RFC 8655 but seem incoherent or incorrect, e.g.: Security aspects which are unique to DetNet are those whose aim is to provide the specific quality of service aspects of DetNet, which are (It's DetNet's aim to provide those, not the security aspects' aim.) traffic. To prevent DetNet packets from being delayed by an entity external to a DetNet domain, DetNet technology definition can allow for the mitigation of Man-In-The-Middle attacks, for example through use of authentication and authorization of devices within the DetNet domain. (I don't think we've seen a clear portrayal of what these MITM protection schemes would actually do, what is being authenticated/authorized, etc., any of the times that it has come up.) I hope we can improve them in this iteration. Section 9 We're already over the 5-author limit; I, for one, would not mind having 7 authors (and skipping this section) instead of the current 6. Section 10.1 Some of these may not strictly need to be in the normative references section, e.g., 2211/2212, but it doesn't seem particularly harmful to keep them here. Section 10.2 I'm quite surprised that draft-ietf-detnet-data-plane-framework is not listed as normative. The reference to RFC 5586 is in "an OAM Associated Channel Header (ACH) [RFC4385] together with a Generic Associated Channel Label (GAL) [RFC5586] MAY be used in place of a d-CW", the normative keyword of which suggests that it should properly be classified as normative. Likewise for RFC 6790 ("Similarly, an Entropy Label Indicator/Entropy Label (ELI/EL) [RFC6790] MAY be carried below the S-Label").
Erik Kline No Objection
Comment (2020-09-07 for -11)
[[ questions ]] [ section 4.1 ] * "The 16 bit sequence number is needed to support some types of legacy clients." Should there be a SHOULD in here about defaulting to, say, use of 28 bit sequence numbers? [[ nits ]] [ section 4.2.1 ] * s/were the sequence/where the sequence/ [ section 4.2.2 ] * s/as outlines in/as outlined in/ [ section 4.2.3 ] * Maybe "are supported the" -> "support the"? [ section 18.104.22.168 ] * s/a service parameters/a service parameter/? * s/provisioning or related/provisioning of related/ * Maybe "requirement for MPLS LSRs to that CoS LSPs do not" -> "the requirement for MPLS LSRs that CoS LSPs do not" [ section 4.4.2 ] * s/to to/to/ [ section 4.6.2 ] * s/that maybe used/that may be used/ * s/can provided/can be provided/ [ section 5.1 ] * "each of the service's outgoing DetNet (member) flow" -> "each of the service's outgoing DetNet (member) flows" [ section 5.1.1 ] * Perhaps s/may be provided discussed/may be provided is discussed/ [ section 5.2 ] * Perhaps s/flow specific resources/flow-specific resources/
Murray Kucherawy No Objection
Comment (2020-09-10 for -12)
Not much to add here. I concur explicitly though with Barry's points about the document references.
Warren Kumari No Objection
Comment (2020-09-09 for -11)
Thanks to Shwetha for the OpsDir review, it was helpful to me for knowing where to focus my review. I think it would be nice if the document had a bit more text around the (lack of) issues caused by: "It is important to note that this document differs from [RFC4448] where a sequence number of zero (0) is used to indicate that the sequence number check algorithm is not used." I had to re-read this a few times and then spend some time trying to figure out what happens with implementations that follow RFC4448; as far as I can tell nothing bad happens, but having an extra sentence or two reassuring readers of this would be nice.
(Barry Leiba) No Objection
Comment (2020-09-03 for -11)
Just some small, non-blocking, and easy-to-address comments here: Abstract: This document builds on the DetNet Architecture and Data Plane Framework. Section 2.1: This document uses the terminology established in the DetNet architecture [RFC8655] and the DetNet Data Plane Framework [I-D.ietf-detnet-data-plane-framework]. The reader is assumed to be familiar with these documents, any terminology defined therein Then how is it possible that data-plane-framework is informative? I think it has to be normative. — Section 4.2.1 — and (2) to allow non-skip zero S/N what simplifies implementation. I can’t parse that. Can you fix the wording or explain what I’m missing? — Section 6 — for the mitigation of Man-In-The-Middle attacks This is entirely up to your judgment, but please consider using phrasing such as “for mitigating attackers-in-the-middle” here. Thanks.
Alvaro Retana No Objection
Comment (2020-09-08 for -11)
(0) I support Magnus' DISCUSS. [I also have some comments about §22.214.171.124/§126.96.36.199 below.] (1) §4.6.2: "DetNet provides zero congestion loss and bounded latency and jitter" This phrase is a variation of the general statement in the Introduction: "DetNet provides these flows with extremely low packet loss rates and assured maximum end-to-end delivery latency." Specific to this document, how does DetNet guarantee this behavior while operating on an MPLS network? I see that §4.5/rfc8655 focuses a series of examples on TSN (which is not required by the architecture, right?), and this document mentions other technology as examples. Still, there is no explicit indication of how the "zero congestion loss and bounded latency and jitter" can be achieved. Instead, the use of these mechanisms is left to the implementation/operator without any guidance. I think that not offering any guidance is a significant gap in a Standards Track document. Most of the mechanisms mentioned as examples in §4.6.2 are "old", for example, RSVP-TE. Pretty much the same mechanisms are listed in §188.8.131.52 when talking about the option of using DetNEt unaware nodes to deliver the same "service parameters": When the DetNet service parameters of the DetNet flow or flows carried in an LSP represented by an F-Label can be met by an existing TE mechanism, the forwarding sub-layer processing node MAY be a DetNet unaware, i.e., standard, MPLS LSR. Such TE LSPs may provide LSP forwarding service as defined in, but not limited to, [RFC3209], [RFC3270], [RFC3272], [RFC3473], [RFC4875], [RFC5440], and [RFC8306]. How does DetNet guarantee "zero congestion loss and bounded latency and jitter" while operating on an MPLS network? The more I read, the more it seems to me that the examples provided support the concept that the same results can be obtained in an MPLS network without DetNet. [I am out of my depth and feel like I'm probably missing something, which is why I am not balloting DISCUSS on this point. However, I believe that the issue of specific guidance remains.] (2) Figure 4 shows the encapsulation of a DetNet App-Flow, which includes several labels + CW... Figure 6 shows an even bigger stack (also with several F-Labels and the new A-Label, S-Label, CWs...). What are the considerations related to the nodes' ability to support the label stack needed for DetNet operation and aggregation? What is the relationship between the maximum number of labels supported in the network and the ability to push the required label stack onto the packet? [Take a look at rfc8491 for a sample definition related to the Max SID Depth (MSD).] (3) §4.2.1 A 0 bit sequence number field length indicates that there is no DetNet sequence number used for the flow. When the length is zero, the sequence number field MUST be set to zero (0) on all packets sent for the flow. ...and ignored by the receiver?? There are several other "MUST be set to zero" phrases in the text. (4) §4.2.2 An S-Label will normally be at the bottom of the label stack once the last F-Label is removed, immediately preceding the d-CW. To support service sub-layer level OAM, an OAM Associated Channel Header (ACH) [RFC4385] together with a Generic Associated Channel Label (GAL) [RFC5586] MAY be used in place of a d-CW. §4.2.1 says that the d-CW "MUST be present in all DetNet packets containing app-flow data". What are the cases when the d-CW may be substituted by the ACH/GAL combination? The PEF/POF functions are specified in terms of the d-CW -- is that functionality lost if the ACH/GAL are used instead? (5) §4.2.2 "...zero or more F-Labels and optionally, the incoming interface, proceeding the S-Label MUST be used together with the S-Label to uniquely identify the DetNet service associated with a received packet. The incoming interface MAY also be used together with any present F-Label(s) and the S-Label to uniquely identify an incoming DetNet service..." These two phrases seem to say the same thing, but with different normative expectations. (6) §184.108.40.206: The interoperable action is not "MUST check", or "MUST ensure", but "MUST discard". OLD> After a DetNet service is identified for a received DetNet MPLS packet, as described above, an implementation MUST check if PEF is configured for that DetNet service. When configured, the implementation MUST track the sequence number contained in received d-CWs and MUST ensure that duplicate (replicated) instances of a particular sequence number are discarded. NEW> After a DetNet service is identified for a received DetNet MPLS packet, as described above, if PEF is configured for that DetNet service, duplicate (replicated) instances of a particular sequence number MUST be discarded. (7) §220.127.116.11: "MAY wish" is not an action that can be enforced OLD> Note that an implementation MAY wish to constrain the maximum number of sequence numbers that are tracked, on platform-wide or per flow basis. Some implementations MAY support the provisioning of the maximum number of sequence numbers that are tracked on either a platform-wide or per flow basis. NEW> An implementation MAY constrain the maximum number of sequence numbers that are tracked on either a platform-wide or per flow basis. (8) §18.104.22.168: The interoperable action is not "MUST check", or "MUST ensure", but "MUST process in order". OLD> After a DetNet service is identified for a received DetNet MPLS packet, as described above, an implementation MUST check if POF is configured for that DetNet service. When configured, the implementation MUST track the sequence number contained in received d-CWs and MUST ensure that packets are processed in the order indicated in the received d-CW sequence number field, which may not be in the order the packets are received. NEW> After a DetNet service is identified for a received DetNet MPLS packet, as described above, if POF is configured for that DetNet service, packets MUST be processed in the order indicated in the received d-CW sequence number field, which may not be in the order the packets are received. (9) §22.214.171.124: "MAY wish" is not an action that can be enforced OLD> Note that an implementation MAY wish to constrain the maximum number of out of order packets that can be processed, on platform-wide or per flow basis. Some implementations MAY support the provisioning of this number on either a platform-wide or per flow basis. The number of out of order packets that can be processed also impacts the latency of a flow. NEW> An implementation MAY constrain the maximum number of out of order packets that can be processed, on either a platform-wide or per flow basis. The number of out of order packets that can be processed also impacts the latency of a flow. (10) [nits] s/MAY required/MAY require s/otherwise received/otherwise receive s/a service parameters that dictates/service parameters that dictate s/that maybe used/that may be used
Martin Vigoureux No Objection
Comment (2020-09-09 for -11)
Hi, thank you for this document. I have few COMMENTs. I support Magnus' DISCUSS. I think implementing replication/duplicate elimination is non trivial at layer "2.5" and implementers would, I guess, benefit from extra clarifications. I'm not an expert in that field but I sense some challenges around the size of the tables needed to keep track of the received S/N, associated expiry timers, race conditions, ... A DetNet control word (d-CW) conforms to the Generic PW MPLS Control Word (PWMCW) defined in [RFC4385]. This kind of suggests that there is a *single* format for d-CW which is: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0| Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ but then As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW is 0x0 the payload following the d-CW is normal user data. However, when the first nibble of the d-CW is 0X1, the payload that follows the d-CW is an OAM payload with the OAM type indicated by the value in the d-CW Channel Type field. suggests that, for OAM, d-CW format becomes 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1|Version| Reserved | Channel Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Is that a d-CW with a different format or is that something else (an OAM ACH) as this text suggests: To support service sub-layer level OAM, an OAM Associated Channel Header (ACH) [RFC4385] together with a Generic Associated Channel Label (GAL) [RFC5586] MAY be used in place of a d-CW. The sequence number length MUST be provisioned on a per Detnet service basis via configuration, i.e., via the controller plane described in [I-D.ietf-detnet-data-plane-framework]. Should there be a requirement that the sequence number length be the same on both ends of the service? In some PREOF topologies, the node performing replication will be sending to multiple nodes performing PEF or POF, and may need to send different S-Label values for the different member flows for the same DetNet service. When replication is configured, the same app-flow data will be sent over multiple outgoing DetNet member flows using forwarding sub-layer LSPs. An S-Label value MUST be configured per outgoing member flow. It's not clear to me whether that last sentence means "a *different* S-Label value per ..." but I kind of sense a conflict between these two paragraphs. Am I wrong? zero or more F-Labels and optionally, the incoming interface, proceeding the S-Label MUST be used Maybe I'm not reading this correctly, but if using the incoming interface is optional, and using zero F-labels is allowed, then it seems to me that this sentence allows for "nothing MUST be used" Also, right after you say: The incoming interface MAY also be used ... So I wonder if mentioning the incoming interface in the previous sentence is necessary. The edge and relay node internal procedures related to PREOF are implementation specific. The order of a packet elimination or replication is out of scope in this specification. For example, a relay node that performs both PEF and PRF first performs PEF on incoming packets to create a compound flow. It then performs PRF and copies the app-flow data and the d-CW into packets for each outgoing DetNet member flow. Again, I kind of sense a conflict between these two paragraphs (first says order is OOS, second gives an order). Am I wrong? RFC7322 limits the number of authors listed on the front page of a draft to a maximum of 5. The editor wishes to thank and acknowledge the follow author for contributing text to this draft. I see 6 authors on front page. I don't have any specific opinion on that but there seems to be a mismatch between that text and the current number of authors on front page. s/MAY required that PEF/MAY require that PEF/ s/F-Labels are supported the DetNet forwarding sub-layer./F-Labels are supported by the DetNet forwarding sub-layer./ s/Valid values included/Valid values include/ s/traffic paramters/traffic parameters/ s/the follow author/the following author/
Éric Vyncke No Objection
(Magnus Westerlund) (was Discuss) No Objection
Thanks for the discussion and resolution of the issues I raised.
Robert Wilton (was Discuss) No Objection
Comment (2020-09-10 for -12)
Discuss resolved. Previous discuss comment: Hi, Thank you for this document. Hopefully a trivial discuss to resolve ... 4.2.1. DetNet Control Word and the DetNet Sequence Number Does this section need to specify the initial value for the sequence number for a new flow? Regards, Rob