MPLS Network Action (MNA) Sub-Stack Specification including In-Stack Network Actions and Data
draft-ietf-mpls-mna-hdr-21

Thank you to the authors and the WG for their work on this document.

This document introduces a significant change to the decades-old MPLS architecture; by incorporating concepts similar to IPv6 extension headers (HBH, DOH, RH) while allowing for arbitrary insertion/removal (push/pop) and "selective hop" processing that exists in MPLS. It also attempts to address backward compatibility through label entry encoding, identifying mutable/immutable bits for load-balancing purposes, and using multiple copies of options to account for Readable Label Depth (RLD) limitations. The challenges associated with such options are well-known; even within a router-centric MPLS environment, this adds considerable complexity to an architecture that is loved for being as simple and fast.

I recognize the significant effort, both technical and in consensus-building, that has gone into this work. Out of respect for the WG consensus, I am balloting ABSTAIN despite my questions, doubts, and concerns regarding the approach and choices made.

Please find my detailed review comments below. I hope they are helpful, should the authors and the WG wish, to improve the document.

The comments are inline in the idnits output of v18 of the document. Please look for the tag <EoRv18> at the end to ensure you have received the full review.


23	   the MPLS packet or perform user-defined operations.  This document
24	   addresses the requirements for In-stack network actions and In-stack
25	   data found in RFC 9613.

<minor> Suggest to remove the last sentence of the abstract. I am suggesting
some editorial changes so that RFC9613 can become an informative reference.


127	   The solution specified in this document addresses the requirements
128	   for In-stack network action and In-stack data (ISD) found in
129	   [RFC9613].

<minor> Suggest:
The requirements for In-stack network action and In-stack data (ISD) are
described in [RFC9613].

139	2.1.  Requirements Language

141	   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
142	   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
143	   document are to be interpreted as described in [RFC2119] [RFC8174]
144	   when, and only when, they appear in all capitals, as shown here.

<minor> Please fix the BCP14 template

157	      +--------------+-----------------------------+---------------+
158	      | BOS          | Bottom Of Stack             | [RFC3032]     |

<minor> The BOS acronym is not defined in RFC3032 but that field is defined
there.

170	      +--------------+-----------------------------+---------------+
171	      | LSE          | Label Stack Entry           | [RFC3032]     |

<minor> Same as previous comment - LSE acronym is not defined in RFC3032.

270	   *  IHS (2 bits): The scope of all the network actions in this sub-
271	      stack.  See Section 5.3.

<minor> I see the term "sub-stack" used without the "MNA" or "NA" prefix in a 
few places in the document. I would suggest using the NAS acronym instead of
"sub-stack" in such places or prefix MNA before the "sub-stack" for clarity.
There is frequent use of the term "MNA sub-stack" and I wonder why all of them
could not be replaced by "NAS" for consistency?

323	   NOTE: A Format A and a Format B LSE MUST be present when a Format C
324	   LSE is carried in the NAS.

<minor> Please remove "NOTE:" since that is generally used in an informative
manner but this is normative text. Same applies in a few other places in the
document.

328	   LSE Format D is used to encode additional data that did not fit in
329	   the LSE with the preceding opcode.

<major> The preceding opcode here imply LSE Format B or LSE Format C with its
opcode, but it could as well be another LSE Format D that does not carry an
opcode? Can this be clarified? Perhaps ... that did not fit in the previous
LSE Formats B, C, or D?

355	5.  The MNA Sub-Stack

357	   The MNA Sub-Stack begins with a Format A LSE (Section 4.1).  The

<major> Perhaps .. MUST begin with ... ?

358	   label value of the LSE contains the MNA bSPL (value TBA) to indicate
359	   the presence of the MNA Sub-Stack.

<minor> There is some very good text explaining the design of Format A that is
in the section 11. Perhaps consider moving it in here and then provide a pointer
to section 11 for the reader to understand how it helps backwards compatibility?

361	   The TC and TTL values of the Format A LSE retain their semantics as
362	   defined in [RFC3032] and [RFC5462].  The TTL and TC values in the
363	   Format A LSE are copied from the forwarding label at the top of the

<major> copied by who? Can you clarify that this is to be done by the node
pushing that NAS? Please also clarify what is meant by "top of the label stack".
If the node pushing a NAS were to also push another forwarding LSE on top of
the NAS, is the TTL/TC copied from that LSE or the LSE that was at the top of
the stack when the packet arrived at that node? Also, please consider moving
this discussion of TTL/TC into the appropriate sub-section(s) of section 9?

364	   label stack.  The penultimate node on the path copies the TTL and TC

<major> You mean the "penultimate node" doing PHP? Or perhaps a node doing pop
in general? Also, consider moving into section 9?

365	   values from the preceding LSE to the next LSE on the label stack,
366	   overwriting the TTL and TC values of the next LSE, as specified in
367	   Section 3.5 of [RFC3443] and Section 2.6.3 of [RFC3270] in the
368	   Uniform Mode LSPs.  If the node performing this copy is not aware of
369	   MNA, this could overwrite the values in the first LSE of the MNA sub-
370	   stack.

<major> The first LSE if the NAS is not clear. I am assuming it is a Format A
LSE? And if so, I fail to see the issue. In other words, I am not understanding
the intent of this text. 

380	   A NAS MAY contain more Format C (Section 4.3) and Format D
381	   (Section 4.4) LSEs, up to the length encoded in the NASL value.  All
382	   Format D LSEs MUST follow a Format C or B LSE and be included in that
383	   LSE's NAL value.

<major> What would be good here is some text describing the thought process behind
these 3 types (B, C & D) of formats and how they can be used in combination of each
other. Then put a reference to the examples so the reader can validate understanding
based on those illustrations. One aspect that is not clear is that the AD for an
opcode can span across more than one Format D.

398	   The label value (most significant 20 bits) in one or more consecutive
399	   LSEs is commonly used for load balancing data flows in an ECMP
400	   environment.  Modifying the first 20 bits in an LSE might alter a
401	   packet's path and result in out-of-order delivery of packets
402	   belonging to a given flow.  To maintain the stability of deployed
403	   services in ECMP environments that rely on label value information
404	   for load-balancing, care must be taken when encoding network action
405	   data in the given LSE.  If the network action data may differ among
406	   packets in the same flow or change during forwarding across the MPLS
407	   network, it MUST NOT be placed in the most significant 20 bits of a
408	   Format B LSE (Section 4.2), a Format C LSE (Section 4.3), or a Format
409	   D LSE (Section 4.4).  Thus, the available bits for data that can
410	   change by a transit node or differ among packets of the same flow in
411	   Format A and Format B LSEs are 0, Format C LSE is 7 (bits 20-22 and
412	   25-28) and Format D LSE is 11 (bits 20-22 and 24-31).

<major> This aspect is not just for the AD but also for the construction of
the NAS (along with its opcodes). If all the packets of the flow are not subject
to the same NAS imposition and encoding, then we have a similar problem. I am
thinking of some OAM operation where the network action is not to be performed
on all packets but some (a sample) of them. Another possibly more important
aspect is why not just use the post-stack solution/option in the case of
mutable information or where things are changing within a flow? Isn't that 
simpler as opposed to this complicated working with bits?

464	   The egress node is included in the HBH scope.  This implies that the
465	   penultimate node MUST NOT remove a HBH NAS.  The egress node MAY
466	   receive a NAS at the top of the label stack as discussed in
467	   Section 10.

<major> s/MAY/may - also I don't see the connect to section 10. Then again, why
just the egress node, even an intermediate node could receive a NAS at the top
of the label stack.

469	   An I2E scope NAS, if present, MUST be encoded after any HBH or
470	   Select-scope NASes.  This makes it easier for the transit nodes to
471	   process a NAS with HBH or Select scope.

<major> Is the packet dropped if this "MUST" is violated?


496	   When a packet with an unknown Network Action is dropped, the node
497	   SHOULD maintain a local counter for this event, and MAY send a rate-
498	   limited notification to the operator.

<major> What is an "unknown Network Action" - is it an unknown opcode, the 
reserved IHS value, anything else? Can this be specified?

510	6.  Special Opcodes

512	   Below are the special opcodes defined to build a basic In-stack MNA
513	   solution and has been assigned through IANA registry (Section 15.4).
514	   In future, additional special opcodes can be defined and their code-
515	   points assigned from the "Network Action Opcodes" IANA registry
516	   (Section 15.4).

<major> I don't see the registry differentiating between "special" and "usual"
opcodes. Can we say "Initial" opcodes? But then I don't know what "special"
opcodes in the future indicate ...

518	6.1.  bSPL Protection

520	   Opcode: 0
521	   Purpose: Legacy implementations may scan the label stack looking for
522	   bSPL values.  As long as the opcode field is non-zero, an LSE cannot
523	   be misinterpreted as containing a bSPL.  Opcode 0 is therefore
524	   reserved and is not used.

<minor> Can we put "Reserved (not to be used)" in the IANA registry entry too?

534	   LSE Formats: B, C, D

536	   Data: The data field carries Network Action Indicators, which should
537	   be evaluated from the most significant bit to the least significant
538	   bit.  If this opcode is used with LSE Format B only, then up to 13
539	   flags may be carried.  If this opcode is used with LSE Format C only,
540	   then up to 20 flags may be carried.  Format D LSEs can be used with
541	   format C LSEs to encode more than 20 flags.  Flags are assigned from
542	   the "Network Action Flags Without Ancillary Data" registry
543	   (Section 15.3).  If flags need to be evaluated in a different order,
544	   multiple LSEs using this opcode may be used to specify the requested
545	   order.  The Flag-Based Network Action Indicators MUST follow the
546	   procedure for data specified in Section 5.2.

<major> As I understand, the bit position depends on the sequence of the
LSE entries with opcode 1. It is not clear that all LSEs in the NAS with opcode
1 need to be contiguous - one of the examples shows when multiple LSEs with
opcode 1 can be present in the same NAS - please clarify in this normative text.
If the LSE Format B is used for another opcode and it is followed by LSE Format 
C with opcode 1, then the bit 0 would be the MSB of that Format C LSE entry's AD.
Can we have B with opcode 1, D, and then C with opcode 1? I see too many 
permutations and combinations that can become very complex unless there is a 
very simple and prescriptive method for encoding of this opcode?


550	6.3.  No-Operation Opcode

552	   Opcode: 2

554	   Purpose: This opcode is reserved to indicate that this opcode does
555	   not perform any Network Action and MUST be skipped.

<major> Then why have it in the first place? What is the real purpose behind 
this? E.g., in IPv6 options headers we have PAD options which are no-op but 
they serve a purpose of alignment. Is it because it is mandatory to have an 
LSE Format B after A and then what if the action needed actually needs LSE 
Format C but we need to put a Format B with no-op opcode to follow the rules? 
This may not be super obvious to a reader, if this was the motivation.
Suggest to explain why/when it is used and then perhaps put a reference to
one of the examples where this is used.

561	6.4.  Extension Opcode

563	   Opcode: 127
564	   Purpose: This opcode is reserved to extend the current opcode range
565	   beyond 127 in future.  If this opcode is not supported, then the
566	   packet with the opcode 127 MUST be dropped regardless of the setting
567	   of the U bit.  Use of this opcode is outside the scope of this
568	   document.

<major> How can one support this opcode when it is unspecified? Why not make 
this "Reserved (for future use)" and indicate that packets with this set are 
to be dropped?

570	7.  NAS placement in the Label Stack

572	   The node adding a NAS to the label stack places a copy of the NAS
573	   where the relevant nodes can read it.  Each downstream node along the
574	   path has a Readable Label Depth (RLD).  If the NAS is to be processed
575	   by a downstream MNA-capable node, then the entire NAS MUST be placed
576	   so that it is within RLD by the time the packet reaches the
577	   downstream MNA-capable node.

579	   If the label stack is deep, several copies of the NAS may need to be
580	   encoded in the label stack.

<major> Consider that the action is to set certain information in the HBH header 
as it traverses the LSP. How is this information conveyed from the top most NAS to
its copies further below? Or, can such actions be only placed in a single copy
in the label stack?

587	   A NAS MUST NOT appear at the top of the stack after popping the
588	   forwarding label on an MNA-incapable node on the path.

<major> The above is not clear - who is popping the label here? Is it the
node before the MNA-incapable node - in this case there is an issue? Or, is it
the MNA-incapable node - in which case there is no issue unless the node
after it is also MNA-incapable and it cannot do anything about it.


598	   forwarding label and before the next forwarding label.  It could be
599	   inserted before or after a HBH NAS.  Note that the case of a NAS with
600	   Select scope with MPLS label swap operation (for example, with RSVP
601	   Traffic Engineering LSPs) is for future study.

<major> Swap is a basic MPLS label operation. I am concerned by how the
processing of a NAS is unspecified in that case. Perhaps Select scope doesn't
make sense and is not processed during a swap? 

606	   Transit, non-penultimate nodes that pop a forwarding label and expose
607	   a copy of a NAS MUST remove it.

<major> The use of the term transit and non-penultimate node is not very clear
here. Consider a SR-MPLS label stack. There is the headend and the endpoint.
However, there are nodes in the middle that perform swap or pop (including for
PHP) - here I am assuming only prefix and adj SIDs for simplicity. So, is
this statement only about the case where multiple copies of a NAS are present
in the label stack? Because the next paragraph seems to contradict. Also, how
does the node know if the multiple NASes in the label stack are copies or
distinct NASes without performing an expensive compare operation?

617	   The node that receives the NAS at the top of the label stack MUST
618	   process and remove it.

<major> Is it required for each node to understand the MNA capability of all
of it neighbors such that it can ensure that it doesn't forward packets to a
non-MNA capable neighbor with the NAS exposed as the topmost label?


620	7.1.  Actions when Pushing Labels

622	   An MNA-capable node may need to push additional labels as well as
623	   push new network actions onto a received packet.

625	   While pushing additional labels on to the label stack of the received
626	   packet, the MNA-capable node MUST verify that the entire top-most NAS
627	   with HBH scope is still within the RLD of the downstream MNA-capable
628	   nodes.  If required, the MNA-capable node MAY create a copy of the

<major> The HBH NAS was in the incoming packet. The node is pushing its own
label stack (say SR BSID) and the entity computing that label stack is doing
what it needs to do based on the RLD of nodes along the path. However, in the
forwarding path, there is no clue whether incoming packets will have HBH NAS
- it seems incredibly complex to expect the node to figure out the RLD capabilities
along the packet path in the fast path. How is that MUST supposed to work?

629	   top-most NAS with HBH scope and insert it within the RLD of the
630	   downstream MNA-capable nodes on the label stack.

<major> What about the MTU implications of doing such copy and insert operations?


639	   The new network actions added MUST NOT conflict with the network
640	   actions in the received NAS with HBH scope.  The mechanism to resolve
641	   such conflicts depend on the network actions and can be based on
642	   local policy.  The MNA-capable node that pushes entries MUST
643	   understand any network actions which it is pushing which may result
644	   in a conflict, and MUST resolve any conflicts between new and
645	   received network actions.  In the usual case of a conflict of
646	   duplicating a network action, the definition of a network action MUST
647	   give guidance on confict resolution.

<major> Are all of these MUSTs even possible to confirm to given that no actions
are currently defined, no conflicts are currently defined and, therefore, no
way to resolve conflicts are defined. All of this seems very aspirational and so
perhaps consider removing all the normative text and rephrase it as a guidance?

669	   *  In the case of MPLS Segment Routing (SR-MPLS), as well as the,
670	      RLD, the path computation system needs to know the MSD [RFC8664]
671	      that can be imposed at the ingress node of a given SR path.  This

<minor> Perhaps:
In the case of MPLS Segment Routing (SR-MPLS), the path computation system needs 
to know the MSD [RFC8664] that can be imposed at the ingress node of a given SR 
path, as well as the, RLD of the nodes along the path.

672	      ensures that the label stack depth of a computed path does not
673	      exceed the maximum number of labels (i.e., MSD) the node is
674	      capable of imposing and the maximum number of labels that can be
675	      read by the MNA-processing nodes in the path.  The MSD MUST
676	      include the MNA Sub-Stacks that will be added.

<major> Perhaps:
The MSD and RLD calculations MUST factor in both the forwarding LSEs and the NAS.

678	9.  Processing the Network Action Sub-Stack

680	   This section defines the specific responsibilities for nodes along an
681	   LSP [RFC3031].

683	9.1.  Encapsulating Node Responsibilities

<major> What is the definition of "Encapsulating Node"? I assume this is the node
that is receiving non-MPLS traffic which it encapsulates in a MPLS label stack?
If so, please clarify.

692	   If there is an existing label stack, the encapsulating node MUST NOT
693	   modify the first 20 bits of any LSE in the label stack when the ECMP
694	   technique in the network is using the hashing of the labels on the
695	   label stack.

<major> How would the encapsulating node know about that? Is that something
to be figured out along the path of a specific LSP or something network-wide?

697	   If the encapsulating node is also a transit node, then it MUST also
698	   follow the rules set out in Section 9.2.

<major> If my assumption of the definition of an "encapsulating node" is correct,
then I am not able to follow how the above sentence can be true.

700	9.2.  Transit Node Responsibilities

702	   The transit node is the node that processes a NAS in the Label stack
703	   but does not push any new NAS.

<major> So, this is in fact a "MNA Transit Node" and not a "MPLS Transit Node"? 
Please crosscheck such terms and see if they need a qualifier to distinguish from
"regular/classic" MPLS.

705	   The transit node MUST follow the procedure for data specified in
706	   Section 5.2.

<major> There is are no procedures for data processing by MNA Transit nodes in
section 5.2. Am I missing something?

724	9.4.  Egress Node Responsibilities

726	   The egress node MUST remove any NAS it receives.

<major> Is it an egress node or the decapsulating node? The later would match
with the encapsulating node. Also, it needs to process the NAS and a little
differently than the MNA transit node since it also needs to deal with the I2E
IHS ones?

728	10.  Network Action Indicator Opcode Definition

730	   The following information MUST be defined for a new Network Action
731	   Indicator opcode request in the document that specifies the Network
732	   Action.

<minor> This is the first time that the term "NAI opcode" is used. Perhaps you
want to introduce it in section 5.1 which is about opcodes?

737	   *  Format: The definition of the new Network Action MUST specify the
738	      LSE Formats.  The opcode can define Network Action in Format B or
739	      C or both Format B and C.  Both Format B and C LSEs MAY optionally
740	      carry Format D LSEs.

<major> I am not sure that I understand the "MAY" here. For a given opcode, it is
ok to have variable length such that it might fit into simply Format B or C in
some cases, but then on need basis the size of data can be increased by arbitrary
number of Ds added to it - am I correct? The question is, is it implicit that 
all opcodes can get extended by D or does the opcode need to specify if it can 
be extended by D (and how many Ds) in its specification? 

742	   *  Scope: The definition of the new Network Action MUST specify at
743	      least one scope (I2E, HBH, Select) for the Network Action, and MAY
744	      specify more than one scope.

<major> How about whether the given opcode can be present more than one time
in the same NAS or more than one time across different NAS with the same IHS?

746	   *  Ancillary Data: The definition of the new Network Action MUST
747	      specify the quantity, syntax, and semantics of any associated
748	      Ancillary Data.  The Ancillary Data MAY be variable length, but
749	      the length MUST be computable based on the data present in the
750	      NAS.

<major> which data present in the NAS? perhaps you mean the NAL ?

765	11.  Backward Compatibility

767	   This section discusses interactions between MNA-capable and MNA-
768	   incapable nodes.

770	   An MNA-encapsulating node MUST ensure that the MPLS Network Action
771	   Sub-Stack indicator is not at the top of the MPLS label stack when
772	   the packet arrives at an MNA-incapable node.  If such a packet did

<major> How is this MUST going to be honored when the document does not specify
the means of doing so?

859	   *  The semantics of a network action are unbounded and may be
860	      insecure.  A network action could be defined that made arbitrary
861	      changes to the memory of the forwarding router, which could then
862	      be used by the encapsulating node to compromise every MNA-capable
863	      router in the network.  The IETF needs to ensure that only secure
864	      network actions are defined.

<major> How is the IETF going to be able to do that when both the flags and the
opcodes have allocations for private and experimental use where allocation does
not require IETF review? This contradicts the next bullet.

872	   *  The MPLS domain border nodes MUST ensure that the MPLS packets
873	      with MNA from any domain with a different administrative control
874	      can be filtered to prevent entering the provider MPLS domain.  The
875	      filtering capability MAY be enabled on a per network action basis
876	      and it can be based on a local policy.  The filtering capability
877	      MUST be implemented on those nodes before deploying MNA in the
878	      provider MPLS domain.  The RLD on the filtering node MUST be
879	      higher than the RLD on all other nodes in the provider MPLS
880	      domain.

<major> The use of "MAY" in what kind of filtering is required and how to do that
coupled with use of "MUST" do filtering is strange. Why not define very specifically
and in detailed terms what type of filtering MUST be implemented?

948	   Bit Position refers to the position relative to the most significant
949	   bit in LSE Format B or C Data fields and any subsequent Format D
950	   LSEs.  Bit Position 0 is the most significant bit in an LSE Format B
951	   or C Data field.  Bit Position 20 is the most significant bit in the
952	   first LSE Format D Data field.  There are 20 bits available in LSE
953	   Format C and 30 bits available in LSE Format D.  There are at most 14
954	   Format D LSEs per opcode (due to NASL limit of 15 and Format D
955	   requires Format C LSE), so there are at most 20 + 14 * 30 = 440 bit
956	   positions.  The Bit Position is an integer with value 0-439.

<major> The above paragraph is normative text and does not belong in the IANA
section. Please move it into section 6.2. The IANA is only going to deal with
the assignments of bits per their policy.



1018	16.  Examples

<minor> Please move examples into an Appendix section.


1067	        Figure 6: Network Action Flags without AD using LSE Format D

1069	   In this example, the NAS contains a Format B LSE with No-Operation
1070	   Opcode value 2.  The next LSE uses Format C, but the Network Action
1071	   Flag is not in a bit position contained within the Format C LSE, so a
1072	   single Format D LSE has been added to the NAS to carry the flag.

<major> This example is confusing. Can you indicate which specific bit position
is required to be set? That will explain why the no-op opcode is used with
the LSE B format and why the flags didn't start from that LSE B itself followed
by a LSE C or D but not both. 


1154	          Figure 9: Network Action With Additional Ancillary Data

1156	   In this example, opcode 9 requires more than one LSE's worth of
1157	   Ancillary Data, so a Format D LSE is added.

<major> Sure, but why is LSE B used with no-op opcode? The answer needs to
be that there was need for a specific size of AD that gets encoded more
efficiently by the scheme shown.

1257	   [RFC9613]  Bocci, M., Ed., Bryant, S., and J. Drake, "Requirements
1258	              for Solutions that Support MPLS Network Actions (MNAs)",
1259	              RFC 9613, DOI 10.17487/RFC9613, August 2024,
1260	              <https://www.rfc-editor.org/info/rfc9613>.

<major> I am wondering why this isn't an informative reference. 

<EoRv18>