Entropy Label for Source Packet Routing in Networking (SPRING) Tunnels
draft-ietf-mpls-spring-entropy-label-12

Thank you for writing this - it provides useful functionality. 

Please see Joe Clarke's OpsDir review here: https://datatracker.ietf.org/doc/review-ietf-mpls-spring-entropy-label-11-opsdir-lc-clarke-2018-06-25/  - it mirrors comments made by a number of ADs, and also contains a significnat number of nits (which would make the document better / more readable).

I had a question:
Section 4.  Entropy Readable Label Depth
" The Entropy Readable Label Depth (ERLD) is defined as the number of
   labels a router can both:
   a.  Read in an MPLS packet received on its incoming interface(s)
       (starting from the top of the stack).
   b.  Use in its load-balancing function."
...
"In a distributed switching architecture, each linecard may have a different capability in terms of ERLD."

In many cases, a device may have a different readable label depth in hardware / fastpath than it does by punting the packet to the CPU / control plane. 
Perhaps the ERLD should be defined as 'Use in its load-balancing function in the dataplace / fast-path" (or something, this will need some wording).

I'd also like to say that I like Section 10 (Options considered) - sections like this make a document much more satisfying (otherwise one has niggling questions like "What didn't they do single ELs at the bottom of the stack?!") - thank you for including it.

I also have some nits:
Section 1.  Introduction
"The hashing technique is required to perfom a per-flow load-balancing and thus prevent packet disordering. "
1: Perform is a type
2: While 'disordering' explains it well, 'reordering' is a much more common term, and will (I think) cause less confusion.

"The MPLS architecture brings some challenges on the load-balancing as an LSR (Label Switch Router) should be able to look at header fields that are beyond the MPLS label stack."
"on the load-balancing" doesn't really parse. Perhaps: "... brings some load-balancing challenges, as..."?

Section 2.  Abbreviations and Terminology
SRGB is not defined on first use, nor it is in the Terminology section. Also, sorting this alphabetically would be appreciated.

Small process nit:

RFC 7855, which is informational, is listed as a normative reference in this
document, which is PS. I don't see RFC 7855 on the downref registry (
https://datatracker.ietf.org/doc/downref/ ), and this document didn't mention
the downref in its IETF LC announcement (
https://mailarchive.ietf.org/arch/msg/ietf-announce/ROPiwT5P_oM5PYBRoVeY7L_vsFQ
)

RFC 8067 §2 lets the IESG process the document anyway, and I think we should,
but I want to flag this for explicit consideration. This also interacts with
Benjamin's observation that perhaps this document should go back to being
informational.

---------------------------------------------------------------------------

§7.2.3:

>  All
>  routers have an ERLD of 10, expect P1 and P2 which have an ERLD of 4.

Nit: "except"

Is there a justification for this document having more than five authors, rather than one or two editors?

I have a similar opinion as others regarding the status of this document: informational vs proposed standard.  While I don't disagree with the intention of the WG to put this work on the standards track, I think that, as a standard, the document still needs some work:  even when using rfc2119 language, the specification is not as precise as I would hope.  A couple of examples from §7:

(a) "This section describes some considerations that could be taken into account when placing ELI/ELs.  This list of criteria is not considered as exhaustive and an implementation MAY take into account additional criteria or tie-breakers that are not documented here."  Even if specific language is used later on, the preface says that the considerations don't have to be taken into account ("could be"), and that there is other criteria (not in this document) that could be used.  How are multiple implementations to consistently interoperate?

(b) "An implementation SHOULD try to maximize the load-balancing where multiple ECMP paths are available and minimize the number of EL/ELIs that need to be inserted."  How does an implementation try?  How would an implementation maximize load balancing?  I don't remember seeing anything like that in rfc6790.


On the process side, the IETF LC should be redone simply because it was done assuming an informational status (regardless of the downref).

[I am not balloting DISCUSS because I'm sure the responsible AD will do the right thing.]

Substantive Comments:

- Adam beat me to the comment about the normative downref to RFC 7855. I agree with his point.

§1, 2nd paragraph: "The MPLS architecture brings some challenges on the load-balancing as
   an LSR (Label Switch Router) should be able to look at header fields
   that are beyond the MPLS label stack."

I'm confused by that statement.  Should this say " ... should _not_ be able ... " or "needs to be able"?

§14: Please elaborate. This adds a new data element and mechanism; please expLain why that doesn't affect the security considerations. (I'm willing to believe that it doesn't, but the section should contain an actual argument for that point.)

Editorial Comments:

- General: The term "Entropy Label" is used in a singular sense on several occasions. That usage needs an article, e.g. "The Entropy Label ..."

I'm balloting No Objection, but would still like to see some discussion on the first point.

I see the WG decided to change from targetting Informational to Proposed
Standard as triggered by the RtgDir review.  It's still unclear to me that
this is the best choice, though.  Perhaps a better test than "does this use
RFC 2119 language to normatively specify behavior?" is "can we concretely
test if two independent implementations interoperate?", as would be needed
to move to full Internet Standard.  It's unclear that there is any
cooperative interoperability needed here, as opposed to unilateral choices
made by one peer in the context of a broader framework.  (Perhaps a new
thing here is ther presence of multiple ELs in the stack; I don't know.)
This makes me wonder if the contents would be better recast as a BCP (being
the best practice for assigning multiple entropy labels for SPRING
tunnels).

I guess this ship has sailed and probably I should keep quiet about it (and
yet I'm still talking), but if I understand correctly the entropy label is
not exactly providing fresh entropy (in the form of random input) for
load-balancing, but rather making existing entropy more easily accessible.
My worldview is not represented in RFC 6790 at all, though, so the desire
to maintain consistency over time would contraindicate any big changes to
this document.  Perhaps in Section 1 "provide entropy" could be "make
entropy accessible", or similar, but I suspect I'm in the rough here and
the right answer is actually to do nothing.

There's a lot of "how to advertise this value is outside the scope of this
document", but the only guidance or pointer to external work seems to be in
Section 7.2.1; maybe there could be more internal references to that
section when the "out of scope" remarks appear.  Something like a
parenthetical, "(see Section 7.2.1 for potential approaches)", perhaps.

Some section-by-section comments follow (mostly nits).

Section 1

   [...] Using the
   entropy label in the hash keys reduces the need of a deep packet
   inspection in the LSR while keeping a good level of entropy in the
   load balancing.

s/need of a/need for/

Section 2

Does LSR expand to "Label Switch Router" (as is done in this text) or
"Label Switching Router" (what's in the RFC Editor's word list)?

Section 3

Expand SRGB on first use.

Section 4

It's a little jarring to talk of "position 1 (top)" which is (IIUC) the
bottommost label in the Figure 2 depiction.  (I'm sure readers will know
the intent, of course.)

Section 5

   When an external controller is used to program a label stack on a
   particular node, this node MAY advertise its MSD value or a subset of
   its MSD value to the controller.  How this advertisement is done is
   outside the scope of this document.  As the controller does not have
   the knowledge of the entire label stack to be pushed by the node, the
   node may advertise an MSD value which is lower than its actual limit.

A node's MSD is a scalar value, not a set, so "a subset of its MSD value"
makes no sense; presumably it is advertising a value smaller than its 
actual MSD value in this case, as is explicitly mentioned in the last
quoted sentence.  Given the apparent duplication, maybe this text could be
trimmed down, something like:

   An external controller can be used to program a label stack on a
   particular node, which requires the node to indicate to the controller
   what MSD value to be used for that node.  How this advertisement is done is
   outside the scope of this document.  As the controller does not have
   the knowledge of the entire label stack to be pushed by the node, the
   node may advertise an MSD value which is lower than its actual limit.


   In the figure 3, an IP packet comes in the MPLS network at PE1.  All

nit: "comes in to" or maybe just "enters"

Not being a routing person, I always have to look up whether a higher
metric means to give that link more or less traffic, but presumably this is
only a problem for me...

Section 6

   Each LSP associated with a binding SID has its own entropy
   label capability.

I'm not sure I understand what this sentence means.  Is it that I can
assign ELs differently for different LSPs, or that the ERLD at each point
step on the path is a function of the LSP, or something else?

Section 7.1.1

IMPORTANT: I'm confused here; we talk about PE1 having a MSD that lets it
introduce two ELI/EL pairs, but the proposed stack only includes one such
pair, despite the text talking about the need to insert a new ELI/EL next
to P6 for its load balancing.

Section 7.2.2.1

Please expand SPT.

Section 7.2.4

   [...] As example, a service provider

nit: As an example

A nit - ELI stands for EL Indicator, not Identifier.

Minor high level comment:
The use of normative and non normative MAY is a bit confusing in this doc. I think the usage is in most cases correct, however, if you could re-phase some of the sentences where your use a lower case "may" to just avoid the word "may", I think it would even make the text easier readable in quite some cases. Any maybe also double-check on the use of MAY (e.g. there are some "an implementation may" and "an implementation MAY" where I'm not certain about the use of upper case MAY).

One mostly editorial request:
sec 1:
"... and, if the upper layer is TCP or UDP, the source port and destination port can be added as well in the hash."
SCTP has port number as well. Maybe better say:
"... and, if provided by the upper layer, the source port and destination port can be added as well in the hash."

* Section 2

      ELI - Entropy Label Identifier

Isn't ELI supposed to be the Entropy Label *Indicator* that indicates that the next label is an entropy label?

* Section 7.1.1.

In this example the label stack is stated as 

<Adj_P1P2, Adj_set_P2P3, ELI1, EL1, Adj_P3P4, Adj_P4P5, Adj_P5P6, Adj_P6PE2, VPN_label>

but the text that follows assumes that there is an entropy label for P6 to follow. Is the example stack missing ELI2 and EL2? I think the text should be made consistent with the label stack either way.