Avoiding Equal Cost Multipath Treatment in MPLS Networks
draft-ietf-mpls-ecmp-bcp-03

[Ballot comment]
I wonder if the document should have a clearer statement
of its limitations. Is this truly best (Bcp) that we can achieve
in this space?

[Ballot discuss]
> It is strongly recommended, however, that applications restrict the
> first nibble values to 0x0 and 0x1.  This will ensure that that their
> traffic flows will not be affected if some future routing equipment
> does similar snooping on some future version of IP.

Does this document make a reservation in the IANA IP
protocol version number space?

  http://www.iana.org/assignments/version-numbers

If not (and it should not), the recommendation above is
at best a currently working heuristic.

Suggested edit:

However, even this approach is only a heuristic and
assumes that there are no further allocations of IP
version numbers for any purpose.

[Ballot comment]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best practice
>      recommendations for anyone defining an application to run over an
>      MPLS network and wishes to avoid such treatment.
>

Avoid what treatment? I know what you mean, but that's because I am familar with the document. How about just replacing "and wishes to avoid such treatment" with "where ECMP may be deployed."

>  within a network core, the hashing in based mainly or exclusively on
>
s/in/is

[Ballot discuss]
>  It is strongly recommended, however, that applications restrict the
>  first nibble values to 0x0 and 0x1.  This will ensure that that their
>  traffic flows will not be affected if some future routing equipment
>  does similar snooping on some future version of IP.
>
From the text above, I don't see how 0x0 and 0x1 is any different than 0x2 and 0x3.

So, to make this true for 0x0 and 0x1, I think you need some text in the "Current Practice" section stating categorically that ECMP MUST NOT be performed on packets with the 1st nibble equal to these values. That is the Current Practice as far as I know, but you might want to underscore it here if you can (though it does border on defining "best future practice").

[Ballot discuss]
I see this document as describing a situation sort of analoguous to NAT, in the sense that both the need for NAT, and the fact that MPLS does not deal gracefully with ECMP, are felt by some to be somewhat ugly, but are nonetheless a reality of the current Internet suite of protocols. Thus it is useful to write up the best way to deal with this situation (regardless of how we feel about how the situation was created). Thus I would like to see this document progressed. There are a few bits that should be fixed first however (in my opinion).

Several IESG members have asked is why the document proposes that applications that want to avoid ECMP treatment should use 0x0 and 0x1 in the first nibble. Why not 0x0, 0x1, 0x2, or 0x3? It seems to me that the document could either allow all four, or explain why only two are allowed (which might just be a matter of compatibility with currently deployed equipment), or perhaps limit to just 0x0.

Another DISCUSS comment suggests that the document should contain a pointer to other documents which describe the security considerations of MPLS networks. If there is no such documents, then perhaps we should start an effort to write one (although I would not hold up this document to wait for such an effort to finish).

In reading section 3 of this document, I am assuming that the application label would either be an identifier for an application flow, or a hash on appropriate application level information. Clearly only the application (or implementer of the application) will know what the application is, and thus the application can figure out what to put into this application Label in order to ensure that packets that need to be kept in order will be put on the same path. However, the document doesn't actually say this anywhere. I would suggest that it should.

Another issue that I noticed: The document never actually says whether or not packets that do contain IP inside (lets assume IPv4, although the same would be true for IPv6) should be hashed ONLY on the IP addresses, or if it is okay to hash on both the set of labels in the label stack, and also the IP addresses. I would think the latter, although if I am missing something and the former is needed for some reason the document should say this.

[Ballot comment]
No further objection, but we should consider putting on an IESG note like:

"the wg knows this pragmatic practice, brittle in the sense
used in RFC 3424, 4.3, and not future-proof.

It's not a condescending note, and I think it's the status.

[Ballot comment]
I think we should consider putting on an IESG note that says
"yes, the wg knows this pragmatic practice, brittle in the sense
used in RFC 3424, 4.3, and not future-proof.

It's not a condescending note, and I think it's the status.

[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  The BCP isn't about how routers should do ECMP, it's about how people defining new MPLS applications should structure their packets if they want to avoid accidentally triggering this known behavior.]

I basically agree with the combination of Mark and Margaret's comments:
- I wondered why not 2 or 3 if 0 or 1 was ok.  (If there's a real reason, that's fine - say so)
- I'd like to see "applications SHOULD use first nybble 0, 1, 2 or 3 if they want to avoid ECMP" and I wish there was somewhere appropriate to say "routers MUST NOT perform ECMP on packets with first nybble 0, 1, 2 or 3"

[Ballot comment]
This document does not address any problem or issue that has anything to do with the Internet.

From the IETF Mission statement (RFC3935):

"The goal of the IETF is to make the Internet work better.
 
The mission of the IETF is to produce high quality, relevant
technical and engineering documents that influence the way people
design, use, and manage the Internet in such a way as to make the
Internet work better.  These documents include protocol standards,
best current practices, and informational documents of various kinds."

I don't see anything in this document that will make the Internet work better.

[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  The BCP isn't about how routers should do ECMP, it's about how people defining new MPLS applications should structure their packets if they want to avoid accidentally triggering this known behavior.]

I basically agree with the combination of Mark and Margaret's comments:
- I wondered why not 2 or 3 if 0 or 1 was ok.  (If there's a real reason, that's fine - say so)
- I'd like to see "routers MUST NOT perform ECMP on packets with first nybble 0, 1, 2 or 3"
and "applications SHOULD use first nybble 0, 1, 2 or 3 if they want to avoid ECMP"

[Ballot discuss]
The document says:

  In order to avoid IP ECMP treatment it is necessary that an applica-
  tion take precautions to not be mistaken as IP by deployed equipment
  that snoops on the presumed location of the IP Version field.  Thus,
  at a minimum, the chosen format must disallow the values 0x4 and 0x6
  in the first nibble of their payload.

  It is strongly recommended, however, that applications restrict the
  first nibble values to 0x0 and 0x1.  This will ensure that that their
  traffic flows will not be affected if some future routing equipment
  does similar snooping on some future version of IP.

It seems to be a common theme in MPLS related documents that IP traffic can be identified by the existence of a 0x4 or a 0x6 in the first nibble, and that anything without those values (or in this case, anything with values lower than 0x02) must not be IP traffic.  However, this is not a good assumption.  There are header compression schemes that compress the IP version number, and it is at least theoretically possible that we will one day use (or re-use) the unused portions of the IP version number space. 

I think that the practice of assuming that the only valid IP version numbers are 0x4 and 0x6 could potentially be damaging to the future extensibility of the Internet.

[Ballot discuss]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best practice
>      recommendations for anyone defining an application to run over an
>      MPLS network and wishes to avoid such treatment.
>

Avoid what treatment? I know what you mean, but that's because I am familar with the document. How about just replacing "and wishes to avoid such treatment" with "where ECMP may be deployed."

I see the intro begins as a copy of the abstract, not very creative!

>  within a network core, the hashing in based mainly or exclusively on
>
s/in/is

>  It is strongly recommended, however, that applications restrict the
>  first nibble values to 0x0 and 0x1.  This will ensure that that their
>  traffic flows will not be affected if some future routing equipment
>  does similar snooping on some future version of IP.
>
From the text above, I don't see how 0x0 and 0x1 is any different than 0x2 and 0x3.

So, to make this true for 0x0 and 0x1, I think you need some text in the "Current Practice" section stating categorically that ECMP MUST NOT be performed on packets with the 1st nibble equal to these values. That is the Current Practice as far as I know, but you might want to underscore it here if you can (though it does border on defining "best future practice").

[Ballot comment]
I don't think it would be appropriate to DISCUSS this, since I can see no way in which this could be changed to clear the objection.  But I personally believe this is a stunningly bad idea.  Attempting
to get around current operational behavior by creating this application label simply pushes the
problem down one more label in the stack.  The nibble-seeking behavior certainly makes it
look like those desiring to perform ECMP are willing to do packet inspection--why do we believe
adding a label will not simply cause this nibbling be done further down?  And as operational
behavior changes, this mechanism will be nothing but cruft.

[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.

[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.

[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  I pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.