Skip to main content

Avoiding Equal Cost Multipath Treatment in MPLS Networks
RFC 4928

Revision differences

Document history

Date Rev. By Action
2015-10-14
03 (System) Notify list changed from mpls-chairs@ietf.org to (None)
2012-08-22
03 (System) post-migration administrative database adjustment to the No Objection position for Margaret Wasserman
2012-08-22
03 (System) post-migration administrative database adjustment to the No Objection position for Jari Arkko
2012-08-22
03 (System) post-migration administrative database adjustment to the No Objection position for Mark Townsley
2012-08-22
03 (System) post-migration administrative database adjustment to the Yes position for Ross Callon
2012-08-22
03 (System) post-migration administrative database adjustment to the No Objection position for Russ Housley
2007-06-29
03 Amy Vezza State Changes to RFC Published from RFC Ed Queue by Amy Vezza
2007-06-29
03 Amy Vezza [Note]: 'RFC 4928, BCP 0128' added by Amy Vezza
2007-06-27
03 (System) RFC published
2007-04-12
03 (System) IANA Action state changed to RFC-Ed-Ack from Waiting on RFC Editor
2007-04-11
03 (System) IANA Action state changed to Waiting on RFC Editor from Waiting on Authors
2007-04-05
03 (System) IANA Action state changed to Waiting on Authors from In Progress
2007-04-02
03 (System) IANA Action state changed to In Progress from No IC
2007-03-27
03 Amy Vezza State Changes to RFC Ed Queue from Approved-announcement sent by Amy Vezza
2007-03-19
03 (System) IANA Action state changed to No IC from In Progress
2007-03-19
03 (System) IANA Action state changed to In Progress
2007-03-18
03 Amy Vezza IESG state changed to Approved-announcement sent
2007-03-18
03 Amy Vezza IESG has approved the document
2007-03-18
03 Amy Vezza Closed "Approve" ballot
2007-03-17
03 Ross Callon State Changes to Approved-announcement to be sent from IESG Evaluation::AD Followup by Ross Callon
2007-03-14
03 Jari Arkko [Ballot Position Update] Position for Jari Arkko has been changed to No Objection from Discuss by Jari Arkko
2007-03-13
03 Ross Callon Ballot has been issued by Ross Callon
2007-03-09
03 Samuel Weiler Assignment of request for Last Call review by SECDIR to Barry Leiba was rejected
2007-03-09
03 (System) Removed from agenda for telechat - 2007-03-08
2007-03-08
03 Amy Vezza State Changes to IESG Evaluation::AD Followup from IESG Evaluation by Amy Vezza
2007-03-08
03 Jari Arkko
[Ballot comment]
I wonder if the document should have a clearer statement
of its limitations. Is this truly best (Bcp) that we can achieve
in …
[Ballot comment]
I wonder if the document should have a clearer statement
of its limitations. Is this truly best (Bcp) that we can achieve
in this space?
2007-03-08
03 Jari Arkko [Ballot comment]
I wonder if the document should have a clearer statement
of its limitations.
2007-03-08
03 Jari Arkko
[Ballot discuss]
> It is strongly recommended, however, that applications restrict the
> first nibble values to 0x0 and 0x1.  This will ensure that that …
[Ballot discuss]
> It is strongly recommended, however, that applications restrict the
> first nibble values to 0x0 and 0x1.  This will ensure that that their
> traffic flows will not be affected if some future routing equipment
> does similar snooping on some future version of IP.

Does this document make a reservation in the IANA IP
protocol version number space?

  http://www.iana.org/assignments/version-numbers

If not (and it should not), the recommendation above is
at best a currently working heuristic.

Suggested edit:

However, even this approach is only a heuristic and
assumes that there are no further allocations of IP
version numbers for any purpose.
2007-03-08
03 Jari Arkko [Ballot Position Update] New position, Discuss, has been recorded by Jari Arkko
2007-03-07
03 Dan Romascanu [Ballot Position Update] New position, No Objection, has been recorded by Dan Romascanu
2007-03-02
03 Samuel Weiler Request for Last Call review by SECDIR is assigned to Barry Leiba
2007-03-02
03 Samuel Weiler Request for Last Call review by SECDIR is assigned to Barry Leiba
2007-03-01
03 Mark Townsley [Ballot Position Update] Position for Mark Townsley has been changed to No Objection from Discuss by Mark Townsley
2007-02-16
03 Ross Callon State Changes to IESG Evaluation from IESG Evaluation::AD Followup by Ross Callon
2007-02-16
03 Ross Callon Ballot has been issued by Ross Callon
2007-02-16
03 Ross Callon Placed on agenda for telechat - 2007-03-08 by Ross Callon
2007-02-16
03 Russ Housley [Ballot Position Update] Position for Russ Housley has been changed to No Objection from Discuss by Russ Housley
2007-02-15
03 Ross Callon [Ballot Position Update] Position for Ross Callon has been changed to Yes from No Objection by Ross Callon
2007-02-15
03 Ross Callon [Ballot Position Update] Position for Ross Callon has been changed to No Objection from Discuss by Ross Callon
2007-02-15
03 (System) Sub state has been changed to AD Follow up from New Id Needed
2007-02-15
03 (System) New version available: draft-ietf-mpls-ecmp-bcp-03.txt
2007-02-01
03 Mark Townsley
[Ballot comment]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best …
[Ballot comment]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best practice
>      recommendations for anyone defining an application to run over an
>      MPLS network and wishes to avoid such treatment.
>

Avoid what treatment? I know what you mean, but that's because I am familar with the document. How about just replacing "and wishes to avoid such treatment" with "where ECMP may be deployed."

>  within a network core, the hashing in based mainly or exclusively on
>
s/in/is
2007-02-01
03 Mark Townsley
[Ballot discuss]
>  It is strongly recommended, however, that applications restrict the
>  first nibble values to 0x0 and 0x1.  This will ensure that that …
[Ballot discuss]
>  It is strongly recommended, however, that applications restrict the
>  first nibble values to 0x0 and 0x1.  This will ensure that that their
>  traffic flows will not be affected if some future routing equipment
>  does similar snooping on some future version of IP.
>
From the text above, I don't see how 0x0 and 0x1 is any different than 0x2 and 0x3.

So, to make this true for 0x0 and 0x1, I think you need some text in the "Current Practice" section stating categorically that ECMP MUST NOT be performed on packets with the 1st nibble equal to these values. That is the Current Practice as far as I know, but you might want to underscore it here if you can (though it does border on defining "best future practice").
2006-07-26
03 Ross Callon
[Ballot discuss]
I see this document as describing a situation sort of analoguous to NAT, in the sense that both the need for NAT, and …
[Ballot discuss]
I see this document as describing a situation sort of analoguous to NAT, in the sense that both the need for NAT, and the fact that MPLS does not deal gracefully with ECMP, are felt by some to be somewhat ugly, but are nonetheless a reality of the current Internet suite of protocols. Thus it is useful to write up the best way to deal with this situation (regardless of how we feel about how the situation was created). Thus I would like to see this document progressed. There are a few bits that should be fixed first however (in my opinion).

Several IESG members have asked is why the document proposes that applications that want to avoid ECMP treatment should use 0x0 and 0x1 in the first nibble. Why not 0x0, 0x1, 0x2, or 0x3? It seems to me that the document could either allow all four, or explain why only two are allowed (which might just be a matter of compatibility with currently deployed equipment), or perhaps limit to just 0x0.

Another DISCUSS comment suggests that the document should contain a pointer to other documents which describe the security considerations of MPLS networks. If there is no such documents, then perhaps we should start an effort to write one (although I would not hold up this document to wait for such an effort to finish).

In reading section 3 of this document, I am assuming that the application label would either be an identifier for an application flow, or a hash on appropriate application level information. Clearly only the application (or implementer of the application) will know what the application is, and thus the application can figure out what to put into this application Label in order to ensure that packets that need to be kept in order will be put on the same path. However, the document doesn't actually say this anywhere. I would suggest that it should.

Another issue that I noticed: The document never actually says whether or not packets that do contain IP inside (lets assume IPv4, although the same would be true for IPv6) should be hashed ONLY on the IP addresses, or if it is okay to hash on both the set of labels in the label stack, and also the IP addresses. I would think the latter, although if I am missing something and the former is needed for some reason the document should say this.
2006-07-26
03 Ross Callon [Ballot Position Update] New position, Discuss, has been recorded for Ross Callon by Ross Callon
2006-07-24
03 Bill Fenner State Change Notice email list have been change to mpls-chairs@tools.ietf.org from swallow@cisco.com, loa@pi.se
2006-05-31
03 Magnus Westerlund [Ballot Position Update] New position, No Objection, has been recorded for Magnus Westerlund by Magnus Westerlund
2006-05-25
03 Cullen Jennings [Ballot Position Update] New position, No Objection, has been recorded for Cullen Jennings by Cullen Jennings
2006-03-29
03 Bill Fenner Shepherding AD has been changed to Ross Callon from Bill Fenner
2006-03-29
03 Bill Fenner Shepherding AD has been changed to Bill Fenner from Alex Zinin
2006-03-09
03 Margaret Cullen [Ballot Position Update] Position for Margaret Wasserman has been changed to No Objection from Discuss by Margaret Wasserman
2006-01-17
03 Alex Zinin State Changes to IESG Evaluation::Revised ID Needed from IESG Evaluation::AD Followup by Alex Zinin
2005-12-01
03 Amy Vezza State Changes to IESG Evaluation::AD Followup from IESG Evaluation by Amy Vezza
2005-12-01
03 Bert Wijnen [Ballot comment]
I share concerns raised by Allison, Margaret and Bill.
2005-12-01
03 Bert Wijnen [Ballot Position Update] New position, No Objection, has been recorded for Bert Wijnen by Bert Wijnen
2005-12-01
03 Allison Mankin
[Ballot comment]
No further objection, but we should consider putting on an IESG note like:

"the wg knows this pragmatic practice, brittle in the sense …
[Ballot comment]
No further objection, but we should consider putting on an IESG note like:

"the wg knows this pragmatic practice, brittle in the sense
used in RFC 3424, 4.3, and not future-proof.

It's not a condescending note, and I think it's the status.
2005-12-01
03 Allison Mankin [Ballot Position Update] Position for Allison Mankin has been changed to No Objection from Undefined by Allison Mankin
2005-12-01
03 Allison Mankin
[Ballot comment]
I think we should consider putting on an IESG note that says
"yes, the wg knows this pragmatic practice, brittle in the sense …
[Ballot comment]
I think we should consider putting on an IESG note that says
"yes, the wg knows this pragmatic practice, brittle in the sense
used in RFC 3424, 4.3, and not future-proof.

It's not a condescending note, and I think it's the status.
2005-12-01
03 Allison Mankin [Ballot Position Update] New position, Undefined, has been recorded for Allison Mankin by Allison Mankin
2005-12-01
03 Bill Fenner
[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  …
[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  The BCP isn't about how routers should do ECMP, it's about how people defining new MPLS applications should structure their packets if they want to avoid accidentally triggering this known behavior.]

I basically agree with the combination of Mark and Margaret's comments:
- I wondered why not 2 or 3 if 0 or 1 was ok.  (If there's a real reason, that's fine - say so)
- I'd like to see "applications SHOULD use first nybble 0, 1, 2 or 3 if they want to avoid ECMP" and I wish there was somewhere appropriate to say "routers MUST NOT perform ECMP on packets with first nybble 0, 1, 2 or 3"
2005-12-01
03 Jon Peterson [Ballot Position Update] New position, No Objection, has been recorded for Jon Peterson by Jon Peterson
2005-11-30
03 David Kessens
[Ballot comment]
This document does not address any problem or issue that has anything to do with the Internet.

From the IETF Mission statement ( …
[Ballot comment]
This document does not address any problem or issue that has anything to do with the Internet.

From the IETF Mission statement (RFC3935):

"The goal of the IETF is to make the Internet work better.
 
The mission of the IETF is to produce high quality, relevant
technical and engineering documents that influence the way people
design, use, and manage the Internet in such a way as to make the
Internet work better.  These documents include protocol standards,
best current practices, and informational documents of various kinds."

I don't see anything in this document that will make the Internet work better.
2005-11-30
03 David Kessens [Ballot Position Update] New position, Abstain, has been recorded for David Kessens by David Kessens
2005-11-30
03 Bill Fenner
[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  …
[Ballot comment]
[Ted: The nibble searching is described as background to why the solution is necessary; the define-your-payload-to-not-look-like-IP is the proposed solution in the BCP.  The BCP isn't about how routers should do ECMP, it's about how people defining new MPLS applications should structure their packets if they want to avoid accidentally triggering this known behavior.]

I basically agree with the combination of Mark and Margaret's comments:
- I wondered why not 2 or 3 if 0 or 1 was ok.  (If there's a real reason, that's fine - say so)
- I'd like to see "routers MUST NOT perform ECMP on packets with first nybble 0, 1, 2 or 3"
and "applications SHOULD use first nybble 0, 1, 2 or 3 if they want to avoid ECMP"
2005-11-30
03 Bill Fenner [Ballot Position Update] New position, No Objection, has been recorded for Bill Fenner by Bill Fenner
2005-11-30
03 Margaret Cullen
[Ballot discuss]
The document says:

  In order to avoid IP ECMP treatment it is necessary that an applica-
  tion take precautions to not …
[Ballot discuss]
The document says:

  In order to avoid IP ECMP treatment it is necessary that an applica-
  tion take precautions to not be mistaken as IP by deployed equipment
  that snoops on the presumed location of the IP Version field.  Thus,
  at a minimum, the chosen format must disallow the values 0x4 and 0x6
  in the first nibble of their payload.

  It is strongly recommended, however, that applications restrict the
  first nibble values to 0x0 and 0x1.  This will ensure that that their
  traffic flows will not be affected if some future routing equipment
  does similar snooping on some future version of IP.

It seems to be a common theme in MPLS related documents that IP traffic can be identified by the existence of a 0x4 or a 0x6 in the first nibble, and that anything without those values (or in this case, anything with values lower than 0x02) must not be IP traffic.  However, this is not a good assumption.  There are header compression schemes that compress the IP version number, and it is at least theoretically possible that we will one day use (or re-use) the unused portions of the IP version number space. 

I think that the practice of assuming that the only valid IP version numbers are 0x4 and 0x6 could potentially be damaging to the future extensibility of the Internet.
2005-11-30
03 Margaret Cullen [Ballot Position Update] New position, Discuss, has been recorded for Margaret Wasserman by Margaret Wasserman
2005-11-30
03 Mark Townsley
[Ballot discuss]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best …
[Ballot discuss]
>      This document describes the Equal Cost Multipath (ECMP) behavior
>      of currently deployed MPLS networks and makes best practice
>      recommendations for anyone defining an application to run over an
>      MPLS network and wishes to avoid such treatment.
>

Avoid what treatment? I know what you mean, but that's because I am familar with the document. How about just replacing "and wishes to avoid such treatment" with "where ECMP may be deployed."

I see the intro begins as a copy of the abstract, not very creative!

>  within a network core, the hashing in based mainly or exclusively on
>
s/in/is

>  It is strongly recommended, however, that applications restrict the
>  first nibble values to 0x0 and 0x1.  This will ensure that that their
>  traffic flows will not be affected if some future routing equipment
>  does similar snooping on some future version of IP.
>
From the text above, I don't see how 0x0 and 0x1 is any different than 0x2 and 0x3.

So, to make this true for 0x0 and 0x1, I think you need some text in the "Current Practice" section stating categorically that ECMP MUST NOT be performed on packets with the 1st nibble equal to these values. That is the Current Practice as far as I know, but you might want to underscore it here if you can (though it does border on defining "best future practice").
2005-11-30
03 Mark Townsley [Ballot Position Update] New position, Discuss, has been recorded for Mark Townsley by Mark Townsley
2005-11-28
03 Ted Hardie
[Ballot comment]
I don't think it would be appropriate to DISCUSS this, since I can see no way in which this could be changed to …
[Ballot comment]
I don't think it would be appropriate to DISCUSS this, since I can see no way in which this could be changed to clear the objection.  But I personally believe this is a stunningly bad idea.  Attempting
to get around current operational behavior by creating this application label simply pushes the
problem down one more label in the stack.  The nibble-seeking behavior certainly makes it
look like those desiring to perform ECMP are willing to do packet inspection--why do we believe
adding a label will not simply cause this nibbling be done further down?  And as operational
behavior changes, this mechanism will be nothing but cruft.
2005-11-28
03 Ted Hardie [Ballot Position Update] New position, Abstain, has been recorded for Ted Hardie by Ted Hardie
2005-11-28
03 Russ Housley
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the …
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.
2005-11-28
03 Russ Housley
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the …
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  A pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.
2005-11-28
03 Russ Housley
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  I pointer to the …
[Ballot discuss]
The security considerations are not sufficient.  While I believe that
  this document is not creating any new concerns.  I pointer to the
  MPLS security considerations in the appropriate specifications seems
  appropriate.  By including the pointer (or pointers) here, we can
  make sure that the implementor reads the appropriate material.
2005-11-28
03 Russ Housley [Ballot Position Update] New position, Discuss, has been recorded for Russ Housley by Russ Housley
2005-11-25
03 Sam Hartman [Ballot Position Update] New position, Yes, has been recorded for Sam Hartman by Sam Hartman
2005-11-23
03 Brian Carpenter [Ballot Position Update] New position, No Objection, has been recorded for Brian Carpenter by Brian Carpenter
2005-11-16
03 Scott Hollenbeck [Ballot Position Update] New position, No Objection, has been recorded for Scott Hollenbeck by Scott Hollenbeck
2005-10-27
03 Alex Zinin Placed on agenda for telechat - 2005-12-01 by Alex Zinin
2005-10-27
03 Alex Zinin State Changes to IESG Evaluation from Waiting for AD Go-Ahead::AD Followup by Alex Zinin
2005-10-27
03 Alex Zinin [Ballot Position Update] New position, Yes, has been recorded for Alex Zinin
2005-10-27
03 Alex Zinin Ballot has been issued by Alex Zinin
2005-10-27
03 Alex Zinin Created "Approve" ballot
2005-10-24
03 (System) Sub state has been changed to AD Follow up from New Id Needed
2005-10-24
02 (System) New version available: draft-ietf-mpls-ecmp-bcp-02.txt
2005-10-18
03 Alex Zinin State Changes to Waiting for AD Go-Ahead::Revised ID Needed from Waiting for Writeup by Alex Zinin
2005-10-18
03 Alex Zinin Waiting for a rev to address GENART review
2005-09-21
03 (System) State has been changed to Waiting for Writeup from In Last Call by system
2005-09-08
03 Michelle Cotton IANA Last Call Comments:
NO IANA Considerations section.
We understand this document to have NO IANA Actions.
2005-09-07
03 Amy Vezza Last call sent
2005-09-07
03 Amy Vezza State Changes to In Last Call from Last Call Requested by Amy Vezza
2005-09-06
03 Alex Zinin Last Call was requested by Alex Zinin
2005-09-06
03 Alex Zinin State Changes to Last Call Requested from AD Evaluation by Alex Zinin
2005-09-06
03 (System) Ballot writeup text was added
2005-09-06
03 (System) Last call text was added
2005-09-06
03 (System) Ballot approval text was added
2005-09-06
03 Alex Zinin State Changes to AD Evaluation from Publication Requested by Alex Zinin
2005-07-13
03 Dinara Suleymanova Draft Added by Dinara Suleymanova in state Publication Requested
2005-07-06
01 (System) New version available: draft-ietf-mpls-ecmp-bcp-01.txt
2004-10-19
00 (System) New version available: draft-ietf-mpls-ecmp-bcp-00.txt