Mobile Access Gateway (MAG) Multipath Options
draft-ietf-dmm-mag-multihoming-07

[Ballot comment]
Thanks for addressing my DISCUSS comment and refine the spec where needed!

This is my old discuss text for the record:

1) This document should not recommend the use of MPTCP for tunneling, as TCP-in-TCP tunnels are generally not recommend if that can be avoided. Please remove the following part from section 3.2 and leave IP-level tunneling as the only option: 
„Packet distribution can be done either at the transport level, e.g. using MPTCP …“

2) Further the following sentences also in section 3.2 should be revised:

„For example, high throughput services (e.g.
  video streaming) may benefit from per-packet distribution scheme,
  while latency sensitive applications (e.g.  VoIP) are not be spread
  over different WAN paths.“
 
High throughput services only benefit from per-packet scheduling if the service requires higher throughput than one of the links can provide. Also video streaming may not be a good example here because high latency variations can lead to stalls. Therefore in general per-flow scheduling should be recommend for all traffic.
It could still be beneficial to schedule flows that require low latency over the link with the lower base latency, or maybe more important lower jitter, however, often it is not known to the network what the requirements on latency are for a given flow. Therefore is should probably be recommended to schedule all traffic on the "better" link (where better can be pre-configured knowledge or measured) as long as the bandwidth of the incoming traffic is smaller than the bandwidth of the that link, and only use a second link (with per-flow scheduling) if the capacity is required.

3) This document does not really normative specify the use of the newly defined options. It only gives an examples but it does not specify normatively any actions that need to performed on receipt of these options. How does the MAG know if the LMA does not support Multipath binding? An LMA that does not implement this spec will not send back an error message. Why are there two different options? What happens if one of the options is present in the Proxy Binding Update but not the other?

[Ballot comment]
Thanks for your work on this draft.  I have the same concern as the SecDir reviewer in reading the draft, the concern about leaking traffic as a result of multiple tunnels is not addressed in the security considerations section.  Hilary's writeup is quite helpful

https://www.ietf.org/mail-archive/web/secdir/current/msg07446.html

Although the editor says this is not an issue, but I don't think it's clear in the draft.

[Ballot comment]
Sri Gundavelli's proposal text in https://mailarchive.ietf.org/arch/msg/dmm/hh4xrA5vOkC3NuH3l-e3zH9-aM8 addresses my concern.

Thanks Sri,
W

--------------

Old discuss position for posterity:
"Section 3.2.  Traffic distribution schemes
"Per-packet management: the LMA and the MAG distribute packets, belonging to a same IP flow, over more than one bindings (i.e. more than one WAN interface)."
This immediately made my out-of-order-packets antenna pop up, so I read the section looking for mitigations. The very next sentence reads:
"Packet distribution can be done either at the transport level, e.g. using MPTCP or at When operating at the IP packet level, different packets distribution algorithms are possible. " -- the fact that this sentence is a: malformed and b: hand-wavy did nothing to allay my concerns, so I read further:
"The distribution algorithm is left to implementer but whatever the algorithm is, packets distribution likely introduces packet latency and out-of-order delivery.  LMA and MAG shall thus be able to make reordering before packets delivery." - I agree with the first sentence (although it is poorly worded), but the second sentence doesn't follow from the first; "shall thus be able to" implies that the prior text somehow provides a solution, not points out a problem (the sentence is also malformed)-- I think you mean something like "The LMA and MAG thus need to be able reorder packets to their original order before delivery."

This then continues with "Sequence number can be can be used for that purpose, for example using GRE with sequence number option [RFC5845]." - I think that the actual reference should be RFC2890, but regardless of this, I don't think that what you are proposing works - "The Sequence Number field is used to maintain sequence of packets **within** the GRE Tunnel." (from RFC2890, emphasis added). This means that sequence numbers are local to the tunnel, and (as I understand it) your solution involves diverse tunnels. Further, Section 2.2. Sequence Number says: "The receiver may perform a small amount of buffering in an attempt to recover the original sequence of transmitted packets. In this case, the packet may be placed in a buffer sorted by sequence number." - if you are proposing using a single sequence number space for multiple tunnels, you will end up with sequence number space gabs, and lots of buffering, etc.
The section ends with: "However, more detailed considerations on reordering  and IP packet distribution scheme (e.g. definition of packets distribution algorithm) are out the scope of this document." - I think that, unless the prior paragraph is significantly reworked, it should not try and suggest any mitigations.

The whole idea of striping packets of a flow across (notably) different transports seems like a really bad idea to me -- is it actually needed?"

[Ballot comment]
Figure 1 labels the bottom-left most construct "Flow0-4" -- I suspect this should be "Flow-4."

I see the requests to remove MPTCP from the document, and the rationale seems sound. If, by some chance, any mention of MPTCP remains in the final version, please make sure it gets expanded (as an acronym) and cited.

The final paragraph in section 3.2 starts with "Because latency introduced by..." -- this isn't the reason damage can arise. The problem is the *variation* in latency. Suggest something like: "Because the variation in packet latency, also known as jitter, introduced by per-packet scheduling between access networks can cause..."

Section 4.1, definition of If-ATT -- suggest citing the specific section: "[RFC5213] section 8.5."

Section 4.1, definition of If-Label -- the value of 0 appears to be reserved? Ideally, this would prescribe specific behavior for an LMA if they receive an If-Label of 0.

Section 4.1, definition of BID -- same comment; ideally, this would specify recipient behavior if set to 0 or 255.

Section 4.3 defines a new response code for LMAs that don't implement this spec. Existing, already deployed LMAs will necessarily have a different reaction to receiving these unknown options than sending this response code. This section should provide guidance for MAGs letting them know what observable behavior they should expect when sending these options to LMAs unaware of this extension at all. Additionally, _if_ such observable behavior is sufficient for the MAG to understand what is happening, I would assert that the new response code is unnecessary and should be removed.

Please expand the following acronyms upon first use and in the title;
see https://www.rfc-editor.org/materials/abbrev.expansion.txt for guidance.

- MAG - Mobile Access Gateway (in the title)
- GRE - Generic Routing Encapsulation
- LMA - Local Mobility Anchor
- LTE - Long-Term Evolution
- MN - Mobile Node
- BCE - Bonding Channel Entity

[Ballot comment]
Please expand the following acronyms upon first use and in the title;
see https://www.rfc-editor.org/materials/abbrev.expansion.txt for guidance.

- MAG - Mobile Access Gateway (in the title)
- GRE - Generic Routing Encapsulation
- LMA - Local Mobility Anchor
- LTE - Long-Term Evolution
- MN - Mobile Node
- BCE - Bonding Channel Entity
- MPTCP - Multipath TCP

[Ballot comment]
I have a few editorial comments/nits:

- There are several acronyms that should be expanded on first mention. (including at least a couple that are expanded on _second_ mention (MAG and LMA).

- 3.2, "Per-packet management" bullet: The second sentence does not make sense.

- 3.2, last paragraph: Are we really talking about the latency introduced by per-packet management, or jitter (i.e. variation in latency)?

- 4.1, definition of "Interface Label": This doesn't really define the term "interface label". It talks about how it is used, but not what it represents.

[Ballot discuss]
Section 3.2.  Traffic distribution schemes
"Per-packet management: the LMA and the MAG distribute packets, belonging to a same IP flow, over more than one bindings (i.e. more than one WAN interface)."
This immediately made my out-of-order-packets antenna pop up, so I read the section looking for mitigations. The very next sentence reads:
"Packet distribution can be done either at the transport level, e.g. using MPTCP or at When operating at the IP packet level, different packets distribution algorithms are possible. " -- the fact that this sentence is a: malformed and b: hand-wavy did nothing to allay my concerns, so I read further:
"The distribution algorithm is left to implementer but whatever the algorithm is, packets distribution likely introduces packet latency and out-of-order delivery.  LMA and MAG shall thus be able to make reordering before packets delivery." - I agree with the first sentence (although it is poorly worded), but the second sentence doesn't follow from the first; "shall thus be able to" implies that the prior text somehow provides a solution, not points out a problem (the sentence is also malformed)-- I think you mean something like "The LMA and MAG thus need to be able reorder packets to their original order before delivery."

This then continues with "Sequence number can be can be used for that purpose, for example using GRE with sequence number option [RFC5845]." - I think that the actual reference should be RFC2890, but regardless of this, I don't think that what you are proposing works - "The Sequence Number field is used to maintain sequence of packets **within** the GRE Tunnel." (from RFC2890, emphasis added). This means that sequence numbers are local to the tunnel, and (as I understand it) your solution involves diverse tunnels. Further, Section 2.2. Sequence Number says: "The receiver may perform a small amount of buffering in an attempt to recover the original sequence of transmitted packets. In this case, the packet may be placed in a buffer sorted by sequence number." - if you are proposing using a single sequence number space for multiple tunnels, you will end up with sequence number space gabs, and lots of buffering, etc.
The section ends with: "However, more detailed considerations on reordering  and IP packet distribution scheme (e.g. definition of packets distribution algorithm) are out the scope of this document." - I think that, unless the prior paragraph is significantly reworked, it should not try and suggest any mitigations.

The whole idea of striping packets of a flow across (notably) different transports seems like a really bad idea to me -- is it actually needed?

[Ballot comment]
1) Sec 3.2: "Packet distribution can be done
      either at the transport level, e.g. using MPTCP or at When
      operating at the IP packet level, different packets distribution
      algorithms are possible. "  is clearly not a properly written sentence.
      I also share Mirja's concerns about TCP inside TCP.

2) Sec 4.1: "This flag MUST NOT be set to a
      value of (1), if the value of the Registration Overwrite Flag (O)
      is set to a value of (1).

  Binding Overwrite (O)"
  Please make the draft consistent as to the name of the flag (O)

[Ballot discuss]
Thanks for your work on this draft.  I had the same concern as the SecDir reviewer in reading the draft, the concern about leaking traffic as a result of multiple tunnels is not addressed in the security considerations section.  Hilary's writeup is quite helpful

https://www.ietf.org/mail-archive/web/secdir/current/msg07446.html

[Ballot comment]
Nit: IANA-3 value is referenced in the IANA Considerations, but is not used. I think you meant IANA-4 in the place where IANA-3 is currently referenced.

[Ballot discuss]
1) This document should not recommend the use of MPTCP for tunneling, as TCP-in-TCP tunnels are generally not recommend if that can be avoided. Please remove the following part from section 3.2 and leave IP-level tunneling as the only option: 
„Packet distribution can be done either at the transport level, e.g. using MPTCP …“

2) Further the following sentences also in section 3.2 should be revised:

„For example, high throughput services (e.g.
  video streaming) may benefit from per-packet distribution scheme,
  while latency sensitive applications (e.g.  VoIP) are not be spread
  over different WAN paths.“
 
High throughput services only benefit from per-packet scheduling if the service requires higher throughput than one of the links can provide. Also video streaming may not be a good example here because high latency variations can lead to stalls. Therefore in general per-flow scheduling should be recommend for all traffic.
It could still be beneficial to schedule flows that require low latency over the link with the lower base latency, or maybe more important lower jitter, however, often it is not known to the network what the requirements on latency are for a given flow. Therefore is should probably be recommended to schedule all traffic on the "better" link (where better can be pre-configured knowledge or measured) as long as the bandwidth of the incoming traffic is smaller than the bandwidth of the that link, and only use a second link (with per-flow scheduling) if the capacity is required.

3) This document does not really normative specify the use of the newly defined options. It only gives an examples but it does not specify normatively any actions that need to performed on receipt of these options. How does the MAG know if the LMA does not support Multipath binding? An LMA that does not implement this spec will not send back an error message. Why are there two different options? What happens if one of the options is present in the Proxy Binding Update but not the other?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has completed its review of draft-ietf-dmm-mag-multihoming-03.txt. If any part of this review is inaccurate, please let us know.

The IANA Services Operator understands that, upon approval of this document, there are three actions which we must complete.

First, in the Mobility Options registry on the Mobile IPv6 parameters registry page located at:

https://www.iana.org/assignments/mobility-parameters/

Value: [ TBD-at-registration ]
Description: MAG Multipath-Binding option
Reference: [ RFC-to-be ]

Value: [ TBD-at-registration ]
Description: MAG Identifier option
Reference: [ RFC-to-be ]

Second, in the Status Codes registry also on the Mobile IPv6 parameters registry page located at:

https://www.iana.org/assignments/mobility-parameters/

a single, new value is to be registered as follows:

Value: [ TBD-at-registration ]
Description: CANNOT_SUPPORT_MULTIPATH_BINDING
Reference: [ RFC-to-be ]

We note that the authors request that the allocated value be greater than 127.

The IANA Services Operator understands that these three actions are the only ones required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is only to confirm what actions will be performed.


Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC: draft-ietf-dmm-mag-multihoming@ietf.org, jouni.nospam@gmail.com, Jouni Korhonen , suresh.krishnan@gmail.com, dmm-chairs@ietf.org, dmm@ietf.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (MAG Multipath Binding Option) to Proposed Standard


The IESG has received a request from the Distributed Mobility Management
WG (dmm) to consider the following document:
- 'MAG Multipath Binding Option'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-06-16. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This specification defines extensions to the Proxy Mobile IPv6
  protocol for allowing a mobile access gateway to register more than
  one proxy care-of-address with the local mobility anchor and to
  simultaneously establish multiple IP tunnels with the local mobility
  anchor.  This capability allows the mobile access gateway to utilize
  all the available access networks for routing mobile node's IP
  traffic.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-dmm-mag-multihoming/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-dmm-mag-multihoming/ballot/


No IPR declarations have been submitted directly on this I-D.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up.

Changes are expected over time. This version is dated 24 February 2012.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

  This document is Proposed Standard and it is also stated in the
  cover page.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This document specifies a multiple proxy Care-of Addresses extension for Proxy Mobile
  IPv6.  The extension allows a multihomed Mobile Access Gateway to register more than
  one proxy care-of-address to the Local Mobility Anchor.

Working Group Summary

  The document has been in a WG for a long time and gone through a bigger
  change to be independent of specific deployment architecture. The solution
  reached consensus in the working group.

Document Quality

  There are commercial pre-standard implementations of the protocol.
  The document has been reviewed thoroughly and there is no need
  to have external directorate reviews.

Personnel

  Jouni Korhonen (jouni.nospam@gmail.com) is the document shepherd.
  Suresh Krishnan (suresh.krishnan@ericsson.com) is the responsible AD.

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

  The shepherd has read the latest revision of the document and thinks
  it is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

  No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

  No.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

  None.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

  Yes. Positive "no IPR" acknowledgements have been received from all
  authors.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

  No.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

  The document has WG consensus and support behind it.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

  No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

  IDnits reports one error:

  ** The abstract seems to contain references ([RFC3963], [RFC4908],
    [RFC5213], [RFC6275]), which it shouldn't.  Please replace those with
    straight textual mentions of the documents in question.

  The shepherd thinks the RFC Editor can easily expand these and remove
  references from the abstract when producing a new revision. The same goes
  for the other IDnits complaints. They will automatically be corrected once
  the RFC editor produces a new revision.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

  No need for the MIB Doctor, media type, and URI type reviews. The
  document has no protocol elements that would need such reviews.

(13) Have all references within this document been identified as
either normative or informative?

  Yes.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

  None.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

  No.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

  No.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

  The document requires two new Mobility Option code points and
  defines one new status code value. Respective IANA registries are
  clearly identified and no new registries are created.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

  None.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

  Shepherd has done IDnits check. No other validations were done.

IPR call:

11/13 Sri "I am not aware of any IPR on this draft."
11/14 Pierrick "I am not aware of any IPR on this draft."