Mobile Access Gateway (MAG) Multipath Options
draft-ietf-dmm-mag-multihoming-07

Sri Gundavelli's proposal text in https://mailarchive.ietf.org/arch/msg/dmm/hh4xrA5vOkC3NuH3l-e3zH9-aM8 addresses my concern.

Thanks Sri,
W

--------------

Old discuss position for posterity:
"Section 3.2.  Traffic distribution schemes
"Per-packet management: the LMA and the MAG distribute packets, belonging to a same IP flow, over more than one bindings (i.e. more than one WAN interface)."
This immediately made my out-of-order-packets antenna pop up, so I read the section looking for mitigations. The very next sentence reads:
"Packet distribution can be done either at the transport level, e.g. using MPTCP or at When operating at the IP packet level, different packets distribution algorithms are possible. " -- the fact that this sentence is a: malformed and b: hand-wavy did nothing to allay my concerns, so I read further:
"The distribution algorithm is left to implementer but whatever the algorithm is, packets distribution likely introduces packet latency and out-of-order delivery.  LMA and MAG shall thus be able to make reordering before packets delivery." - I agree with the first sentence (although it is poorly worded), but the second sentence doesn't follow from the first; "shall thus be able to" implies that the prior text somehow provides a solution, not points out a problem (the sentence is also malformed)-- I think you mean something like "The LMA and MAG thus need to be able reorder packets to their original order before delivery."

This then continues with "Sequence number can be can be used for that purpose, for example using GRE with sequence number option [RFC5845]." - I think that the actual reference should be RFC2890, but regardless of this, I don't think that what you are proposing works - "The Sequence Number field is used to maintain sequence of packets **within** the GRE Tunnel." (from RFC2890, emphasis added). This means that sequence numbers are local to the tunnel, and (as I understand it) your solution involves diverse tunnels. Further, Section 2.2. Sequence Number says: "The receiver may perform a small amount of buffering in an attempt to recover the original sequence of transmitted packets. In this case, the packet may be placed in a buffer sorted by sequence number." - if you are proposing using a single sequence number space for multiple tunnels, you will end up with sequence number space gabs, and lots of buffering, etc. 
The section ends with: "However, more detailed considerations on reordering  and IP packet distribution scheme (e.g. definition of packets distribution algorithm) are out the scope of this document." - I think that, unless the prior paragraph is significantly reworked, it should not try and suggest any mitigations.

The whole idea of striping packets of a flow across (notably) different transports seems like a really bad idea to me -- is it actually needed?"

Figure 1 labels the bottom-left most construct "Flow0-4" -- I suspect this should be "Flow-4."

I see the requests to remove MPTCP from the document, and the rationale seems sound. If, by some chance, any mention of MPTCP remains in the final version, please make sure it gets expanded (as an acronym) and cited.

The final paragraph in section 3.2 starts with "Because latency introduced by..." -- this isn't the reason damage can arise. The problem is the *variation* in latency. Suggest something like: "Because the variation in packet latency, also known as jitter, introduced by per-packet scheduling between access networks can cause..."

Section 4.1, definition of If-ATT -- suggest citing the specific section: "[RFC5213] section 8.5."

Section 4.1, definition of If-Label -- the value of 0 appears to be reserved? Ideally, this would prescribe specific behavior for an LMA if they receive an If-Label of 0.

Section 4.1, definition of BID -- same comment; ideally, this would specify recipient behavior if set to 0 or 255.

Section 4.3 defines a new response code for LMAs that don't implement this spec. Existing, already deployed LMAs will necessarily have a different reaction to receiving these unknown options than sending this response code. This section should provide guidance for MAGs letting them know what observable behavior they should expect when sending these options to LMAs unaware of this extension at all. Additionally, _if_ such observable behavior is sufficient for the MAG to understand what is happening, I would assert that the new response code is unnecessary and should be removed.

Please expand the following acronyms upon first use and in the title;
see https://www.rfc-editor.org/materials/abbrev.expansion.txt for guidance.

 - MAG - Mobile Access Gateway (in the title)
 - GRE - Generic Routing Encapsulation
 - LMA - Local Mobility Anchor
 - LTE - Long-Term Evolution
 - MN - Mobile Node
 - BCE - Bonding Channel Entity

Nit: IANA-3 value is referenced in the IANA Considerations, but is not used. I think you meant IANA-4 in the place where IANA-3 is currently referenced.

1) Sec 3.2: "Packet distribution can be done
      either at the transport level, e.g. using MPTCP or at When
      operating at the IP packet level, different packets distribution
      algorithms are possible. "  is clearly not a properly written sentence.
      I also share Mirja's concerns about TCP inside TCP.

2) Sec 4.1: "This flag MUST NOT be set to a
      value of (1), if the value of the Registration Overwrite Flag (O)
      is set to a value of (1).

   Binding Overwrite (O)"
   Please make the draft consistent as to the name of the flag (O)

I have a few editorial comments/nits:

- There are several acronyms that should be expanded on first mention. (including at least a couple that are expanded on _second_ mention (MAG and LMA).

- 3.2, "Per-packet management" bullet: The second sentence does not make sense. 

- 3.2, last paragraph: Are we really talking about the latency introduced by per-packet management, or jitter (i.e. variation in latency)?

- 4.1, definition of "Interface Label": This doesn't really define the term "interface label". It talks about how it is used, but not what it represents.

Thanks for your work on this draft.  I have the same concern as the SecDir reviewer in reading the draft, the concern about leaking traffic as a result of multiple tunnels is not addressed in the security considerations section.  Hilary's writeup is quite helpful

https://www.ietf.org/mail-archive/web/secdir/current/msg07446.html

Although the editor says this is not an issue, but I don't think it's clear in the draft.

Thanks for addressing my DISCUSS comment and refine the spec where needed!

This is my old discuss text for the record:

1) This document should not recommend the use of MPTCP for tunneling, as TCP-in-TCP tunnels are generally not recommend if that can be avoided. Please remove the following part from section 3.2 and leave IP-level tunneling as the only option: 
„Packet distribution can be done either at the transport level, e.g. using MPTCP …“ 

2) Further the following sentences also in section 3.2 should be revised:

„For example, high throughput services (e.g.
   video streaming) may benefit from per-packet distribution scheme,
   while latency sensitive applications (e.g.  VoIP) are not be spread
   over different WAN paths.“
   
High throughput services only benefit from per-packet scheduling if the service requires higher throughput than one of the links can provide. Also video streaming may not be a good example here because high latency variations can lead to stalls. Therefore in general per-flow scheduling should be recommend for all traffic. 
It could still be beneficial to schedule flows that require low latency over the link with the lower base latency, or maybe more important lower jitter, however, often it is not known to the network what the requirements on latency are for a given flow. Therefore is should probably be recommended to schedule all traffic on the "better" link (where better can be pre-configured knowledge or measured) as long as the bandwidth of the incoming traffic is smaller than the bandwidth of the that link, and only use a second link (with per-flow scheduling) if the capacity is required.

3) This document does not really normative specify the use of the newly defined options. It only gives an examples but it does not specify normatively any actions that need to performed on receipt of these options. How does the MAG know if the LMA does not support Multipath binding? An LMA that does not implement this spec will not send back an error message. Why are there two different options? What happens if one of the options is present in the Proxy Binding Update but not the other?

Thank you for making the change to address Mirja's Discuss, which I agreed with. I also agree with your proposed resolution.