Minutes IETF97: banana
minutes-97-banana-00

BANdwidth Aggregation for interNet Access (BANANA) BOF
IETF 97, Seoul, South Korea
Thursday Afternoon Session I (13:30 - 15:00)

Chairs:
Margaret Cullen <mrcullen42@gmail.com>
Brian Trammell <ietf@trammell.ch>

## Administrivia (5 mins)

The discussion is in context of these questions:
- Is the problem statement valid?
- Is there IETF scope to do this?
- Is there interest?
- Should the work be joined with other WGs, or is there need for a new WG?

## BOF Scope and Problem Description (Margaret Cullen)

Bandwidth aggregation and failover solutions for multi-access, without end
clients not being multi-access aware. Get higher bandwidth through
aggregation, and reliability through failover Today, traffic is sent through
the default path today, and we don’t use the other path. Any given flow won’t
fail over to the other access link if the first goes down.

Solution scenarios:

1. Single operator, in which multiple accesses are provided by one party. That
operator splits the traffic for you, and recombines it before sending out into
the internet. Not hard to solve, but more limited. Requires LTE/Cable/DSL all
from the same source.

2. Proxy/NAT service in the network to allow aggregation of local links into a
cloud server. This allows use of different providers, but does require a
single session with a box in the cloud.

3. Aggregate links edge-to-edge. A Content Provider Edge by the content
receives the multiple connections and re-aggregates the content before passing
on to the end device.

Proposals:

- GRE tunnel bonding traffic shared per-packet and tunneled to de-aggregation
  point
- MPTCP proxy solution to have an MPTCP connection to an aggregation service.
- Easy to do TCP, can possibly add UDP.
- Multipath bonding at layer 3; research work for edge-to-edge UDP
- MAG multipath binding, based on Mobile IP. May not work for all scenarios.
- Bonding solution for hybrid access (3GPP solution, one network is 3GPP,
  requires single operator)

Challenges include:

- performance (knowing where the bottleneck is to be useful);
- home scenarios only have a few flows;
- some traffic must have policies for legal reasons;
- managing tunnels or using proxies is not easy;
- configuration needs to be done;
- routing back across links;
- making sure we don’t ossify on TCP;
- make sure there is security;
- and allowing multipath aware end-to-end protocols to work without a proxy

Questions?

Aaron Falk: Surprised to not see any mentions of ICMP
Margaret: Good point

Pete Resnick: These are all about something in front of your host, to something
in front of the other end. Why not have it be more host based? Margaret: If you
can do MPTCP all the way through, that’s even better. We want a way to help
unchanged devices get multi-path support.

Sri Gundavelli: I believe Mobile-IP does map to all of the scenarios.
Bob Hinden: There are load-sharing RFCs that do solve these problems (RFCs 4311
& 8028) Is this v4? v6? Margaret: Probably has to be both families.

Suresh: Adding a challenge: congestion control is important

Rick Taylor: You said it’s difficult to discover quality of the network. The
first hop is easier than the rest.

## BANANA Solution Space & Experiences

### GRE Tunnel Bonding Solution Overview (Mingui Zhang)

For each link, there is a GRE tunnel set up. They are bonded together to form
a single bonding connection. Single provider solution. Can use as if it is a
traditional IP link. Uses coloring mechanism to route through LTE vs DSL.
Packets reordered at egress. Sequence number added to all packets to allow
reordering, supporting v4 and v6. Creates an IP-level overlay. A new GRE
protocol type has been assigned by IEEE for tunnel bonding.

### Deployment Experience (Nicolai Leymann)

Deutsche Telekom has a deployment of the GRE solution. Integrates DSL and LTE.
Between 1Mbit and 200Mbit. Trying to go to over 550Mbit. 150K deployment base
in 2015, after 2014 roll out. 1 contract for entire solution, data for both
links. Trying to make solution available to all devices in a home without
modification. Support both TCP and UDP, v4 and v6. Tunnel is v6 only. A single
session should be allowed to take the entire bandwidth. Puts traffic first on
fixed line (DSL) then moves to LTE when needs more. Certain traffic is
bypassed; user can choose how to filter based on names or addresses, or
traffic classes Solution is not bound to the carrier network infrastructure
GRE was chosen for expediency of deployment

### MPTCP Proxy (SungHoon Seo)

KT’s GiGA LTE deployment. Launched MPTCP solution in 2015, which aggregates
GiGA Wi-Fi and LTE access. Works for smartphones by accessing MPTCP proxy
gateways to achieve as much as 1Gbit. Uses SOCKSv5 over MPTCP; 2 subflows per
session, one over LTE and one over Wi-Fi. Does MP_CAPABLE on LTE, the MP_JOIN
on Wi-Fi. (So LTE is the primary subflow) Supports any apps using TCP by
routing through MPTCP proxy.

Jana Iyengar: Is MPTCP supported in the kernel?
SungHoon: Yes, and the SOCKS in in the library.

Tommy Pauly: Why do we need to only start the MPTCP connection over LTE? Why
not start over Wi-Fi if LTE not available? SungHoon: Policy decision, and
customers usually have LTE

Sri: Why use SOCKS proxy?
SungHoon: Need the client information

## Related Research - Multipath Bonding at Layer 3 (Brian Trammell)

Research project to do MPTCP to aggregate LTE and DSL but for UDP. But UDP
already lets you do multipath! Need to figure out how to schedule the traffic
over the links. Goal was to fill the fixed link first, and use the mobile link
for excess demand.  Used weighting scheme to schedule the traffic Works
relatively well in measurements One concern is tuning the parameters to play
with TCP traffic nicely. Varies depending on TCP congestion control being
used on the network. Not using a standardized framing for this, but can in the
future Future work: interop with MPTCP proxies, apply GRE?, allow disability
reordering sensitivity

## Related Work in Other SDOs: Broadband Forum TR-348 (Dave Allan)

Focus on an operator that controls both fixed and wireless access.
Goals:
- Could deploy by sending a box that works over wireless only first, then can
be hooked up to fixed - Increase bandwidth - Improve reliability Hybrid may be
more economic than pulling fiber everywhere Only going forward with a
double-ended solution (have both sides of proxying) New document is looking at
MPTCP, MIP/PMIP, and network-based tunnels to put the technologies together in
some form to create a solution. Looking at the deployment and operation issues.

## Open Discussion

Aaron Falk: Work done before bonding satellite and broadband. There’s a lot
satellite connectivity still at the fringes of the network. We will need to
look at disparities in latencies between links. Rick Taylor: +1. The latencies
on the different links will kill you. Margaret: Some of Brian & Mirja’s work
shows that this effect will vary based on the application.

?: Worked on similar systems with train access, with half a dozen wifi links.
High dynamics of links going up and down quickly. Those should be kept in
sight. David Schinazi: Happy that this discussion is looking at standards-based
solutions. How does this interact with other groups look at protocol
ossification? Is it in scope to make sure we can keep innovating within these
tunnels? Brian Trammell: Anything that only works with TCP or non-congestion
controlled UDP is a non-starter. Same for QUIC-only, or anything else.

Aaron Falk: Lot of work done by operators on this so far. Are operators
interested in standards-based solutions? Or do they only like proprietary?
Nicolai: Our interface is open. We’re very interested in standardizing. We just
want to make sure it meets our needs. That includes supporting all protocols.
I’d prefer to have only one solution for all the protocols, and not do the work
again for every protocol. Aaron: Are you okay with not controlling which
packets go where? Nicolai: We want control (UE can have some too, but not too
much)

?: I heard that GRE was an independent stream. The MPTCP work seems to be
picked up by the WG. Margaret: Not sure if MPTCP was going to do the proxy, and
didn’t want to do signaling. MIP is in DMM. GRE is only an independent draft.
Mirja: For MPTCP, part of it was going to be based on this discussion. The
charter can document usage, but probably won’t look at signaling. This work is
not specific to MPTCP. ?: My impression at the MPTCP session was that this was
more favorable for taking the proxy work. We do want this in the IETF. Suresh:
I would like to use the time to agree on a problem, and see if IETF should
solve this. What is the definition of the problem? How does it run over the
Internet?

Jana: It seems like there is a problem; the operators are deploying their own
solutions—do they need to agree really? How long do we expect the problem of
DSL + LTE being the state of the art? Dave Allan: We have 6 or 7 operators
trying to do the same thing. Not uniform, but some common patterns certainly.
Jana: But do we need interoperability between boxes from multiple solutions?
Where do the CPEs come from?

Rick: We need to build this with open standards; I want to build one end. This
should be somewhere in the IETF.

Blaine McDonnel: All three possibilities have different applications. Would
like to hear how much traffic and subscribers would actually be combining?

Tommy: I’m concerned that if operators are saying they want one protocol level
for multipath aggregation in these cases, you are at odds with the desire to
long-term have end-to-end multipath protocols (like MPTCP and QUIC) that don’t
offer the same control natively as aggregating solutions. Perhaps we do have
room to work on a Path-level protocol level that is common for MPTCP and QUIC,
etc.

Spencer: If you have a multi-operator solution, do you still need a single
operator solution that is distinct.

?: There’s clear motivation and concrete requirements to do this work. I want
to emphasize that the output will not assume operator networks, or the current
deployment access networks.

Wim: There is work to be solved here, that is being done across the IETF.
Better to do the work in each working group for the protocols, such as MPTCP.

Liang: We used to deploy dual uplink in China, but not for hybrid access; only
used to transition to fiber. Not doing it now, because it wasn’t that cost
effective. The group should emphasize multiple wireless links more than wired.
Also glad that latency was pointed out.

Alan Ford: In MPTCP, we need a better view of the problems we need to solve.
There are three proposals bouncing around, but we want a clearer problem
statement.

## Questions & Next Steps (Chairs & ADs)
Hums!
- Is the problem statement clear, well-scoped? 50/50
- Is there interest among people to work on this? ~35 people raised their hands
- How many people are already working on this in an IETF context? ~20 people
working on drafts

Suresh: I would like these people to talk about this on the mailing list. I
fear that everyone has their favorite pony they want to promote. We should get
a more solid idea about people’s stance before going into solutions.

Brian: Please contribute to the list to discuss about this and clarify the
problem statement and goals.