BANdwidth Aggregation for interNet Access (BANANA) BOF IETF 97, Seoul, South Korea Thursday Afternoon Session I (13:30 - 15:00) Chairs: Margaret Cullen Brian Trammell ## Administrivia (5 mins) The discussion is in context of these questions: - Is the problem statement valid? - Is there IETF scope to do this? - Is there interest? - Should the work be joined with other WGs, or is there need for a new WG? ## BOF Scope and Problem Description (Margaret Cullen) Bandwidth aggregation and failover solutions for multi-access, without end clients not being multi-access aware. Get higher bandwidth through aggregation, and reliability through failover Today, traffic is sent through the default path today, and we don’t use the other path. Any given flow won’t fail over to the other access link if the first goes down. Solution scenarios: 1. Single operator, in which multiple accesses are provided by one party. That operator splits the traffic for you, and recombines it before sending out into the internet. Not hard to solve, but more limited. Requires LTE/Cable/DSL all from the same source. 2. Proxy/NAT service in the network to allow aggregation of local links into a cloud server. This allows use of different providers, but does require a single session with a box in the cloud. 3. Aggregate links edge-to-edge. A Content Provider Edge by the content receives the multiple connections and re-aggregates the content before passing on to the end device. Proposals: - GRE tunnel bonding traffic shared per-packet and tunneled to de-aggregation point - MPTCP proxy solution to have an MPTCP connection to an aggregation service. - Easy to do TCP, can possibly add UDP. - Multipath bonding at layer 3; research work for edge-to-edge UDP - MAG multipath binding, based on Mobile IP. May not work for all scenarios. - Bonding solution for hybrid access (3GPP solution, one network is 3GPP, requires single operator) Challenges include: - performance (knowing where the bottleneck is to be useful); - home scenarios only have a few flows; - some traffic must have policies for legal reasons; - managing tunnels or using proxies is not easy; - configuration needs to be done; - routing back across links; - making sure we don’t ossify on TCP; - make sure there is security; - and allowing multipath aware end-to-end protocols to work without a proxy Questions? Aaron Falk: Surprised to not see any mentions of ICMP Margaret: Good point Pete Resnick: These are all about something in front of your host, to something in front of the other end. Why not have it be more host based? Margaret: If you can do MPTCP all the way through, that’s even better. We want a way to help unchanged devices get multi-path support. Sri Gundavelli: I believe Mobile-IP does map to all of the scenarios. Bob Hinden: There are load-sharing RFCs that do solve these problems (RFCs 4311 & 8028) Is this v4? v6? Margaret: Probably has to be both families. Suresh: Adding a challenge: congestion control is important Rick Taylor: You said it’s difficult to discover quality of the network. The first hop is easier than the rest. ## BANANA Solution Space & Experiences ### GRE Tunnel Bonding Solution Overview (Mingui Zhang) For each link, there is a GRE tunnel set up. They are bonded together to form a single bonding connection. Single provider solution. Can use as if it is a traditional IP link. Uses coloring mechanism to route through LTE vs DSL. Packets reordered at egress. Sequence number added to all packets to allow reordering, supporting v4 and v6. Creates an IP-level overlay. A new GRE protocol type has been assigned by IEEE for tunnel bonding. ### Deployment Experience (Nicolai Leymann) Deutsche Telekom has a deployment of the GRE solution. Integrates DSL and LTE. Between 1Mbit and 200Mbit. Trying to go to over 550Mbit. 150K deployment base in 2015, after 2014 roll out. 1 contract for entire solution, data for both links. Trying to make solution available to all devices in a home without modification. Support both TCP and UDP, v4 and v6. Tunnel is v6 only. A single session should be allowed to take the entire bandwidth. Puts traffic first on fixed line (DSL) then moves to LTE when needs more. Certain traffic is bypassed; user can choose how to filter based on names or addresses, or traffic classes Solution is not bound to the carrier network infrastructure GRE was chosen for expediency of deployment ### MPTCP Proxy (SungHoon Seo) KT’s GiGA LTE deployment. Launched MPTCP solution in 2015, which aggregates GiGA Wi-Fi and LTE access. Works for smartphones by accessing MPTCP proxy gateways to achieve as much as 1Gbit. Uses SOCKSv5 over MPTCP; 2 subflows per session, one over LTE and one over Wi-Fi. Does MP_CAPABLE on LTE, the MP_JOIN on Wi-Fi. (So LTE is the primary subflow) Supports any apps using TCP by routing through MPTCP proxy. Jana Iyengar: Is MPTCP supported in the kernel? SungHoon: Yes, and the SOCKS in in the library. Tommy Pauly: Why do we need to only start the MPTCP connection over LTE? Why not start over Wi-Fi if LTE not available? SungHoon: Policy decision, and customers usually have LTE Sri: Why use SOCKS proxy? SungHoon: Need the client information ## Related Research - Multipath Bonding at Layer 3 (Brian Trammell) Research project to do MPTCP to aggregate LTE and DSL but for UDP. But UDP already lets you do multipath! Need to figure out how to schedule the traffic over the links. Goal was to fill the fixed link first, and use the mobile link for excess demand. Used weighting scheme to schedule the traffic Works relatively well in measurements One concern is tuning the parameters to play with TCP traffic nicely. Varies depending on TCP congestion control being used on the network. Not using a standardized framing for this, but can in the future Future work: interop with MPTCP proxies, apply GRE?, allow disability reordering sensitivity ## Related Work in Other SDOs: Broadband Forum TR-348 (Dave Allan) Focus on an operator that controls both fixed and wireless access. Goals: - Could deploy by sending a box that works over wireless only first, then can be hooked up to fixed - Increase bandwidth - Improve reliability Hybrid may be more economic than pulling fiber everywhere Only going forward with a double-ended solution (have both sides of proxying) New document is looking at MPTCP, MIP/PMIP, and network-based tunnels to put the technologies together in some form to create a solution. Looking at the deployment and operation issues. ## Open Discussion Aaron Falk: Work done before bonding satellite and broadband. There’s a lot satellite connectivity still at the fringes of the network. We will need to look at disparities in latencies between links. Rick Taylor: +1. The latencies on the different links will kill you. Margaret: Some of Brian & Mirja’s work shows that this effect will vary based on the application. ?: Worked on similar systems with train access, with half a dozen wifi links. High dynamics of links going up and down quickly. Those should be kept in sight. David Schinazi: Happy that this discussion is looking at standards-based solutions. How does this interact with other groups look at protocol ossification? Is it in scope to make sure we can keep innovating within these tunnels? Brian Trammell: Anything that only works with TCP or non-congestion controlled UDP is a non-starter. Same for QUIC-only, or anything else. Aaron Falk: Lot of work done by operators on this so far. Are operators interested in standards-based solutions? Or do they only like proprietary? Nicolai: Our interface is open. We’re very interested in standardizing. We just want to make sure it meets our needs. That includes supporting all protocols. I’d prefer to have only one solution for all the protocols, and not do the work again for every protocol. Aaron: Are you okay with not controlling which packets go where? Nicolai: We want control (UE can have some too, but not too much) ?: I heard that GRE was an independent stream. The MPTCP work seems to be picked up by the WG. Margaret: Not sure if MPTCP was going to do the proxy, and didn’t want to do signaling. MIP is in DMM. GRE is only an independent draft. Mirja: For MPTCP, part of it was going to be based on this discussion. The charter can document usage, but probably won’t look at signaling. This work is not specific to MPTCP. ?: My impression at the MPTCP session was that this was more favorable for taking the proxy work. We do want this in the IETF. Suresh: I would like to use the time to agree on a problem, and see if IETF should solve this. What is the definition of the problem? How does it run over the Internet? Jana: It seems like there is a problem; the operators are deploying their own solutions—do they need to agree really? How long do we expect the problem of DSL + LTE being the state of the art? Dave Allan: We have 6 or 7 operators trying to do the same thing. Not uniform, but some common patterns certainly. Jana: But do we need interoperability between boxes from multiple solutions? Where do the CPEs come from? Rick: We need to build this with open standards; I want to build one end. This should be somewhere in the IETF. Blaine McDonnel: All three possibilities have different applications. Would like to hear how much traffic and subscribers would actually be combining? Tommy: I’m concerned that if operators are saying they want one protocol level for multipath aggregation in these cases, you are at odds with the desire to long-term have end-to-end multipath protocols (like MPTCP and QUIC) that don’t offer the same control natively as aggregating solutions. Perhaps we do have room to work on a Path-level protocol level that is common for MPTCP and QUIC, etc. Spencer: If you have a multi-operator solution, do you still need a single operator solution that is distinct. ?: There’s clear motivation and concrete requirements to do this work. I want to emphasize that the output will not assume operator networks, or the current deployment access networks. Wim: There is work to be solved here, that is being done across the IETF. Better to do the work in each working group for the protocols, such as MPTCP. Liang: We used to deploy dual uplink in China, but not for hybrid access; only used to transition to fiber. Not doing it now, because it wasn’t that cost effective. The group should emphasize multiple wireless links more than wired. Also glad that latency was pointed out. Alan Ford: In MPTCP, we need a better view of the problems we need to solve. There are three proposals bouncing around, but we want a clearer problem statement. ## Questions & Next Steps (Chairs & ADs) Hums! - Is the problem statement clear, well-scoped? 50/50 - Is there interest among people to work on this? ~35 people raised their hands - How many people are already working on this in an IETF context? ~20 people working on drafts Suresh: I would like these people to talk about this on the mailing list. I fear that everyone has their favorite pony they want to promote. We should get a more solid idea about people’s stance before going into solutions. Brian: Please contribute to the list to discuss about this and clarify the problem statement and goals.