Minutes ======= Meeting : IETF104 Thursday, March 28, 2019 (CET) Time : 13:50-15:50 Thursday Afternoon session I (120min) Location : Grand Ballroom, Hilton Prague Chairs : Pascal Thubert pthubert@cisco.com Ethan A. Grossman eagros@dolby.com Responsible AD : Suresh Krishnan Live minutes : https://etherpad.tools.ietf.org/p/notes-ietf-104-paw Live feeds : https://datatracker.ietf.org/meeting/agenda/ Note Takers: Ethan Grossman Headcount: 77 (retrospective, from Blue sheet). 13:51 Intro and Status [ 5mn] * Note-Well, Blue Sheets, Scribes, Agenda Bashing * Status on existing documents (chairs) * if you haven't subscribed to the paw mailing list yet, DONT do so and wait for the new name to be announced at the end of this meeting. * Two drafts already published under the paw name. Third one under bier. 13:57 BoF presentation [15mn] Scope of the work (chairs) * We want to avoid the term deterministic, in wireless we prefer the notion of predictability. Determinism means repeatable, predictable activities. We extracted what we could do for wireless networks in that regard, and what we care about for our use cases is to deliver the packet within an expected time boundary, with enough quality to meet some requirements, like mean time between fault, like "never miss four packets in a row", something like that. In the end it's about machine-to-machine communications, allowing repeatable actions, removing elasticity so that machines can have automated systems using those networks. With wireless we can control the time of transmission, if we have a MAC/PHY that allows that, like TSCH or FDMA (as opposed to CDMA). Then we can control time of delivery, which we can do now with those technologies. What we can't do is control the delivery ratio, particularly in the ISM band where you have collisions from uncontrolled devices. We can use the various mechanisms to optimize that ratio. Mitigations depend on the threat. So it takes many forms of diversity (frequency, time, space, beam, coding, etc.) to alleviate the many causes of frame loss. Diversity helps alleviate the causes of packet losses, and we have to exploit it. * Scheduling also helps. * Bounded latency represents one of the critical aspects in the network * In wireless, scheduling avoids dead times with no transmission. Also allows nodes to go to sleep, saving battery life. A device does not have to wake-up periodically "just in case" a packet has to be received. The sender must be scheduled. * We can't prevent all collisions but we can reduce them dramatically by scheduling; scheduling helps in removing intra-network interference, by scheduling nodes so that they do not send when it is not their turn. * To achieve this, we have to adopt accurate strategies, different that we use in wired networks. * How we do it is why we are here. Understanding what those properties are and how to program them into a schedule. * Must select appropriate radios, and identify what must be specified to the intermediate hops. Identify diversity capabilities. * in wireless networks, the property of the radio is very location dependent. We have to separate clearly what we do at the routing vs. forwarding levels. We must continuously re-evaluate the properties of the paths (cost of re-evalaution vs. performance guarantees) * routing layer: identify and enable many ways to get there. Forwarding level: use the one out of the many possibilities that is available to give the best result at a moment in time. Must understand properties of path. More important to us than to DetNet. Continuously evaluate the path. If we use too many paths simultaneously we are interfering with the rest of the world and using too much power. If we use too few paths we will lose packets. This must be done by the forwarding layer, using OAM, not by the controller. * Janos Farkas: Are we to specify the scheduling part of the radio? * Pascal: We at least must specify what can be scheduled. Work with IESG to decide on data models to be built. We are the source of what goes into that data model. Because it is a specialized skill to know what is usefull and what can be specified for a given radio, and how. Decide which working group the work needs to be done in. * Janos Farkas: For 802.11, schedule is controlled by 802.11 spec. * Pascal: Yes, same for 3GPP. But we can ask them for a certain "service" and they'll say how to schedule it in their network for a single hop. They we will build a multi-hop network based on that. But it has to be abstract, e.g. TSCH is not FDMA, we need something that works for both. But at least LT and AX both look similar, both FDMA, so we can go around that. TDMA vs FDMA, that's the beginning of it. Use cases (Carlos Bernardos) * we detail some use cases, not all, but about what makes them each different from each other. * Focus on several use cases (i.e. amusement parks, industrial applications, audio/video) * Amusement Park requirements: support of heterogeneous traffic, scheduling across different technologies, IP-enabled interconnection across large areas. Multi-scale technology, small and large devices that all need to communicate. Some are moving, so need wireless. * Industrial application: process control loop with low end-to-end latency, losses and jitter. Also best effort traffic on same network. Loss and delay can be minimized, doesn't have to be perfect. Isolation between flows. Some moving parts. * Professional audio and video systems: uninterrupted playback, synchronized stream playback * different types of traffic and QoS criteria. Bounded low latency. Multipath for resiliency. * UAV control: e.g. drones, for surveillance, transport. Need for (obviously wireless) communications between UAVs. Self-configuration of network in flight. Control to AV path, also AV to AV paths. * Edge Robotics cotrol: we need a connectivity between the (mobile) robots and the intelligent controllers * service decomposition along a service chain * these are just some use cases, there are more (work on going: wireless interactive gaming, etc. * should we now insert additional cases? and detail their characteristics? * Ravi Ravindran: lots of these uses cases can be covered by 5G. Do we consider cellular in this work? * Pascal: single hop cannot do all of that. We are considering multi-hop. Like if you can do everything with TSN then you don't need DetNet. * Ravi Ravindran: are ad-hoc networks included? * Pascal: There is tension between ad hoc mobile and determinism. There are some parts in common, would like to eventually see work on that, mobility that can be deterministic, but we do not start with the premise of mobility or ad-hoc. Ravi Ravindran: UAV to UAV would be like ad hoc? Pascal: Yes, but in something like a factory you have enough control over the mobility of the devices that the network that you form doesn't change that much. * Juliusz Chroboczek: You said that some of your use cases can't deal with any packet loss. This is radio, how do you garantee Packet Delivery Radio? * Pascal: we never said that. We don't guarantee "no packet loss". Even wired systems don't say that. * J: If it isn't packet loss ratio, what are the properties you are guaranteeing? * Pascal: That is important work for this group, and it will never be expressed as "never lose a packet", and even "10 to the minus blah" is also not that useful. What is useful is something like MTBF, mean time between fault, so for example in a factory control loop the protocols expect that one or two packets may be lost, and the robots or whatever can carry through accordingly. But if there are more than say 4 packets in a row lost, then the robot will assume there is a loss of connectivity, and go into "emergency stop" mode. So what is necessary is to provide the necessary quality of service for the given application, usually expressed in these machine to machine systems as a MTBF, for example in this case the MTBF requirement is "never lose more than 3 packets in a row". * Juliusz Chroboczek: what keywords do I google to learn more about what you are describing? * Ethan Grossman: It sounds like you are saying "if you lose packets, how are you going to guarantee that all packets will arrive at the end?" The intent is that even if some packets are lost, through the use of redundant paths, in the end the stream will be delivered such that it will meet the MTBF requirements. * Juliusz Chroboczek: No, my question is not about mechanism. It is "if you are claiming to achieve a certain property, how is this property expressed?" * David Black: Key word to start with "Forward Error Correction". * Pascal: You are welcome to come write a draft on this, because it is a key question, not joking. There was a lot of work on this, starting with MTBF will help you find those papers from like 25 years ago, but anything about "ten to the minus" is not of interest to us here beause the factory can block with the 10 minus. * Diego Dujovne (Jabber): which lower layer wireless prootocols will you use for mobile applications? * Pascal: this corresponds to the next slides * Ajeya Gupta (Jabber): Are we looking to include vehicle ECU to ECU wireless communications? I am not talking about V2X/V2V type of communication. Rather, intra-vehicle communication. Pascal: Not listed in use cases, but reducing weight of wires and routing wires around the car are challenges there, so we hope that we can help with that. * David Black: What is the relationship betweem these use cases and the use cases draft written at DetNet? Use cases look very similar. * Ethan Grossman: In DetNet We decided to postpone work on wireless because how you achieve determinism is different for wireless, and most people in DetNet were more concerned with wired, and we needed to contain scope of DetNet. PAW is now picking up that thread. * David Black: The reason I ask is because some of the use cases seem very similar, just that in this case you need to use wireless. Ethan Grossman: Yes, that is exactly correct. * Erick Nordmark: Even if you have various diversity such as bouncing signals off of walls, guaranteeing that you won't exceed four packets lost in a row seems hard. Is there any research that shows that this is actually possible to guarantee? Maybe you could send us some pointers to that research, the state of the art? Pascal: Yes there are papers and theses. If you push that to the chances of ten minus so much, but a factory should be able to run for years without ever losing 4 in a row. * Erik Nordmark: So very low probability, like earthquake. * Erik Nordmark: What is the intended architecture? ARQ? Pascal: Yes, ARQ on a lot of steroids, more about that later. * Alissa Cooper: slide 8 says "losses cannot occur". This seems to be causing some confusion. I was confused. If that isn't a requirement, that might be where the confusion is coming from. * Carlos: In the slide I should have written it how Pascal said, but the draft says about not losing too many packets, with the example of 4; the draft is better about explaining this. Pascal: We need a draft on existing art on how to measure errors - just claiming 10 to minus 5 or ten doesn't help. We have the same issue for DetNet. ~14:25 related work at the IETF * DetNet (Lou Berger (presenter), Janos Farkas) [10mn] * I co-chair DetNet and also co-chair TEAS. * DetNet is about guaranteed delivery, meaning mitigate lossyness of network, via redundant traffic delivery. That is our architecture, so could also use Network Coding. We care about latency - packets for a given flow delivered within the time requirements. DetNet is a multi-hop solution, like IP. * wireless is still in the DetNet use case draft; the difference is that we are concerned with how to make a multi-hop network work, but not how to make a multi-hop RF system work, which sounds like it is what is going on here. Are we looking at a layered approach, standard IP multihop over some RF subnet where we have a clean layered solution? Or are we trying to optimize the RF network at the same time as optimizing our IP network? Analogy to DetNet relation to Ethernet: Is DetNet trying to optimize Ethernet at the same time as IP? * In other words, could we make an abstraction of the lower layers? Or do we have to adopt a very link-dependent design? I'm asking the question, and this is for the group to decide. Pascal: With DetNet you didn't have to interact with the TSN Ethernet layer too deeply, we would expect a similar extent of integration with radio layers here. * Pascal: trying to integrate deeply with the radio layers would be even more difficult than DetNet integrating more deeply with TSN. What I am asking for is a clean inheritance from DetNet and 6TiSCH, pushing harder to extend DetNet into the wireless world. * Lou: layered approach, makes a lot of sense to me. * re-details the charter of the DetNet WG, and the progress of the WG. Agreed on data place, but taking a long time to document it. Model of control is stopping at a YANG model. Control may be a recharter, or moved to other WGs. * charter is the multihop network, not the specifics of a given network. * the focus is on the network layer * DetNet has higher delivery requirements that other WGs at the IETF, even TEAS (Transport Engineering). TEAS (overall architecture) and CCAMP (binding to specific technologies). Maybe that is how to split this here - maybe PAW is about how to map DetNet down to radio layers. Pascal: That is how I see it. Lou: That's fits in with how we have done it in other groups. A lot of coordination to be done there, as with other WGs. * Network layer down, Transport and above are out of scope for DetNet. Coordinating with various other groups, chairs, IESG. * Pascal: You list 6TiSCH, but 6TiSCH will not work on the DetNet piece. 6TiSCH will probably end here. PAW will be a spinoff, more specialized. PAW is not in competition with 6TiSCH, but rather its continuation with different means. DetNet will be creating requirements for control plane, then passing the work to groups that own the protocols, e.g. TEAS, or CCAMP after initial set of mapping to various technologies. Currently doing that work in DetNet so is possible that would continue, but what you are suggesting is breaking off the wireless part into a different WG; there are pluses and minuses; that will require a lot of coordination, we will need to talk to each other to deconflict. * Anyway, this is a non-WG-forming BOF, so we should not be talking about this. We can worry about that in Montreal. 14:38 * CCAMP (Daniele Ceccarelli (presenter), Fatai Zhang) [ 5mn] Co-chair of CCAMP. * CCAMP is about the lower layer: control plane and measurement plane for everything non-IP, at the lower layer. Switches, wireless, e.g. currently working on microwave. * standardization of the control and management planes for all the things which are not IP (TDMA, wireless, uwave links, etc.) * currently working on TE technologies for non-packet technologies: G-MLPS. * Owns the LMP protocol, link management. * YANG, a lot of models. * Pascal: Already working on microwave. * Charlie Perkins: Are LMP measurement techniques shown in slide for CCAMP applicable to wireless links? * Lou Berger: MANET working group working on DLAP, which is LMP for wireless. Bad coordination since DLAP didn't know about LMP. LMP could have been extended to be DLAP, but it was too late. So now have LMP for wired and DLAP for wireless. 14:44 * 6TiSCH (Thomas Watteyne, Pascal Thubert (presenter)) [ 5mn] Pascal (as 6TiSCH co-chair.) * you can allocate time-frequency ressources to a particular transmission. * DetNet provides deterministic service on the same network with best effort ("stochastic") IP. What 6TiSCH did was to enable IPv6 for best effort traffic in the slots that are not used at a given time by deterministric traffic. That is, providing best effort service on a deterministic MAC. But 6TiSCH will not address the deterministic part of it, i.e. 6TiSCH will not define how detnet is applied over TSCH. That was due to being in the wrong Area, and not having the right people in the room. Close to done with that work, and closing WG. * 6TiSCH wrote an architecture that supports both deterministic and non-deterministic traffic, and pushed to DetNet. However, wireless was not first priority for DetNet. * * We want to make the solution more generic, a combination of 6TiSCH and DetNet and to propose something independent from TSCH, specialized to our problem. * Ravi Ravindran: what workloads are you considering? You mentioned audio and video, but 15.4 is few 100's of kbits/s. Are you also considering higher bandwidth links? Are you considering constrained devices mixed with high bandwidth links? * Pascal: 15.4e (TSCH mode). constrained network. * Ravi Ravindran: it may be a conflict with some applications (video, etc.) which require high bandwidth - so this isn't for video right? * Pascal: 6TiSCH is for sensor networks, factory automation, process control, in industrial environment. * Carlos: we are not limited to a specific radio technology * Pascal: We will have different radios for different needs * Mina Rady (Jabber): concerning spatial reuse of channels in the mesh, are you assuming a certain spatial organization of the mesh? (i.e. not just a randomly distributed mesh)? * Pascal: No, not assuming that. 6TiSCH allocation of channels is not centrally allocated, it is distributed, done by screening the air. We find timeslots unused in our area and use them for peer to peer communication. The topology is discovered in the field and network organized from that. We alllocate the resources adaptively * Remi Liu: 6TiSCH is hop by hop, so the motivation to hive PAW is to centralize the routing and scheduling? Pascal: Yes. Lou: You clarified what 6TiSCH does, can you clarify what PAW is to do? Pascal: We expect this group to complete the dot of the 6TiSCH architecture, which describes both stochastic and deterministic flows. But 6TiSCH WG was focused on stochastic flows, and we expected to inherit from the work of detnet for deterministic flows. As we thought about it we decided to disband 6TiSCH and start a new group to do the deterministic part since 6TiSCH was not the right group to do it. * Lou: would be good to clarify if this is going to be deterministic for 6TiSCH or for other type of radios. * Pascal: Not just for 6TiSCH. Another reason that 6TiSCH is not appropriate for this work is that we want to extend the work to other radio technologies. We are presenting here four different radio technologies that we are addressing. 14:54 Technologies * 5G URLLC (Bikramjit Singh) [10mn] Works for 3GPP, not IETF. Working on physical technology including grant-free. * URLLC targets low latency applications, typically 0.5ms in Up/Download, with a six nines reliability in Rel-16. * New Radio (NR) defined in Rel-15 (2018). * timeslot used to be 1us in 4G, submultiples in 5G (with larger subcarrier spacing). * semi-persistent scheduling for grant free access. * Premeption and Fast HARQ help to guarantee a low delay and high reliability * lower code rates allow for better reliability. Automatic repetition. Packet duplication. Accurate time delivery for synchronization. * Synchronization represents a key characteristic (typically, we need a few hundreds of nanoseconds) * Configured grant: transmission opportunities are pre-allocated according to a periodic sechdule. Relates to latency. * Pascal: when we take a look at the architecture, that's time to see if we cannot do something at the layer 3 to solve this problem. We see that in this picture that 5G is talking to TSN directly, and it would be better to generalize this, so that it went through IP, i.e. DetNet. * Lou: Note that they are not talking about network coding or network FEC, presumably since they are solving it at Layer 1. But the best way to do that for a multi-hop network may be to do it across multiple hops, different than simple 1+1 e.g. looking at network coding, maybe in a separate group. 15:07 * LDACS (Corinna Schmitt, remote) [10mn] * besides the previous use cases, we have also the aeronautic applications. Wireless communications are extensively used for air traffic management * replace analog with digital communication. Many aviation agencies intersted in this. * more traffic looking for new frequency bands. * LDACS (Digital Aeronautics Communication Systems), in L-band. * the different planes have to communication with each other, and with the ground stations * For more info on this system, see references at the back of the presentation. * Centralized approach when plane on the ground, decentralized in flight. * Successful deployments of prototypes 15:15 * 802.11ax and EHT (Dave Cavalcanti, remote) [10mn] * Update on TSN in IEEE 802.11 * tradidtionaly 80.11 focused on delivering more bandwidth. CSMA makes latency unpredictable. * Controlled latency and higher reliability needed, some work in 802.11ax. 11be approved, just starting, higher throughput. * 11ax brings centralized scheduling (multi-user OFDMA) to WiFi, the STA send a frame wihout an EDCA access . We can now schedule multiple users without contention, and we remove the random access delay. * 11ax example: with 11ac, multiple users have to be scheduled consecutively, with a medium access delay (e.g. 758µs for 5 users compared with 178µs if the user is alone) * with 11ax, con schedule to transmit them all together in a single PPDU with OFDM. * from 20MHz to 160 MHz bandwidth, room for many users. In controlled environments, can provide low latency and high reliability. * 11be use cases: gaming, industry automation, real time video, etc. * we have to address the worst case latency. And that's now integrated in the 11be working plan * there already is some TSN support in 11ax. (synchronization) * more of TSN features in 11be. * 11be can support time-sensitive applications in managed network scenarios. integrates with Ethernet since both are 802 protocols. * 15:23 * IEEE 802.15.4 TSCH / 6TiSCH Tracks (Xavier Vilajosana, remote) [10mn] * will describe label switching and tracks. * TSCH can be tuned for latency, for redundancy, for bandwidth. * acknowledgement in a each slot. * 6TiSCH took TSCH to use it for building an IP mesh network. * work at 6TiSCH does not address how to build paths with deterministic properties. * 6TiSCH allows building G-MPLS networks. * label switching helps to link flows and resources, it borrows the same ideas as RSVP-TE, etc. * 6TiSCH defined notion of track. * the node which is the source of the flow assigns a track id, and is the owner of the track. Typically, a collection of cells (bandwidth resource) is allocated to a track. * we may use G-MPLS in transport mode on top of a TSCH network. However, it corresponds to a high overhead for packet processing. * Pascal (as the 6TiSCH chair): 6TiSCH defined all these notions, but we never implemented any of these protocols. This is why we are in the same room now. * Lou: I thought we would be mapping DetNet over 6TiSCH and other technologies. But I just heard you say we are here to build a control plane for 6TiSCH. * Pascal: no, 6TiSCH is just one radio. 6TiSCH is already conserving multihop. * Lou: So the idea is to solve the general problem of DetNet of wireless, and reuse that solution to solve 6TiSCH multi-hop. * Pascal: Yes, 6TiSCH is a user of what we are building in PAW. 6TiSCH is the only radio that presents this problem as a mesh network - it is multi-hop. * 15:33 drafts and WIP * draft-thubert-paw-for-tisch (Pascal (presenter) / Xavier) [ 5mn] How can we extend DetNet to solve the kind of problems presented by Xavi? * shows packet replication on the 6TiSH schedule. Also replicate packet elimination. But there is more needed for wireless. * overhearing allows the nodes to exploit the broadcast nature of radio transmissions. Two receivers are programmed to capture the same packet * constructive interference: send tightly synchronised signals from diferrent transmitters to increase quality of signal at the receiver. * Also ARQ (retry request). Controller can program enough timeslots so that ther is enough retransmission opportunities for the requested PDR. * we should use all the forms of diversity (spatial, time, frequency) to improve the reliability. * we have also to not deplete our energy to not explore all the possibilities. We have to carefully monitor the quality to adopt the most relevant strategies * Janos Farkas: This sounds like unfinished 6TiSCH work? Could be done in 6TiSCH? * Pascal: right, unfinished drafts, expected input from DetNet. Besides, 6TiSCH is an interior IPv6 over foo crowd. Here, we're talking exterior, e.g. CCAMP, DetNet. We needed to change area. * Would like to combine contributors from various Areas that are interested in deterministic performance. * Suresh Krishnan: Janos has a good point. Currently only identifying what is the work that needs done. * My main point for this is to determine if there is a problem to be solved, is there interest. Then parcel out work to this group, or to other groups, if needed. * Lou: 6TiSCH work as described here belongs in CCAMP, they have done a lot of that. * Mina Rady (jabber): I am curious for your thoughts on the feasibility of such optimizations in a distributed (hop by hop) manner * Pascal: if we're talking about cells allocations in 6TiSCH, etc., this is already done in 6TiSCH. Please use the mailing list 15:43 * draft-papadopoulos-paw-pre-reqs (Georgios Papadopoulos) [10mn] * PRE problem statement * take advantage of broadcast nature of wireless medium. * PRE recent results * Alternative Parent selection. Careful to control flooding when doing packet replication upward. Benefit of knowing common ancestors. * Promiscous overhearing need to bypass MAC address filtering. * We have then to eliminate the duplicated packets * We have to select carefully select the next hops (parents) to maximize the probability to have common parents (for duplicated packets elimination) * Proposed 3 different selection modes. * the jitter can then be neglected compared with classical single path forwarding * partially implemented in Contiki, code on GitHub, Wireshark dissectors available. * Pascal: we consider a distributed routing protocol, compared with detnet and TSN. We construct distributively a track. We control the replication to not flood the network. 15:51 Next Steps (skipped for lack of time) * Going through the proposed charter (Chairs) [20mn] * see the proposed charter on-line. Please subscribe to the ML 15:52 * BoF in Montreal, renaming to SPAWN, AOB [QS] * PAW (too near of PAWS, which deals with white space) is proposed to be renamed SPAWN, from the discussion on the ML * Suresh: I found two interesting pieces of work to do: 1) Mainly OAM work, like the packet loss eval, balances the resiliency with energy usage 2) Signaling to do paw flow, with spatial time and frequency constraints. We need to figure out how to get these done, like if this should be a new WG or just parcel out the work to other groups and have some loose coordination between the groups. I was hoping we could do this today but we ran out of time. We need something more specific, more socialized, with DetNet, TEAS and all. As Lou said, some of this already has home in Routing. We have to figure out if this requires a new WG or not. We have time to do that between now and Montreal. We need to make sure everyone understands the deliverables and where they could be done. * Pascal: let's discuss on the SPAWN mailing list. SPAWN will be created very soon. Please register to the novel SPAWN ML. 15:54 meeting adjourns