Version: September 17, 2021
Thanks to our note takers, notably Ethan Grossman
WG ICS: https://datatracker.ietf.org/meeting/upcoming.ics?filters=detnet
Time Zone Converter: https://www.timeanddate.com/worldclock/converter.html?iso=20210913T130000&p1=1440&p2=tz_pt&p3=tz_mt&p4=tz_et&p5=tz_bst&p6=tz_cest&p7=tz_cst-china&p8=tz_jst
Session ICS: https://datatracker.ietf.org/meeting/interim-2021-detnet-01/sessions/detnet.ics
Webex: https://ietf.webex.com/ietf/j.php?MTID=m959a3dfa459a201a2b7555463ec2d647
Jabber: xmpp:detnet@jabber.ietf.org?join
Jabber Logs: http://jabber.ietf.org/logs/detnet
Materials: https://datatracker.ietf.org/meeting/interim-2021-detnet-01/session/detnet
Note taking: https://notes.ietf.org/notes-ietf-interim-2021-detnet-01-detnet
Presenter: Chairs
Presenter: Peng Liu
Draft: https://datatracker.ietf.org/doc/draft-geng-detnet-requirements-bounded-latency/
https://datatracker.ietf.org/doc/draft-du-detnet-layer3-low-latency
Yaakov Stein: Regarding slide on applications. Delay is aspect of both, (Yaakov) knows both well. Phase control less prevalent, other methods better, not tight delay bound, variation is more important. Based on 60Hz, mechanical things moving, so talking about 10’s of ms - critical delay, but ms not us.
Toerless Eckert: To answer Yaakov: Tests shown on slide were about jitter not latency, due to q’ing mechanism used. Constrained jitter, but latency is long due to “speed of light” latency, not in the useconds.
Balazs Varga: Good numbers on the tests shown. Which data planes were used?
Peng Liu: First and second based on IP data plane. Last is not traditional network, was just something they built.
Balazs Varga: Have you used any of the queuing mechanisms discussed in your slide?
Peng Liu: Yes, first one with freq sync. Related to CQF but different.
Yaakov Stein: Issue of delay variation rather than delay - important for both applications shown here, but once have long xmission delay, can we just use a buffer to absorb jitter (delay variation)? Also questions on srtsn - want to mention that scalability is a driving force behind srtn - deterministic in theory but determinism is statistical, not absolute.
Toerless Eckert: What is important part?
Peng Liu: Delay variation is the important part.
Toerless Eckert: OK, so will be discussed in my (Toerless) presentation.
Presenter: Toerless Eckert
Draft: https://datatracker.ietf.org/doc/draft-eckert-detnet-bounded-latency-problems
David Black: re first line on comparison table. Which data planes would be used? How much modification to data plane is acceptable, e.g. additional headers and labels?
Toerless Eckert: As little as possible, starting with MPLS since most pragmatic starting place. Would like better QoS header, maybe could bring to MPLS design team to make more concrete. Would be good start for IPv6 extension header.
David Black: Question was not about what can be done, but what are req’ts that constrain the solution(s)?
Toerless Eckert: 3 bits of exp header.
David Black: Not the minimum modifications that do something - what are the constraints imposed by the problems being posed?
Toerless Eckert: Don’t understand?
David Black: This has a “solution in search of a problem” flavor. Compare with respect to DetNet data plane: like “we don’t need new headers, just using diffserv field”
Toerless Eckert: We want to use dataplanes that are predominant in service providers. Use them effectively same as we do with preof.
David Black: I am looking for a discussion of problem space in terms of header introduction & modification requirements (limitations). There are a lot of assumptions here - are header modifications out of scope?
Toerless Eckert: We need to weigh pros and cons of diff’t mechanisms. If stateless, will require in-packet info, like L4S with new header bits. If so then we need new extensions. E.g. as done in 6MAN, MPLS - we need to evolve to serve these applications better.
David Black: To make this work it looks like it may need header modifications - need to know what those are.
Lou Berger: Need to be having this discussion to determine if there is a problem that the working group wants to work on, so we will continue after next presentation.
Presenter: Jean-Yves Le Boudec
Draft: https://datatracker.ietf.org/doc/
Toerless Eckert: Great preso. On TSN ATS stuff called “hybrid” - mean don’t have per flow scheduling, just per class. But is on order of per class per interface, so works with 12-24 ports but for city size may have 8 thousand scheduling entities, so need to aware of this and not lose in comparison.
Jean-Yves Le Boudec: Agree.
Greg Mirsky: For definition of delay, in context of RFCs (5481 and ?) Which have you considered?
Jean-Yves Le Boudec: Bound is difference of any two packet delays of a flow.
Greg Mirsky: So sounds like minimum delay
Jean-Yves Le Boudec: Is the difference between worst case and best case delays.
Greg Mirsky: OK, a bit different, but effectively two ways to present same thing.
Johannes Specht: Regarding clocks do you mean synch or async?
Jean-Yves Le Boudec: Works for both. With dampers or regulators don’t need synch’d clocks. Will improve with sync but not at level of lifetime of single packet. Given that times are in microseconds with error of nanosec, better sync doesn’t help.
Johannes Specht: Synch adds complexity, so nice to have something that doesn’t depend on it.
Janos Farkas: Re timestamping packets on the fly: Is that trivial or with large how networks how would that work?
Jean-Yves Le Boudec: Can be difficult so need to allow for some looseness in the timing accuracly. Requires dedicated HW, expensive. Affects accuracy of timestamps, creating residual jitter at end.
Toerless Eckert: PTP by itself does timestamping, so have experience at speeds up to 100GBits so that isn’t an issue. More important for dampers is delay on falling hop, how to do that - interesting scaling question.
Yaakov Stein: That is what is solved by damper proposals presented. Jitter control is an example - don’t delay packet in buffer, but in output link declare when packet becomes visible. Diff’t than standalone. Scalable HW implementataions like (?) that are efficient but at expanese of granularity of (?)
[Lou: Cutting line. ]
Balazs Varga: Timestamping of PTP packets and HW timestamping of wire speed data packets is significantly different. PTP is a slow protocol and current HW filter PTP packets in hardware. This is not an option for data packets which filtering rules are flow specific. So HW timestamping of detnet flows is much more complicated than PTP related timestamping.
Questions to answer (from chairs) – Bolding are key points noted by chairs
- What problems do we think need to be solved?
Lou Berger: Have concensus on control of delay variance, consistent theme with interest in working on it. If you support this or disagree, please speak up.
Toerless Eckert: We need per hop per flow stateless queueing model.
David Black: Would suggest discussion in terms of avoiding per flow network state, e.g., via flow aggregation.
Lou Berger: What is a “flow” - need to be clear. Detnet supports flow aggregates, flow as 5 or 6 tuple.
Toerless Eckert: Don’t want to overprovision these flows. If adjust (?)
If have fewer per hop per flow queues but need to maintain them at same frequency…
David Black: Problem statement needs strong grip on what service providers want.
Toerless Eckert: We are agreeing on this.
Lou Berger: Seems like we agree on that.
Yaakov Stein: I am interested in a slightly diff’t use case: xhauling. Latency bound more than delay variation bound, 10s of us range, need to keep this inscope. (close to hardware limits)
Lou Berger: Are you saying don’t lose bounded latency?
Yaakov Stein: Many applications discsussed have 10s of ms. Don’t forget places with tight latency bounds, not much above physical limits. Could be front haul or F1 relatively easy, for F2, few flows, can address by themself. But has to deal with control traffic also.
Toerless Eckert: What is it that Yakov is saying we are missing? Can we combine jitter with pipe latency?
Lou Berger: I heard: Tightness of latency vs latency variation.
Lou Berger: Any other problems the group wants to consider?
(No further comments on that)
- What are the requirements of this/these problem/s?
Lou Berger: Some examples: Are we allowed to modify the packet header? Are we concerned with us vs ms?
David Black: Can we discuss based on class of network? Which data plane? Things like that. Since different people have different motivating examples to support their statements. But they seem to be in different types of networks.
Stewart Bryant: On whether we can modify the packet headers depends what you mean by modify. I don’t see any packets today that can handle this state. Currently nowhere to put it, need to carry more data. So must allow some changes to data plane, or will be fatal to whole effort.
Uma Chunduri: What kind of application (usage of transport layer protocols) being discussed, like QUIC. How does it tie into transport layer?
Toerless Eckert: We didn’t cross correlate bounded latency and jitter with DetNet use cases. Most people who have done these services had done their own transport, so haven’t looked at transport layer, like building on top of RTP. But goes beyond this queuing discussion, transport for DetNet.
Lou Berger: Requirements have been shared in slide (slide 4 of requirements). Anything we can think of that is not represented there?
Janos Farkas: Be more specific about how many “large number of devices”. Like a chain of 100 or 150 - is that large scale? Can do that many with TSN.
Lou Berger: Yes may be different use cases.
Peng Liu: Key issue is performance and cost. So sync vs async important. Cost of sync in large scale network? Some feel not a problem, some feel it is. For transmission depends on (?) For large scale may span whole country. Hard to deploy.
Uma Chunduri: Requirements on PE nodes - for some use cases. What can be done on edge nodes - need a requirement statement on this.
Toerless Eckert: Regarding scale, have thousands of PEs, hundreds of (?).
We can calculate that even with aggregation will have 100s of thousands of flows to consider. What is HW cost that makes it feasible for service provider networks. Total number of interfaces in aggregation space, can be up to a thousand.
Stewart Bryant: Performance as a requirement? Will end up with usual “high vs ordinary” performance, and “lots vs few” nodes. So need to be in sweet spot for each of those.
Lou Berger: Implied consensus is that there is a problem the group would like to work on. Any objections to haveing WG explore this topic?
(No comments)
Lou Berger: Maybe only people interested in this topic are here, so need to go to the list, but plan would be to get req’ts doc done then go to process to look at solutions. Peng’s draft is a good starting point (for problem statment and requirements). If you want to add to this, please contact Peng. It is also possible to submit an individual draft if you prefer.
Lou Berger: Appreciate contributions to date, have good starting point.
Presenter: Yaakov Stein
Draft: https://datatracker.ietf.org/doc/draft-stein-srtsn
David Black: I would direct you to RFC 2475 on traffic conditioning. Is in scope here since DetNet has specific service model, expertise is clearly in routing (specifically detnet WG) not transport.
Yakov Stein: Agree on that.
Johannes Specht: You are proposing a series of local deadlines for path in header?
Yakov Stein: Please ask question on the list, busy now.
(no time for more questions)
Presenter: Toerless Eckert
Draft: https://datatracker.ietf.org/doc/draft-eckert-detnet-mpls-tc-tcqf
(No comments)
Presenter: Pascal Thubert
Draft: https://datatracker.ietf.org/doc/draft-pthubert-detnet-ipv6-hbh
Lou Berger: Need feedback from the group on level of interest in this.
Toerless Eckert: Native IPv6 is fine but issue to capture is related to Yaakov’s proposal: Is there simple standardizable latency calculations for this? Calculations would be done by controller plane - can we build an interoperable system?
Pascal Thubert: If you use SRH you can encode target time by which need to leave node.
Toerless Eckert: But how does queueing or scheduling within router work - how does router react to them? Can this be standardized or do we leave latency control to the operator?
Pascal Thubert: There is already an RFC (Deadline at 6LOW) for these deadline headers.
Toerless Eckert: Statistical, not usually considered in DetNet.
(out of time)
Lou Berger: We will look for additional input from WG, an interesting topic.
(N/A no time)