NMRG 51st meeting IETF 104, Prague RG Chair:    Laurent Ciavaglia Available During Session:    Slides: https://datatracker.ietf.org/meeting/104/session/nmrg    Meetecho: http://www.meetecho.com/ietf104/nmrg/    Etherpad: https://etherpad.ietf.org/p/notes-ietf-104-nmrg Available Post Session:    Audio recording: https://www.ietf.org/audio/ietf104/    Video recording: https://www.youtube.com/user/ietf/playlists    Etherpad: https://etherpad.ietf.org/p/notes-ietf-104-nmrg *********** * Session 1   Monday, 25 March 2019, Morning Session II 1120-1220   Room: Grand Ballroom  96 on-site participants + 12 on Meetecho. Minutes taker: Daniel King. Meetecho scribe: Unknown.   Agenda:   1. Introduction, Chairs   11:20, 5 min.   2. Intent Classification, Richard Meade (Remote)    11:25, 10 min. + 5 min. Q&A Abstract: This document discusses what intent means to different users, describes different ways to classify intent and an associated taxonomy of this classification.  Richard presented the slides on Intent Classification Various criteria and approaches to classify or categorize intents e.g. user types, use cases, time, "lifecycle" (transient/persistent intent), granularity Q&A:     Daniel K.: Can you clarify the intention here, to have one intent framework (to rule them all). There are already existing ones (group       policies, policy definitions, for a variety of deployment scenarios: DC, Cloud, network, et el.). Should we look at what's existing, how      they are used, use cases, spot any gaps and develop a framework?     Richard: there will be different levels/layers of intents. more existing work at lower layers. less at business layer. not proposing new      definitions, but maybe more focus on the higher, business layer.      but classifying.      Will L.: We expect this to be the base work, starting point for the RG work. We will clarify the scope of the IBN work in IETF/IRTF     Laurent: Please see if there is a relationship with draft-clemm-nmrg-dist-intent.     What is the use of the proposed classification: clarification needed on the purpose, how will this be actively used in an intent-based system?   3. IBN – the technology, Jeff Tantsura    11:40, 20 min. + 5 min. Q&A Abstract: Taxonomy of IBN introduces 4 levels of maturity, from basic automation to self-operating networks. Ability to constantly validate that the operational state is the intended state is fundamental for IBN to coherently provide full life cycle management, from design to deployment to operations. This is to introduce the concept of IBA - Intent Based Analytics that are context and intent aware and gather only data that is relevant to the intent as the opposite to “big data fishing”. Dan B: You mentioned NB Intent is not well-defined, can we use some SB Frameworks?  Jeff T: I don’t think we see any (IETF) SB modeling work that is applicable for IB, it would need a hammer to make it work Benoit: What about KPIs per service?  Jeff: Yes, it’s something we want to cover. Consider a VPN: jitter, latency, et al. We should be able to express KPIs per service, via IB API Benoit: (?) (?): What is the definition of telemetry, scope of the measurement Daniel King: There is a draft (Network Telemetry Framework) floating around attempting to define Telemetry, and applicable technologies Jeff: Yes, lots of telemetry around, there are plenty of means to collect but the major issue us storage and analysis Daniel King: Are you considering "network innocence or guilt" (Root Cause Analysis)? It’s an area the ITU-T have been working on and may be applicable here. The thing impacting the service may not be network-based as well.   Jeff: Yes, this is the right place to investigate root cause Will: I looked for IB definitions in previous IETF documents and could not find one.  Jeff: Unless you can build life cycle management phases (design, deploy, operate(?), then this work is not useful. Continuous operation is key, this requires those telemetry capabilities Will: The tools here are in part IETF, but I think some capabilities are external(?) to IETF What are the work items, and where will they sit (IRTF/IETF)? The ML techniques, and processes are probably outside the scope of this organization Laurent: The discussion is welcome, but this is an RG and we should not spend too much time on IETF areas Benoit: I am working on this area myself in my company, and just wondering what is really missing (that’s not solved by vendor specific capabilities), APIs, context definitions?   Charles channel Jabber: What about events that are not monitored by telemetry/operations? Jeff: Then we cannot do anything, except trigger some additional capability to query another data stream if we know it may provide further insight Laurent: interesting proposal. ability to extract knowledge with less data? Work on evaluation and measurement to provide results confirming or infirming assumptions, evaluate the gain/cost ratio...   4. Discussion   12:05, 15 min.        . Update to NMRG work plan on IBN        . Deliverables, milestones, criteria for RG adoption           good set of comments and level of interest in exploring further the IBN proposals in the group. Please continue to use the mailing list to express your comments/questions/issues and try to resolve them. Extend, update the current work plan with:   –Problem statement, scope, design challenges and goals   –Concepts definition    –Intent at design and run time, impact of CI/CD(?)   –Interoperability   –Continuous measure/validation of intent realization   –Evaluation, validation, implementation   Deliverables, Milestones, Criteria for RG adoption will be discussed at next virtual meeting in April. Progressing the work (RG support) –Regular, dedicated virtual meetings (1 or 2 per month?) –Interim meetings: June, TBC; July-IETF105; October, TBC –Hackathon…? –Mailing list, other tools needed?       *********** * Session 2   Thursday, 28 March 2019, Afternoon Session I 1350-1550     Room: Karlin 1/2     41 on-site participants + 6 on Meetecho. Minutes taker: Will Liu. Meetecho scribe: Juergen Schoenwaelder   Agenda:   1. Introduction, Chairs   13:50, 10 min.    2. NMRG 1999-2019, a Retrospective, Past NMRG Chairs   14:00, 15 min. + 10 min. Q&A Abstract: A retrospective of NMRG from its creation in 1999 up to now (2019), presented by previous NMRG chairs (Juergen, Olivier and Lisandro). The goal is to reflect on the evolutions the RG has been through and how this could help us to define what the RG work could be for the next period (5-10 years ahead).  Funny people think about the future of 20 years old RG How measure RG outcome, success? 8 RFCs, 51 meetings. Always attractive, lively debates... 3 phases of the RG, the technologies, and types of problems addressed. Measurement: Netflow, IPFIX Phase 2 autonomic networks: ANIMA WG "takes" the autonomic network topic but do not explore certain areas -> autonomic 3.0, network automation with new technologies (virtualization, AI...) asynchronous management in DTN, we know about use cases but not enough on network management, all welcome to help Phase 3 looking for new topics over the last meetings (IDN, AI), AI: avoid focusing on particular use cases, best ways to integrate AI techniques in or for the network Future: is management still the right term to use? still valid? No further Q&A:        3. AI for NM, Speakers Albert Cabellos, Jérôme François, Laurent Ciavaglia, on behalf of authors team    14:25, 30 min. + 15 min. Q&A  Abstract: Collective discussion about the AI for NM topic and what the RG work should be, building upon presentations and discussions at previous NMRG meetings (e.g. IETF100/Singapore, IETF 102/Montreal). Frame the topic. Browse through the  Q&A:     Rick Taylor: is there a special use for accountability and explain-ability? Is there a difference when AI is applied to NM rather than for cars or vision? So is there any particular accountability for NM. For distributed AI, we know how to do distributed management, have some best practices     Albert: no difference in accountability and explain-ability for network     Rick: agree. Accountability and explain-ability waste of time as there is so much to do for other items     Laurent: we should not work on accountability and explain-ability (how to) but where to interface with the network     Rick: agree: guidelines and references but no spending time to define a new system for accountability and explain-ability no     Sabine: distributed AI, point of difficulty, the big challenge: make sure to have harmonized decisions, build a framework to ensure that decisions taken are understandable to each other -> ensure they share the same view of the network. Rules for distributed decisions systems     Rick: we know how to do that with humans, the Internet was built distributed     Laurent: distributed AI exists today but may not fit to the nature of the network (data are temporary), how to make the nodes/agent collaborating together, AI elements will be spread into different network elements and we will have to integrate them. It is not necessarily that hey all collaborate but hey will be there and have to consider when managing networks     Sabine: what is necessary information/data model to make sure to have a common view of the network. Information is about entities at different levels     Albert: struggle between AI is great vs how this interferes with protocols. We don't see a clear roadmap for the AI at the IETF. First try to understand how AI techniques will impact on network operations by inviting network experts. we need expose nmrg requirements. to merge nm and ai     ... (miss something here, somebody gets it?)     Juergen: let's pose the challenge and ask research community to solve it     ... : expose our requirements to the ML communities, even if we don't aim to build new protocols, we can define informational drafts     Rick: I believe these datasets in our community at IETF but need a lot of energy to collect, host     ... : suggestion write a MIB     Colin: need to reinforce the group with AI people     Laurent: how to attract people from the AI communities, no incentives to come     Pedro (Jabber): AI or just machine learning? Not exploiting all AI such as reasoning and planning, please consider it     Albert: we should not give requirement to AI community. Invite people that apply IA (example: DRL for packet classification) and think about if this impact on protocols     Rick: Point raised by Pedro very relevant     Laurent: Point raised by Pedro very relevant, interesting because ties to intent policy based mgmt.   4. Discussion on future of the RG, Chairs, Open Mic   15:10       . General information on the RG       . Way forward for the RG, NMRG re-charter