Minutes IETF104: nmrg
minutes-104-nmrg-00

NMRG 51st meeting
IETF 104, Prague

RG Chair:
   Laurent Ciavaglia <laurent.ciavaglia@nokia.com>

Available During Session:
   Slides: https://datatracker.ietf.org/meeting/104/session/nmrg
   Meetecho: http://www.meetecho.com/ietf104/nmrg/
   Etherpad: https://etherpad.ietf.org/p/notes-ietf-104-nmrg

Available Post Session:
   Audio recording: https://www.ietf.org/audio/ietf104/
   Video recording: https://www.youtube.com/user/ietf/playlists
   Etherpad: https://etherpad.ietf.org/p/notes-ietf-104-nmrg

***********
* Session 1
  Monday, 25 March 2019, Morning Session II 1120-1220
  Room: Grand Ballroom

96 on-site participants + 12 on Meetecho.

Minutes taker: Daniel King.
Meetecho scribe: Unknown.

  Agenda:
  1. Introduction, Chairs
  11:20, 5 min.

  2. Intent Classification, Richard Meade (Remote)
  11:25, 10 min. + 5 min. Q&A
Abstract: This document discusses what intent means to different users,
describes different ways to classify intent and an associated taxonomy of this
classification.

Richard presented the slides on Intent Classification
Various criteria and approaches to classify or categorize intents e.g. user
types, use cases, time, "lifecycle" (transient/persistent intent), granularity
Q&A:
    Daniel K.: Can you clarify the intention here, to have one intent framework
    (to rule them all). There are already existing ones (group policies, policy
    definitions, for a variety of deployment scenarios: DC, Cloud, network, et
    el.). Should we look at what's existing, how they are used, use cases, spot
    any gaps and develop a framework? Richard: there will be different
    levels/layers of intents. more existing work at lower layers. less at
    business layer. not proposing new definitions, but maybe more focus on the
    higher, business layer. but classifying. Will L.: We expect this to be the
    base work, starting point for the RG work. We will clarify the scope of the
    IBN work in IETF/IRTF Laurent: Please see if there is a relationship with
    draft-clemm-nmrg-dist-intent. What is the use of the proposed
    classification: clarification needed on the purpose, how will this be
    actively used in an intent-based system?

  3. IBN – the technology, Jeff Tantsura
  11:40, 20 min. + 5 min. Q&A
Abstract: Taxonomy of IBN introduces 4 levels of maturity, from basic
automation to self-operating networks. Ability to constantly validate that the
operational state is the intended state is fundamental for IBN to coherently
provide full life cycle management, from design to deployment to operations.
This is to introduce the concept of IBA - Intent Based Analytics that are
context and intent aware and gather only data that is relevant to the intent as
the opposite to “big data fishing”.

Dan B: You mentioned NB Intent is not well-defined, can we use some SB
Frameworks? Jeff T: I don’t think we see any (IETF) SB modeling work that is
applicable for IB, it would need a hammer to make it work Benoit: What about
KPIs per service? Jeff: Yes, it’s something we want to cover. Consider a VPN:
jitter, latency, et al. We should be able to express KPIs per service, via IB
API Benoit: (?)

(?): What is the definition of telemetry, scope of the measurement
Daniel King: There is a draft (Network Telemetry Framework) floating around
attempting to define Telemetry, and applicable technologies Jeff: Yes, lots of
telemetry around, there are plenty of means to collect but the major issue us
storage and analysis Daniel King: Are you considering "network innocence or
guilt" (Root Cause Analysis)? It’s an area the ITU-T have been working on and
may be applicable here. The thing impacting the service may not be
network-based as well. Jeff: Yes, this is the right place to investigate root
cause

<end of presentation>

Will: I looked for IB definitions in previous IETF documents and could not find
one. Jeff: Unless you can build life cycle management phases (design, deploy,
operate(?), then this work is not useful. Continuous operation is key, this
requires those telemetry capabilities Will: The tools here are in part IETF,
but I think some capabilities are external(?) to IETF What are the work items,
and where will they sit (IRTF/IETF)? The ML techniques, and processes are
probably outside the scope of this organization Laurent: The discussion is
welcome, but this is an RG and we should not spend too much time on IETF areas
Benoit: I am working on this area myself in my company, and just wondering what
is really missing (that’s not solved by vendor specific capabilities), APIs,
context definitions? Charles channel Jabber: What about events that are not
monitored by telemetry/operations? Jeff: Then we cannot do anything, except
trigger some additional capability to query another data stream if we know it
may provide further insight Laurent: interesting proposal. ability to extract
knowledge with less data? Work on evaluation and measurement to provide results
confirming or infirming assumptions, evaluate the gain/cost ratio...

  4. Discussion
  12:05, 15 min.
       . Update to NMRG work plan on IBN
       . Deliverables, milestones, criteria for RG adoption

<Laurent summary...>
good set of comments and level of interest in exploring further the IBN
proposals in the group. Please continue to use the mailing list to express your
comments/questions/issues and try to resolve them. Extend, update the current
work plan with:
  –Problem statement, scope, design challenges and goals
  –Concepts definition
  –Intent at design and run time, impact of CI/CD(?)
  –Interoperability
  –Continuous measure/validation of intent realization
  –Evaluation, validation, implementation

Deliverables, Milestones, Criteria for RG adoption will be discussed at next
virtual meeting in April.

Progressing the work (RG support)
–Regular, dedicated virtual meetings (1 or 2 per month?)
–Interim meetings: June, TBC; July-IETF105; October, TBC
–Hackathon…?
–Mailing list, other tools needed?


***********
* Session 2
  Thursday, 28 March 2019, Afternoon Session I 1350-1550
  Room: Karlin 1/2

41 on-site participants + 6 on Meetecho.

Minutes taker: Will Liu.
Meetecho scribe: Juergen Schoenwaelder

  Agenda:
  1. Introduction, Chairs
  13:50, 10 min.

  2. NMRG 1999-2019, a Retrospective, Past NMRG Chairs
  14:00, 15 min. + 10 min. Q&A
Abstract: A retrospective of NMRG from its creation in 1999 up to now (2019),
presented by previous NMRG chairs (Juergen, Olivier and Lisandro). The goal is
to reflect on the evolutions the RG has been through and how this could help us
to define what the RG work could be for the next period (5-10 years ahead).
Funny people think about the future of 20 years old RG How measure RG outcome,
success? 8 RFCs, 51 meetings. Always attractive, lively debates... 3 phases of
the RG, the technologies, and types of problems addressed. Measurement:
Netflow, IPFIX Phase 2 autonomic networks: ANIMA WG "takes" the autonomic
network topic but do not explore certain areas -> autonomic 3.0, network
automation with new technologies (virtualization, AI...) <Rick Taylor>
asynchronous management in DTN, we know about use cases but not enough on
network management, all welcome to help Phase 3 looking for new topics over the
last meetings (IDN, AI), AI: avoid focusing on particular use cases, best ways
to integrate AI techniques in or for the network Future: is management still
the right term to use? still valid? No further Q&A:


  3. AI for NM, Speakers Albert Cabellos, Jérôme François, Laurent Ciavaglia,
  on behalf of authors team 14:25, 30 min. + 15 min. Q&A
Abstract: Collective discussion about the AI for NM topic and what the RG work
should be, building upon presentations and discussions at previous NMRG
meetings (e.g. IETF100/Singapore, IETF 102/Montreal). Frame the topic. Browse
through the

Q&A:
    Rick Taylor: is there a special use for accountability and explain-ability?
    Is there a difference when AI is applied to NM rather than for cars or
    vision? So is there any particular accountability for NM. For distributed
    AI, we know how to do distributed management, have some best practices
    Albert: no difference in accountability and explain-ability for network
    Rick: agree. Accountability and explain-ability waste of time as there is
    so much to do for other items Laurent: we should not work on accountability
    and explain-ability (how to) but where to interface with the network Rick:
    agree: guidelines and references but no spending time to define a new
    system for accountability and explain-ability no Sabine: distributed AI,
    point of difficulty, the big challenge: make sure to have harmonized
    decisions, build a framework to ensure that decisions taken are
    understandable to each other -> ensure they share the same view of the
    network. Rules for distributed decisions systems Rick: we know how to do
    that with humans, the Internet was built distributed Laurent: distributed
    AI exists today but may not fit to the nature of the network (data are
    temporary), how to make the nodes/agent collaborating together, AI elements
    will be spread into different network elements and we will have to
    integrate them. It is not necessarily that hey all collaborate but hey will
    be there and have to consider when managing networks Sabine: what is
    necessary information/data model to make sure to have a common view of the
    network. Information is about entities at different levels Albert: struggle
    between AI is great vs how this interferes with protocols. We don't see a
    clear roadmap for the AI at the IETF. First try to understand how AI
    techniques will impact on network operations by inviting network experts.
    we need expose nmrg requirements. to merge nm and ai ... (miss something
    here, somebody gets it?) Juergen: let's pose the challenge and ask research
    community to solve it ... : expose our requirements to the ML communities,
    even if we don't aim to build new protocols, we can define informational
    drafts Rick: I believe these datasets in our community at IETF but need a
    lot of energy to collect, host ... : suggestion write a MIB Colin: need to
    reinforce the group with AI people Laurent: how to attract people from the
    AI communities, no incentives to come Pedro (Jabber): AI or just machine
    learning? Not exploiting all AI such as reasoning and planning, please
    consider it Albert: we should not give requirement to AI community. Invite
    people that apply IA (example: DRL for packet classification) and think
    about if this impact on protocols Rick: Point raised by Pedro very relevant
    Laurent: Point raised by Pedro very relevant, interesting because ties to
    intent policy based mgmt.

  4. Discussion on future of the RG, Chairs, Open Mic
  15:10
      . General information on the RG
      . Way forward for the RG, NMRG re-charter