# Network Management Operations (nmop) WG Agenda - Interim on Anomaly {#network-management-operations-nmop-wg-agenda---interim-on-anomaly} * When: Wed, Sept 11, 2024 * Co-Chairs: Benoît Claise & Mohamed Boucadair ## Compact Agenda {#compact-agenda} | Slot | Topic | Presenters | | :-: :-: :- | 14:00 - 14:10 | Agenda Bashing & Introduction | Chairs | | 14:10 - 14:30 | Swisccom: Network Incident Network Analytics Postmortem | Thomas Graf | | 14:30 - 14:40 | Bell Canada: Network Incident Network Analytics Postmortem | Dan Voyer | | 14:40 - 14:55 | Orange: Knowledge Graphs for Enhanced Cross-Operator Incident Management & Network Design | Lionel TailHardat | | 14:55 - 15:10 | INSA: Practical Anomaly Detection in Internet Services: An ISP centric approach | Alex Huang Feng | | 15:10 - 15:25 | An Architecture for a Network Anomaly Detection Framework | Wanting Du | | 15:25 - 15:45 | Experiment: Network Anomaly Lifecycle | Vincenzo Riccobene | | 15:45 - 16:00 | Discussion, Open Issues, Next Steps | Chairs | ## Detailed Agenda {#detailed-agenda} ### 1. Agenda Bashing & Introduction (Chairs) (10 min) {#1-agenda-bashing--introduction-chairs-10-min} Chairs slides are available [here][1]. No agenda bashing ### 2. Part 1: Operators Inputs (60 min) {#2-part-1-operators-inputs--60-min} Operators presenting their problems and views, in the context of network anomaly. #### 2.1. Swisscom: Network Incident Network Analytics Postmortem (20 min) {#21-swisscom-network-incident-network-analytics-postmortem-20-min} * Presenter: Thomas Graf * Reading Material: Describes incidents in terms of what happened, which operational metrics where available, which analytical metrics described the symptoms and what improvements in the network anomaly detection system and network telemetry protocols are proposed. Presented [slides][2]. * Data: IPFIX from the core network (BGP, IP-MPLS) * Dataset: SRv6 IS-IS ABR Route aggregation, august 14th, post maintenance windows analysis. * Forwarding plane drops, no alerts * Two applications showed loss of connectivity, alerts * Post analysis showed that loss of connectivity comes from the reconfiguration work, with effects such as congestion and state changes. * Automated anomaly detection (as opposed to dashboard-based analysis): outlier detection to compute a composite time-stamped score (concern, traffic loss, etc.) ... kind of fuzzy reasoning. * Challenges in the volume of VPNs to analyse. * Wish to add network topology to contextualize the scores * Observing configuration state change with transaction ID (draft-ietf-netconf-transaction-id, draft-ietf-netconf-configuration-tracing) could have been useful to go faster on the anomaly detection. Rob Wilton: If the case of packets being dropped and wanting further on-path tracing, how do you know which packets to do the tracing on? Thomas Graf: Effectively you do statistically tracing/sampling, with higher priorities given to more important traffic classes. #### 2.2. Bell Canada: Network Incident Network Analytics Postmortem (10 min) {#22-bell-canada-network-incident-network-analytics-postmortem-10-min} * Presenter: Dan Voyer * Reading Material: Describes incidents in terms of what happened and give insights on the current network telemetry rollout in the network and their preparation for anomaly detection deployment. Rob: You mentioned that it is hard to corrolate that data, is this because there is too much data from devices, or that it is hard to find the right fields, or the wrong data is being exported? Dan: It is really a combination of all three. Going to closed loop automation means that you have to rely on the quality and consistency of the data to make robust decisions on config changes. Comment from Thomas on chat: 15:50 I think the hard, challenging things is to obtain the right data, correctly structured in near real-time. Chosing the right data is not so a problem, since the understanding what connectivity service is provided and what metrics are needed to validate that the service is properly working is mostly a given. Presented [slides][3]. * Data wrangling is complicated because of many legacy systems and heterogeneous data. * Work agreement signed between Bell Canada and Swisscom to make things simpler, notably in knowledge sharing in outage causes and remediation actions (post maintenance analysis). #### 2.3. Orange: Knowledge Graphs for Enhanced Cross-Operator Incident Management and Network Design (15 min) {#23-orange-knowledge-graphs-for-enhanced-cross-operator-incident-management-and-network-design-15-min} * Presenter: Lionel Tailhardat * Reading Material: [draft-tailhardat-nmop-incident-management-noria][4] Presented [slides][5]. Rob: What languages are using to model the graph, is this semantic web? Lionel: Yes, it is semantic web, I'm using RDFS/OWL and RML. Rob: Do you see any issues with the mapping? Any corner cases not covered? Lionel: no as for now, i.e. using data from several internal datasources, but not YANG conf yet (work in progress); see these two following works on data integration: * Lionel TAILHARDAT, Raphaël TRONCY, and Yoan CHABOT. 2023. “Designing NORIA: a Knowledge Graph-based Platform for Anomaly Detection and Incident Management in ICT Systems”. In KGCW’23: 4th International Workshop on Knowledge Graph Construction, May 28, 2023, Crete. CEUR-WS.org, online CEUR-WS.org/Vol-3471/paper3.pdf \[[pres-pdf][6]\] * Lionel TAILHARDAT, Raphaël TRONCY, and Yoan CHABOT. 2024. “NORIA-O: An Ontology for Anomaly Detection and Incident Management in ICT Systems”. In 21st European Semantic Web Conference (ESWC), Resources track, May 26-30, 2024, Hersonissos, Greece. Best paper award nominee. https://doi.org/10.1007/978-3-031-60635-9\_2 \[[pres-pdf][7], [Videolectures.net pres-video][8]\] Thomas: Are the control plane and forwarding plane being represented in YANG data models, and network relationships are coming from the "ietf-network.yang" at digitial map, and then you use the knowledge graph to combine the different network planes. Lionel: Yes, that is a good summary. #### 2.4. INSA: Practical Anomaly Detection in Internet Services: An ISP centric approach (15 min) {#24-insa-practical-anomaly-detection-in-internet-services-an-isp-centric-approach-15-min} * Presenter: Alex Huang Feng * Reading Material: [https://hal.science/hal-04655324][9] ; [https://hal.science/hal-04307611][10] Presented [slides][11]. No questions. ### 3. Part 2: NMOP Plan of Work (50 min) {#3-part-2-nmop-plan-of-work---50-min} *Presenters Guidelines and Presentation Objectives*: * Don't repeat what was presented at the last IETF meeting * Please focus on: - Open issues (favor the discussion, plan the Q&A in your timeslot) - Items that are meant to ease structuring the effort - Future plans - Experiments explanation, including how to interconnect with other experiments #### 3.1. An Architecture for a Network Anomaly Detection Framework (15 min) {#31-an-architecture-for-a-network-anomaly-detection-framework-15-min} * Presenters: Wanting Du, Pierre Francois * Reading Material: [draft-netana-nmop-network-anomaly-architecture][12] Presented [slides][13]. Wanting Du presenting. Lionel: It could be interesting to map parts of the proposal to tasks in some IT Service Management or Incident Management business process (e.g. ITIL). Wanting: Thanks a lot. We take that into consideration. #### 3.2. Experiment: Network Anomaly Lifecycle (20 min) {#32-experiment-network-anomaly-lifecycle-20-min} * Presenters: Vincenzo Riccobene * Reading Material: [draft-netana-nmop-network-anomaly-lifecycle][14] Presented [slides][15]. Lionel: Regarding the symptoms list in the data model, is it that you say that a network anomaly can only occurr when there is co-occurrence of alarms/symptoms? Vincenzo: There can be network anomalies with only one symptom, based on the data model, but the ability to group network anomalies is very important as it allows to group together symptoms that are related to each other (for instance if mutiple symptoms are related to the same issue, e.g. data plane and control plane). Lionel: How do you incorporate symptoms into the data model to make it shareable? Or do you only consider refering to an external detection model, and then for analysts to go to the model to understand the anomaly mechanism? Vincenzo: The symptom is meant to be the atomic entity of the behaviour, as the symptom is defined on time series data and the semantic for the symptom is defined in the network-anomaly-semantic draft. Lionel: If I was to collaborate with you, it would be to making the data model more self sufficient. Vincenzo: Thank you - I would be interested in working with you too. #### 3.3. Discussion, Open Issues, & Next Steps (15 min) {#33-discussion-open-issues--next-steps-15-min} * WG Chairs lead discussion Adrian: Wanted to point that "anomaly" is defined in the terminology draft. I want to check that the folks who are working in this area are happy with this definition. Questions: Is this definition okay, and is this the right reference? Benoit: How would you phrase your question? Adrian: If there is an immediate question, that is fine, otherwise, can bring it back to the WG mailing list. Also have a definition of "incident" that is pointing to the Incident YANG draft. Also define "symptom" which is used in this conversation. Dan: Well written definition are really important. I want to motivate you to continue this work and not let it go. Adrian: I think it comes to this: that this definition should remain in the terminology document and folks should complain if the term is not right. Alex: I think that having a draft with all the terminolgy is useful, so we should have a definition of anomology. Benoit: Is there an issue regarding the terminology draft (and the framework) being Informational? Adrian: This will depend on the IESG at the time. I would say that it should be informational. Need to reference the terminology to understand the terms used in a specification, but not to implement the specification. Benoit: Will check offline. Benoit: reading the questions on the [slide][16] that were derived from the various presentations (created on the fly by the WG chairs, during the call, to understand the WG interest): 1) Is it worth to document lessons learned/shared out of this valuable « bring your own outage » (credits to Rob :-)) effort? Share Quick Wins? 2) Is the WG interested to investigate means that would help with auto- generation of correlation and then help with efficient mitigation? Do we have a candidate technical approach? Can data annotations be useful here? 3) Call for contributions to the KG experiment with a focus on anomaly • Share more lessons on automatic mapping of YANG to ontologies? 4) We have so far two drafts to "plug" in the network anomaly detection framework • Do they fulfill the needs expressed so far? • Do we need others? Michael: Have posted a knowledge graph framework for network operations. I have some questions on "Orange: Knowledge Graphs for Enhanced Cross-Operator Incident Management and Network Design" preso Benoit: Can post to the mailing list. Michael: We have our own draft, but probably more aligned to requirements. Mahesh: Wanted to thank all the presenters. Particularly liked Thomas's presentation. Mentioned that you wanted to get to the point where you can determine what config change caused the outage. I do have a student mentioned my Colin Perkins, and perhaps they could present in NMOP. Lionel: On question 2), yes we do have technical approaches to learn behavioral models and issue annotations, with either a human in the loop or automated approach (or both) ... this is implicitely related to a "yes" on question 3). Thomas: Interested in both points (2) and (3). Vincenzo: the experiment on labelling can be used for annotion mentionned in point (2). [1]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-chairs-slides-01 [2]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-swisscom-network-incident-network-analytics-postmortem-00 [3]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-bells-journey-network-observability-00 [4]: https://datatracker.ietf.org/doc/draft-tailhardat-nmop-incident-management-noria/ [5]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/session/nmop [6]: https://genears.github.io/pubs/KGCW-2023-noria-platform-pres.pdf [7]: https://genears.github.io/pubs/ESWC-2024-noria-o-pres.pdf [8]: https://videolectures.net/eswc2024_tailhadat_anomaly_detection/ [9]: https://hal.science/hal-04655324 [10]: https://hal.science/hal-04307611 [11]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-insa-practical-anomaly-detection-in-internet-services-an-isp-centric-approach-00 [12]: https://datatracker.ietf.org/doc/draft-netana-nmop-network-anomaly-architecture/ [13]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-an-architecture-for-a-network-anomaly-detection-framework-00 [14]: https://datatracker.ietf.org/doc/draft-netana-nmop-network-anomaly-lifecycle/ [15]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-experiment-network-anomaly-lifecycle-00 [16]: https://datatracker.ietf.org/meeting/interim-2024-nmop-03/materials/slides-interim-2024-nmop-03-sessa-chairs-questions-00