NMRG 68th meeting IETF 116, Hybrid Wednesday 2023-03-29 15:30 JST (06:30 UTC) https://datatracker.ietf.org/meeting/116/session/30272.ics Contacts: RG Chairs Laurent Ciavaglia laurent.ciavaglia@nokia.com Jérôme François jerome.francois@inria.fr RG Secretaries Jéferson Campos Nobre jcnobre@inf.ufrgs.br Pedro Martinez-Julia pedromj@gmail.com Useful links: Materials: https://datatracker.ietf.org/meeting/116/session/nmrg Meetecho: https://meetings.conf.meetecho.com/ietf116/?group=nmrg Notes: https://notes.ietf.org/notes-ietf-116-nmrg Video recording: https://www.youtube.com/user/ietf/playlists (available post-meeting) * 15:30 Introduction and RG status, RG Chairs - Need dedicated discussions on IBN use cases to address them collectively (interim meeting to be announced) - RG annoucements on the ML for the research agenda: conclude IBN items, 3 topics increasing (AI, Digital twin, green networking): need to define what the RG would like to do for these topics (outcomes?) * Artificial Intelligence and Data for Network Management 15:40 Research Challenges in Artificial Intelligence for Network Management, Jérôme François https://datatracker.ietf.org/doc/draft-francois-nmrg-ai-challenges Laurent: content relatively stable. Evaluate value and impact of content and proceed towards publication. 15:50 Considerations of deploying AI services in a distributed approach, Yong-Geun Hong https://datatracker.ietf.org/doc/draft-hong-nmrg-ai-deploy Laurent: Alignment with challenges document is good, but too much integration is not beneficial. Document has been there for some time and stable. Important to define good positioning and the impact of the research if the document is published, e.g. as recommendations for distributed AI deployment, experience/use case report, then be ready to move to RG document. Yong-Geun: Agree. More time needed. And more feedback is important and welcome. Action: call for RG review and feedback on the document * 16:05 Network dataset quality problem, Katarzyna Wasielewska Diego Lopez: Are you aware of the complementary between "privatier" and your proposal. They were analyzing the bias of data in neural models. Laurent: Previous presntation in NMRG about presentation (Artur) on trusthworthiness on AI models. Diego: Evolution of tools to assess how datasets could better characterize models, evaluate models, etc. This tool is focused on the dataset. Could this also use to assess the metadata? Katarzyna: Right. This solution is focused on the relationships between observations and labels of data, it is difficult to incorporate the metadata. Artur: the approaches are complementary and can be used tohgether. Our work is more on the x affects the y. Did you consider the dimensionality of correlation worsening with data? Katarzyna: Not yet. Marco: when you have a classifier, you measure the correlation between x and y. If you permute, you lose correlation. Lower correlation means lower recall. If you have good results in your classifier, the dataset is also good? Katarzyna: You can have high correlation, but you do not know why. If we have problems with x-y correlation, it is one reason for the result. We can have other problems with the datasets that affect performance results, for example with misslabels. High-quality datasets must be first considered to find the relations and then assess other datasets. (side note: measuring model robustness or data set quality?) *16:35 Weaving YANG into a network data fabric, Diego R. Lopez Pedro: Why not connect all stramlined components to the bus? Diego: They depict a conceptual view, actually the broker is connected to the bus. * Network Digital Twin 16:45 Functional Design Aspects of Performance-Oriented Digital Twins, Christopher Janz https://datatracker.ietf.org/doc/draft-janz-nmrg-performance-digital-twin * Green Networking 16:55 Challenges and Opportunities in Green Networking (draft-cx-green-ps) and Green Networking Metrics (draft-cx-green-metrics), Alexander Clemm https://datatracker.ietf.org/doc/draft-cx-green-ps https://datatracker.ietf.org/doc/draft-cx-green-metrics