NMRG 78th meeting
IETF 121, Dublin + Online
Session 1: Thursday 2024-11-07 13:30-15:00 UTC,
https://datatracker.ietf.org/meeting/120/session/33172.ics
Session 2: Friday 2024-11-08 15:30-17:00 UTC,
hhttps://datatracker.ietf.org/meeting/121/session/33565.ics
Contacts:
Useful links:
** Agenda Session 1 Thursday 2024-11-07 13:00-15:00 UTC **
1.1 - Introduction, RG Chairs, 10 min
1.2 - Use Cases and Practices for Intent-Based Networking,
draft-kdj-nmrg-ibn-usecases, Paul, 10 min
1.3 - Future Research Directions on Energy-Aware Security Mechanisms,
draft-soares-nmrg-green-security, Jéferson Nobre, 10 min
1.4 - An Autonomic Control Loop with AI-planning and NLP for achieving
self-reconfiguration in an sliced network, Angela María Rodríguez, 20
min
- Data quality:
1.5 - Evaluating and Enhancing Test Set Quality for ML- and DL-based
IDS Testing, Omar Anser, 20 min
1.6 - The complex interplay between data and model quality: a case
study in traffic anomaly detection, José Camacho, 20 min
1.7 - Towards data-driven evaluation of intrusion detection systems,
Gregory Blanc, 20 min
** Agenda Session 2 Friday 2024-11-08 15:30-17:00 UTC **
2.1 - Introduction, document status, RG activities, RG Chairs, 10 min
AI-based network management
2.2 - AI based network management agent (NMA),
draft-zhao-nmop-network-management-agent, Xing Zhao, 10 min
2.3 - AI framework for network management,
draft-pedro-nmrg-ai-framework, Pedro Martinez-Julia, 10 min
2.4 - Considerations of network/system for AI services,
draft-hong-nmrg-ai-deploy, Yong-Geun Hong, 10min
2.5 - An Evolution of Cooperating Layered Architecture for SDN (CLAS)
for Compute and Data Awareness, draft-contreras-coinrg-clas-evolution,
Luis Contreras, 10 min
2.6 - Network Digital Twin: Concepts and Reference Architecture,
draft-irtf-nmrg-network-digital-twin-arch, Qin Wu / Mohamed Boucadair /
Cheng Zhou, 10 min
2.7 - NMRG research agenda discussion, NMRG chairs, 30 min
Notes
Session 1 Thursday 2024-11-07 13:00-15:00 UTC
1.1 - Introduction, RG Chairs, 10 min
1.2 - Use Cases and Practices for Intent-Based Networking,
draft-kdj-nmrg-ibn-usecases, Paul, 10 min
- Chairs: clarify the status of use cases that have been merged
(continuation, expired) and the agreement of the authors
1.3 - Future Research Directions on Energy-Aware Security Mechanisms,
draft-soares-nmrg-green-security, Jéferson Nobre, 10 min
Q-Jerome: any existing study?
Q-Jerome: what is the rationale to have specific energy measurement
methodology for security functions versus any type of functions
Alexander Clemm: +1 on need to articulate what would be special about
measuring energy cost for security versus other functions
- Q-Suleman: security protocols? security frameworks? can you clarify
what is your target
- A: at this point, we are not sure about what to consider. As a
researhc we can go holistically but we will probably need to focus
(at this point not yet defined as we are just formulating the
problem)
1.4 - An Autonomic Control Loop with AI-planning and NLP for achieving
self-reconfiguration in an sliced network, Angela María Rodríguez, 20
min
- Q-Chris: can you describe what's going on in the solver? you mention
NLP?
- 2 main modules but not yet integrated (future work): NORA +
Automated planning. NORA needs a grammar. A tool was trained with
samples of human-expression configuration where entities are
labelled.
-
Q-José: I believe there is confusion. The NLP is to understand the
intent, is not connected to the solver
-
Data quality:
1.5 - Evaluating and Enhancing Test Set Quality for ML- and DL-based
IDS Testing, Omar Anser, 20 min
-
Q-Renzo: maybe inherently the phenomenon you are observing has low
"noise" variance , and then you are artificially adding "noise" to
something does not have . In any case you are making assumptions, my
quesiton is maybe those assumpions are wrong? or I guess those
methods help you to not distort too much the input data. Was more of
an open question.
- Maybe I'm not fully understanding your question? In my experiments,
I’m not adding any noise; I’m using the data as it is. The only
'assumption' I’m making is that the hyperparameters of the model
might lead to overfitting on the data. It’s not exactly an
assumption, though; it’s a strong intuition that we haven’t proven,
as proving it is not straightforward. The issue is that the model
seems to be overfitting on the training data, but we’re unable to
detect it with the test set, which seems to be of poor quality. This
poor test set quality makes it challenging to accurately assess the
model’s performance
- Q-Renzo: But at the end of you presentation you suggest some methods
that changes the data (synthetically add more "diversity"), this
part of your presnetation was too fast, but my appreciation was that
you add variance/modify your data because you asume the data is not
good enogh, then you "enhance it", synthecially, but my open
questions is, what makes you believe you can synthetically modify
some data and represent better the underlying phenomenon? maye on
the mic we can discuss better if now I will try to properly rephrase
my questions over the Mailing List
- The idea isn’t to add synthetic data like typical data augmentation
methods but to generate consistent, real data from client-server
applications (such as SSH traffic in our case). This is legitimate
network traffic. We apply the tc command to modify network
conditions, positioning the data within the latent space to ensure
it is well-distributed and diverse, aligning with our metric
requirements. This helps place the data close to the decision
boundary of other classes. To optimize these tc configurations, we
are using reinforcement learning. Importantly, our method remains
model-independent by leveraging an autoencoder with a contrastive
loss
1.6 - The complex interplay between data and model quality: a case study
in traffic anomaly detection, José Camacho, 20 min
1.7 - Towards data-driven evaluation of intrusion detection systems,
Gregory Blanc, 20 min
Grouped Q&A for 1.5, 1.6, 1.7:
- Q from [?]: [question whether large language models are in scope
of NMRG]
- Laurent: general question. take it offline / after the meeting
- Sulaïman: representation and importance of features in data, can
biases the data, a technique is to drop certain features to have more
diversity in the data; for ML models it is very important to consider
the performance (models/IDS operating at line rate)
Session 2 Friday 2024-11-08 15:30-17:00 UTC
2.1 - Introduction, document status, RG activities, RG Chairs, 10 min
AI-based network management
2.2 - AI based network management agent (NMA),
draft-zhao-nmop-network-management-agent, Xing Zhao, 10 min
- Chris: be aware of the consequences of having too independent
"agents" / closed loops, how are the conflicts identified and
resolved? On the other hand, commonalities
- Alex: how much of your proposal is AI indenpendent? can be a
difference to previous/other works. the name itself "NMA" is not
conveying
- Laurent: put more effort in positioning wrt. the state of the art,
and how the design choices were made (e.g. base layer, instance
layer) ; show more the research process, because it is challenging
to udnerstand the architecture when presented as the result without
the underlying assumptions
2.3 - AI framework for network management,
draft-pedro-nmrg-ai-framework, Pedro Martinez-Julia, 10 min
-
Jerome: how compare to ML pipelines and ML automation frameworks
- Pedro: beyong single "domain"/scope of applicability of ML
pipeline
-
Laurent: part of your work can be related to the work presented in
2.2 by Xing
- Pedro: yes. one instance of their work can be represented as a
module in our model.
-
Laurnet: the service bus you have in your model, is it for
ML-related functionality or for general purpose communicaztion
between network functions? any specifics of ML to be conveyed on the
service bus?
- Pedro: more than plain "funcitons", aiming to reasoning
functions
-
Xing: what are the relationships between the services and the (SDN)
controllers
- Pedro: they are independent
-
Xing: how do you consider the deployment in real networks? as this
is very different from current ways of deploying. Maybe be worth to
add explanations on this aspect
- Pedro: not so different from the OSM architecture. These
services are connected on top of SDN. e.g. "other AINEMA
services" can be connectivity, telemetry... goal was not to
expose too many services, thus aggregate the er a generic name
-
Xing: interact with controllers in each level: domain, e2e?
- Pedro: if they have one north-bound interface, yes, they can
connect to this architecture
2.4 - Considerations of network/system for AI services,
draft-hong-nmrg-ai-deploy, Yong-Geun Hong, 10min
- Laurent: wordings on addressing the AI-NM challenges: this work can
have pieces to address some of the challenges but be careful not to
convey that the challenges are fully addressed with this work
- Laurent: clarify how much of this work is covering (/overlapping)
with the AINEMA work. It seems some aspects are addressing the
funcitonality expected in AINEMA
- Laurent: now that this document is RG document it is under the
shared responsibility of the research group. Let's focus on
producing the best qulity and outcome of this research.
2.5 - An Evolution of Cooperating Layered Architecture for SDN (CLAS)
for Compute and Data Awareness, draft-contreras-coinrg-clas-evolution,
Luis Contreras, 10 min
-
Laurent: what woulD be most useful for your to connect this work
with NMRG? also in which are the work identifies gaps: e.g.
knowledge, controls?
- Luis: especially in the knowledge, common view. controls will
remain per domain/technology.
-
Xing: is this work coming from CATS WG?
-
Roland: cf. KIRA, control plane fabric... topology discovery. some
functionality cannot be broken due to misconfiguration. willing to
explore furhter the relation/extension of KIRA to CLAS
2.6 - Network Digital Twin: Concepts and Reference Architecture,
draft-irtf-nmrg-network-digital-twin-arch, Qin Wu / Mohamed Boucadair /
Cheng Zhou, 10 min
- Chris: this work has reached a point across different groups of
catching and agreeing fundamental functionalities [of NDT]. Agreed
not gone into all aspects of implementations. This work is useful as
reference model.
- Laurent [as RG co-chair]: moving to RG last call
2.7 - NMRG research agenda discussion, NMRG chairs, 30 min
Side meeting 04/11/2024
- Chris: the main document has reached stability. The document
identifies core components whatever use-cases. The original
intention is satisfied and can go for RFC. Main objective of NMRG is
achieved, there will be standardization work in NMOP.
- Marco: there is no one-fits-all solution and practical challenges
beyond architecture depends on use cases. Aggregate experiences from
multiple participants having hands-on epxerience with NDT (maybe in
a document) such as protocols. Or in best cases, we identify
relevant key use cases to develop new standards/protocols at IETF
- Arashmid: performace between twins are highly different. How fare
cen we to represent a twin in real-time, i.e. what are the
capabilities we can have with a NDT- and how can we go beyond
- Laurent: does the main document captures enough the view of the
group. We can also have a second round of alternative architectures
- Arashmid: if we have consensus on requirements,
- Laurent: be careful with requirements as we are not a WG.
Requirements = engineering work. Several times in prev meetings: do
we have architecture alternative to be explored.
- Chris: the document is about a conceptual architecture / high-level
- Arashmid: requirements is more asusmptions as you have assumptions
- Marco: the architectural document should allow to explore different
alternative architectures. We should have a clear view of what a NDT
brings in comparison to a network simulator. There are specificities
for NDT such as syncrhonozation of states
- Depends on the use case, you can have different portion of your
simulated environments, for example control plane and data plane. If
you do not know the use case, you cannot define a NDT
- Chris: it is reflected in the architecture document
- There a lot of papers on NDT. If we form a RG, what will be the
output of the group.
- Laurent: depends on RG. Generally, any form of outcome is possible.
Writing an article with RG participants is an outcome, not
necessarily a RFC. Other example: organization of a workshop. We
don't expect a RG to produce specifications
- If you want to define NDT, you need to define different models.
- Wes: RG aims is to produce knowledge and understanding. then it can
be transfered to IETF WG. Maybe a good output would be the output of
NDT.
- Diego: we keep talking about use cases and models. We could have
demonstrators, showing a NDT running. Share experiences.
- Laurent: if many finds the same problem/challenge, that can be a
good way to identify what the group could work on.
- Diego: to be careful, demonstrators need to identify clear research
parts
- Chris: lay the fundation of a NDT is. If something need to be
updated, we can do but we need to finalize this document after 3
years. Then we can move forward towards other types of work
- Oscar: We do not have a clear definition of NDT. Based on given
inputs (e.g. topology, router configuration, router vendors), what
can I build? What approximation of functionalities?
- Wes: if there are potnetial multiple models, it s good to try them
all
- Laurent: from RG prespective, formulate a research question or
problem statement with the work you propose even if you present a
prototype with components. In addition to IETF meetings, there is a
workshop of NDT where many participants are also NMRG participants.
We have to consider all the persspectives, do not only consider IETF
meetings.
- Thomas: at NMOP WG, we are working on digital map that could serve
as an enabler for NDT. We are discussing how much detailed a NDT can
be and will be happy to hear from NMRG
- Oscar: but digital map is narrowed
- Thomas: I was referring to how much detail the topology should be.
- Olga: we have 120 use cases for digital map
- Marco: maybe it could be to sketch 2/3 reference architecture that
are relevant and different (overlay network, router)
- Chris: connection with digital map. We need to have discussions with
NMOP.