**NMRG 81st meeting
IETF 122, Bangkok + Online

2025-03-21 03:30-05:30 CET
https://datatracker.ietf.org/meeting/122/session/33989.ics

**Contacts:

• RG Chairs
• Laurent Ciavaglia
  laurent.ciavaglia@nokia.com
• Jérôme François
  jerome.francois@uni.lu
• RG Secretaries
• Jéferson Campos Nobre
  jcnobre@inf.ufrgs.br
• Pedro Martinez-Julia pedromj@gmail.com

**Useful links:

• Materials: https://datatracker.ietf.org/meeting/122/session/nmrg
• Meetecho: https://meetings.conf.meetecho.com/ietf122/?session=33989
• Notes: https://notes.ietf.org/notes-ietf-122-nmrg
• Video recording: https://www.youtube.com/user/ietf/playlists
  (available post-meeting)

**Agenda Thursday 2024-11-07 09:30-11:30 UTC+7 **

• Introduction, document status, RG activities and news, RG Chairs, 15
  min

• Network Digital Twin: Concepts and Reference Architecture,
  draft-irtf-nmrg-network-digital-twin-arch, Cheng Zhou 10 min + 5 min
  Q&A

• NDT – Practical Considerations and Examples, Marco Liebsch, 10 + 5
  min Q&A

• Artificial Intelligence (AI) for Network Operations,
  draft-king-rokui-ainetops-usecases, Reza Rokui (or Cheng Li), 15min

  • 5 min Q&A

• Large Model based Agents for Network Operation and Maintenance,
  draft-chuyi-nmrg-ai-agent-network-00, Chuyi, 10 min + 5 min Q&A

• A Framework for LLM-Assisted Network Management with
  Human-in-the-Loop, draft-cui-nmrg-llm-nm, Mingzhe Xing, 10min + 5
  min Q&A

• Open discussions on GenAI in Network Management:
  -- Should we develop this topic within NMRG?
  -- How to develop this as a group (i.e. beyon individual
  contributions)?
  -- How to sync with other activities, RGs/WGs?

Notes

• Introduction, document status, RG activities and news, RG Chairs, 15
  min

• Network Digital Twin: Concepts and Reference Architecture,
  draft-irtf-nmrg-network-digital-twin-arch, Cheng Zhou 10 min + 5 min
  Q&A

  • Laurent: work is ready for IRSG review. Good milestones reach on
    this topic. Work already cited outside of the group.
  • Laurent: Comment on data generation and modelling for digital
    twins. It is good to continue working on NDT beyond this first
    draft. You can either use the twin to generate data for test
    cases (if the twin is accurate enough) or generate data to build
    the NDT itself.
  • Cheng: two good points we continue research
  • Diego: simulation vs emulation. Simulation is abstract.
    Emulation is more realitic, for example to generate packets.
  • Jerome: challenge the comment and keep simulation as an
    important component of DT. In some cases, you can have more
    accuracy and scale with simulation than emulation
  • Dan: both terms are useful and not the same thing.
  • Diego: ideally combine simulation and emulation.
  • Oscar: same discussion in SIMAP where we define the term and
    their use. Emulation: short tterm vs simulation long-term
  • Pedro: The problem is "just a simulation". There is no "just"
    there. A simulation can be deeper than experimentation.
    (Emphasize "can").

• NDT – Practical Considerations and Examples, Marco Liebsch, 10 + 5
  min Q&A

  • Paul: what emulator do you use and experience? open-source, your
    own?

    • Marco: we have access to both. We need to generate traffic
      to stress the model.

  • Paul: DO you consider the user equipment mobility?

    • Marco: not yet.

  • Paul: focus on CORE network

  • Jerome: about the goals: analyze the behavior of NDT under
    traffic conditions and based on defined models.

    • Marco: putting the system in stress situations. finding
      limits. deployment dependent

  • Cheng:

    • Marco: need to tailor the solutions to build your twin.
      AI/ML is not the only set of tehcniques to model and
      buildthe logic of the twins

  • Arashmid: how much work to transfer from one operator to
    another.

    • Marco: building such a model with compatibility to
      production networks. Not yet investigated, this will come
      next. The behavior of this NF depends on the deployement as
      this is software. Real 5G nework is more complex.
      -

• Artificial Intelligence (AI) for Network Operations,
  draft-king-rokui-ainetops-usecases, Reza Rokui, 15min + 5 min Q&A

  • Pedro: Current practical generative AI is being limited to LLM.
    What other practical machines / models are you considering?

  • Xing: draft in ops on agent framework. see how can collaborate.

    • XingZhao05:01 This is the link of draft on AI based network
      management
      agent：https://datatracker.ietf.org/doc/draft-zhao-nmop-network-management-agent/
      This draft is trying to give a common framework of (Gen-)AI
      based agent for network management operations. The new
      version has added the content of multi-agent cooperation,
      etc. Any feedbacks are welcome!
      https://datatracker.ietf.org/doc/draft-zhao-nmop-network-management-agent/

  • Takuya/KDDI: clarify the scope of the draft. focus on inference.
    training is also important

    • Reza: use case dependent

  • Scott: due diligence on SOTA. External xork and references (ITU,
    AI readiness Plugfest)

    • Scott Mansfield04:54
      https://datatracker.ietf.org/liaison/1970/
    • Scott Mansfield04:55 IAB response:
      https://datatracker.ietf.org/liaison/1972/

  • Pedro: What other practical models or practcial?

  • Reza: dependent of use cases but we have not defined that yet.
    If you have a question for a specific use case, go on the
    mailing list
  • Dan: with some use cases, we have user inputs. 20+ use cases:
    some are cookbook on how to combine IETF protocols with an AI
    system. Need for longer teerm research for some, engineering for
    others.
  • Laurent: differentiate in this document IETF / IRTF work.
    articulate research questions challengres relevant for NMRG

• Large Model based Agents for Network Operation and Maintenance,
  draft-chuyi-nmrg-ai-agent-network-00, Chuyi, 10 min + 5 min Q&A

  • Jerome: in the document: function/operation of agents: need to
    describe the challenges or constraints for each function. also
    consider SOTA review.
  • Jerome: connect wth AINetOPs proposal (Reza et al.) and Agent
    framework (Xing)
  • Jerome: part on data implication is relevant and connect to the
    data topic part of the NMRG research agenda
  • Mingzhe: insights on multi-agents to solve some specific
    networking tasks, any implementations
    • Chuyi: multi-agent collaboration
    • Diego: research problem is to ensure agents collaborate

• A Framework for LLM-Assisted Network Management with
  Human-in-the-Loop, draft-cui-nmrg-llm-nm, Mingzhe Xing, 10min + 5
  min Q&A

• Open discussions on GenAI in Network Management:
  -- Should we develop this topic within NMRG?
  -- How to develop this as a group (i.e. beyon individual
  contributions)?
  -- How to sync with other activities, RGs/WGs?