HotRFC Lightning Talks at IETF 126

Sunday, July 19, 2026
Starting Time: An hour after the Welcome Reception starts (18:00)
Room: Grand Klimt Hall 2
Organizers: Heather Flanagan, Liz Flynn
Email: hotrfc@ietf.org

Call for Participation


SESSION AGENDA

  1. Open Fibre Data Standard (OFDS) - mapping the terrestrial Internet
  2. BGP Secure Origin Delegation Authorization (SODA)
  3. PAVA: A new approach to BGP AS_PATH validation
  4. Beyond Throughput: Switching Efficiency for AI Data Center Networks
  5. DIVE: A DNS-based Alternative to CA-Based Code Signing for HTTP
    Artifacts
  6. Self-Certifying Identity and Capability-Based Delegation for
    Autonomous AI Agents
  7. PQC DNSSEC MTL Updates
  8. Interactions between L4S and FQ-based Queue Disciplines
  9. AI Inference Streaming: The Protocol Gap
  10. Practical Cyber Security
  11. AIDNS: Measuring Security and Stability Impacts of AI on the DNS
    Root
  12. KYAPay: Verified Agent Identity and Payments for Practical
    Interoperable Agentic Commerce on the Existing Web
  13. Agent Accountability, Composed — a verifiable record of what an AI
    agent did, across SCITT, RATS, and the agent-gateway work
  14. HTTP Events Query
  15. Confidential Computing and Digital Sovereignty
  16. KIRA – Routing for Resilient Control Planes
  17. Problem Statement and Roadmap for BGP FlowSpec Monitoring
  18. Who Authorizes the Request? Intent Admission in Agentic Systems
  19. Framework for Normalizing Multi-Vendor Network Inputs for
    LLM-Assisted Network Management
  20. Application-Centric Deployment across the IoT Computing Continuum
  21. From Fragmented Benchmarks to an Interoperable Format for
    Evaluating Network LLM Agents

1. Open Fibre Data Standard (OFDS) - mapping the terrestrial Internet

Presenter, Affiliation: Dan York, Internet Society, in person

Datatracker slides here

Abstract: OFDS is a community-driven solution for mapping the
Internet's terrestrial fiber networks. It provides a common language for
describing fiber optic networks, and enables the visualization (ex
maps), exchange, and analysis of fiber optic network infrastructure data
for researchers, network operators, governments and more. A Wednesday
side meeting is planned to explain OFDS, describe its uses so far, and
to gauge interest within the IETF/IRTF community for doing more with
OFDS.

Looking for: Collaborators and implementers - The community behind
OFDS is exploring whether the IETF might be the right standards
development organization (SDO) to bring OFDS to go through a formal
standardization process - and is seeking input from the IETF community.

Related I-Ds: More info at: https://ofds.info/en/

Coordinates:
Side meeting on Wednesday, July 22, 2026, from 14:45-15:45 in Grand
Klimt Hall 3
Contact Dan York - york@isoc.org - or Steve Song - song@isoc.org


2. BGP Secure Origin Delegation Authorization (SODA)

Presenter, Affiliation: Shenglin (Forrest) Jiang, Zhongguancun Laboratory, in person

Datatracker slides here

Abstract: Current Route Origin Authorization (ROA) allows a prefix
holder to authorize an origin AS, but it does not efficiently support
dynamic origin delegation scenarios such as DDoS mitigation, outsourced
routing, address leasing, and CDN traffic engineering. This talk
introduces BGP Secure Origin Delegation Authorization (SODA), a
mechanism that enables explicit, verifiable delegation authorization
while avoiding the operational overhead of frequent ROA updates. The
presentation will provide an overview of the design and invite
discussion on its applicability and future standardization.

Looking for: Feedback on the design, deployment considerations, use
cases, and implementation experience, as well as potential collaborators
interested in the topic.

Related I-Ds:
https://datatracker.ietf.org/doc/draft-jiang-idr-bgp-soda/

Coordinates: Shenglin (Forrest) Jiang, jiangshl@zgclab.edu.cn
Discussion is welcome on the IDR and SIDROPS mailing lists. I will also
be available during IETF 126 for further discussion.


3. PAVA: A new approach to BGP AS_PATH validation

Presenter, Affiliation: Maxence Fléchier, UGA (Université Grenoble Alpes), in person

Datatracker slides here

Abstract: BGP path security is an actively discussed domain. PAVA
tries to bring path validation through the use of the DNS. PAVA
complements ASPA to offer greater control and fine-grain tuning in AS
relationships, with the help of a different distribution system.

Looking for: Looking for suggestions and collaboration.

Related I-Ds:
https://datatracker.ietf.org/doc/draft-flechier-sidrops-pava/

Coordinates:

Contact us by mail (maxence.flechier@universite-grenoble-alpes.fr) or in
person during IETF126, at the GROW meeting where I will be presenting,
or other BGP-related meetings.


4. Beyond Throughput: Switching Efficiency for AI Data Center Networks

Presenter, Affiliation: Niangen Ye, Shanghai Jiao Tong University, in person

Datatracker slides here

Abstract: Large-scale LLM training is making communication
efficiency a central concern in AI data center networks. However, common
metrics such as throughput and utilization do not clearly show how
effectively switching resources contribute to useful training progress,
nor do they separate inefficiencies caused by communication semantics,
topology, and resource allocation.

This talk introduces our recent work on Switching Efficiency, a metric
framework that evaluates AI data center networks in terms of
computationally effective data throughput per unit switching capacity.
The framework further decomposes efficiency into three factors:
Data Efficiency, Routing Efficiency, and Port Utilization, to help
identify distinct bottlenecks.

Using this framework, we analyze representative architectures such as
3D-Torus and Rail-Optimized networks under dense and MoE LLM training
workloads. Our results suggest that while existing designs may align
reasonably well with sparse or imbalanced traffic, their efficiency can
degrade significantly under large-scale all-to-all communication.

We also examine how design choices such as tiered bandwidth allocation,
server size, in-network computing, and multi-plane design affect
different efficiency dimensions.

The goal of the talk is to share this early-stage framework with the
IETF/IRTF community and invite discussion on its usefulness for
measurement, observability, benchmarking, and architecture evaluation in
AI-oriented network environments.

Looking for: Feedback, discussion, and potential collaborators,
especially on metric definition, observability, measurement methodology,
and relevant IETF/IRTF venues for further discussion.

Related I-Ds:
https://datatracker.ietf.org/doc/draft-ye-ippm-switching-efficiency/

Coordinates:
* Niangen Ye, yng2020@sjtu.edu.cn
* IPPM mailing list post: https://mailarchive.ietf.org/arch/msg/ippm/lVcNIPRtZ4nMCIELZITHmLvZvVw/
* Paper: arXiv:2604.14690


5. DIVE: A DNS-based Alternative to CA-Based Code Signing for HTTP Artifacts

Presenter, Affiliation: Matéo Florian Callec, Independent, presenting remotely

Datatracker slides here

Abstract: DIVE (Domain-based Integrity Verification Enforcement) is
a lightweight protocol that verifies the integrity of HTTP artifacts by
publishing public keys directly in DNSSEC-protected DNS records. This
approach eliminates the need to trust a third-party Certificate
Authority, and makes key distribution and revocation as simple as
updating a DNS record.

Looking for:
I am looking for feedback, potential co-authors, collaborators and
implementers to see if there is interest in developing this concept
further.

Related I-Ds:
- I-D: https://datatracker.ietf.org/doc/draft-callec-dive/
- Open Source implementation: https://github.com/diveprotocol/opendive-client

Coordinates:
- Email: mateo@callec.net
- Website: https://diveprotocol.org


6. Self-Certifying Identity and Capability-Based Delegation for Autonomous AI Agents

Presenter, Affiliation: Andrzej Duda, UGA (Université Grenoble Alpes), in person

Datatracker slides here

Abstract: We propose self-certifying identifiers derived from public
keys, realized as DNS-based Self-certifying Identifiers (SIDs) and IPFS-
based Identifiers (IIDs), which provide lightweight decentralized
identities for AI agents.

Looking for: Looking for suggestions and collaboration

Related I-Ds: https://datatracker.ietf.org/doc/draft-duda-agent-id-framework/

Coordinates: Contact us by mail (andrzej.duda@univ-grenoble-alpes.fr) or in person during IETF126


7. PQC DNSSEC MTL Updates

Presenter, Affiliation: Andrew Kaizer, Verisign, in person

Datatracker slides here

Abstract: This presentation provides a quick summary of post-quantum
cryptography (PQC) challenges for DNSSEC and how our new draft using
Merkle Tree Ladder (MTL) with ML-DSA can address some of these
challenges as the IETF continues to standardize PQC DNSSEC solutions.

Looking for: Collaborators, implementers

Related I-Ds:

Coordinates: Find us throughout the week, email us (see forthcoming
slides), and/or attend the PQ-DNSSEC side meeting on Thursday at
9:30-11:00 at Grand Klimt Hall 3


8. Interactions between L4S and FQ-based Queue Disciplines

Presenter, Affiliation: Mohit P. Tahiliani, National Institute of Technology Karnataka,
Surathkal, India, presenting in-person

Datatracker slides here

Abstract: L4S enables applications to achieve low queuing delays
through scalable congestion control and ECN-based queue management. At
the same time, flow-queuing (FQ) based algorithms such as FQ-CoDel,
FQ-PIE, and CAKE are seeing deployment in operating systems, home
gateways, and ISP networks. While both technologies aim to improve
network performance, their interaction has received relatively little
attention.

In particular, existing flow-queueing mechanisms employ DRR-based
scheduling, which may introduce scheduler-induced latency even when
queue management maintains short queues. This raises several questions
regarding the treatment of L4S and Classic traffic, including scheduler
latency, quantum selection, fairness, sparse-flow optimization, and the
coexistence of L4S and Classic traffic.

This talk presents an open problem at the intersection of L4S and
FQ-based queue disciplines. It seeks community feedback on the
implications of DRR-based scheduling and whether existing FQ/L4S
mechanisms require additional considerations.

Looking for: We are looking for people interested in working in this
area, and can help us test and validate these challenges in real
deployments.

Related I-Ds:

Coordinates: Mohit P. Tahiliani, tahiliani@nitk.edu.in


9. AI Inference Streaming: The Protocol Gap

Presenter, Affiliation: Santosh Pallagatti, Zscaler, in person presentation.

Datatracker slides here

Abstract:

LLM inference providers stream response tokens over standardized
transports (HTTP/2, SSE, WebSocket) but use entirely vendor-specific
JSON schemas, event types, and stream lifecycle behaviors at the
application layer. The same logical operation, delivering a generated
text token to a client, is expressed differently by every provider, with
no common payload specification. This draft
(draft-spk-agentproto-llm-stream-00) proposes keeping SSE as the
transport and standardizing only the payload: a common request media
type (application/llm-request+json), a defined set of SSE event types
(stream.start, content.delta, tool.call, stream.end), and a JSON event
envelope with a canonical field for generated text. The result is a
single protocol contract for LLM inference streaming that any client
library, vendor, or platform can implement.

Looking for: collaborators

Related I-Ds:
* draft-spk-agentproto-llm-stream-00: yet to be uploaded and will be
done on 17th July when window opens.

Coordinates: Santosh Pallagatti, santosh.pallagatti@gmail.com


10. Practical Cyber Security

Presenter, Affiliation: Michael P, UK NCSC, presenting in-person

Datatracker slides here

Abstract: Cyber attacks are increasing in frequency, sophistication
and impact, posing significant threat to the security of networks, the
privacy of data and the availability of services. These attacks target
all systems, from critical infrastructure to enterprise networks to
individuals. The changes in attacker tradecraft means that a
collaborative approach is required.

This talk will introduce a side meeting scheduled for IETF 126 to
discuss the current active malicious threats the cyber security
community is combatting, increase understanding of how to defend against
them, and explore the challenges and changes that attendees are seeing.

Looking for: We are looking for collaboration. There is real benefit
in understanding the cyber risks from a range of perspectives to and
gain insight from across the community to upskill as a group.

Related I-Ds:

Coordinates:
* Michael P, Michael.p1@ncsc.gov.uk
* Side meeting details: Grand Klimt Hall 3 at 16:00-17:00 on Wednesday
22nd July. For more details see https://sidemeetings.ietf.org/


11. AIDNS: Measuring Security and Stability Impacts of AI on the DNS Root

Presenter, Affiliation: James Kunle Olorundare, SMIEEE (Nigerian Communications Commission / Internet Society Nigeria Chapter), in person

Datatracker slides here

Abstract: The DNS architecture was fundamentally designed for
human-centric browsing patterns, but the rapid transition to autonomous
"agentic" behaviors is creating massive, high-frequency
"cache-shattering" query traffic. This talk introduces the need for an
IRTF Research Group (AIDNS) to systematically study these novel,
machine-driven stressors on global resolution latency and Root Server
Operator (RSO) infrastructure resilience.

Looking for: Multi-disciplinary collaborators (network engineers and
AI researchers) to join the BoF, participate in a longitudinal telemetry
study, and help build open-source measurement toolkit prototypes during
the Hackathon.

Related I-Ds:

Coordinates: Interested participants can reach me directly during
the IETF meeting or contact me via email(olorundarek@gmail.com) to
coordinate mail lists and side-meeting setups. Or whatsapp:
+2348036551591


12. KYAPay: Verified Agent Identity and Payments for Practical Interoperable Agentic Commerce on the Existing Web

Presenter, Affiliation: Michael B. Jones, Self-Issued Consulting, LLC – Presenting in person

Datatracker slides here

Abstract: This presentation will introduce the KYAPay token, a JSON
Web Token (JWT) [RFC 7519] that is being used in practice to enable
seamless interoperable agentic commerce on the existing Web. It enables
parties involved in agentic interactions to decide whether to authorize
access by the agent based upon who the human or organizational principal
is authorizing the actions and properties of the agent. It eliminates
the need to subject agentic interactions to "Are you a human?" tests,
which currently reject all agentic interactions. We will describe how
KYAPay works and how it enables open participation in secure agentic
commerce authorized by human beings. See https://kyapay.org for some of
the organizations already using KYAPay.

This presentation will introduce the standards underpinnings that are
being developed for KYAPay and solicit feedback, collaboration, and
participation.

Looking for: Raising awareness of the KYAPay solution, soliciting
feedback on the approach and specifications, collaboration on applying
and refining the solution, and participating in advancing the necessary
standards for interoperable ubiquitous agentic commerce.

Related I-Ds:
* https://datatracker.ietf.org/doc/draft-skyfire-oauth-kyapay-token/
* https://datatracker.ietf.org/doc/draft-skyfire-oauth-amr-values/
* https://datatracker.ietf.org/doc/draft-skyfire-oauth-id-verification/
* https://datatracker.ietf.org/doc/draft-skyfire-oauth-kyapay-token-
exchange/

Coordinates: KYAPay will be presented at the Thursday OAuth working
group meeting. Michael Jones will be onsite in Vienna all week and
would welcome one-on-one or group discussions on the topic. Contact him
at michael_b_jones@hotmail.com or in person.


13. Agent Accountability, Composed — a verifiable record of what an AI agent did, across SCITT, RATS, and the agent-gateway work

Presenter, Affiliation: Steven Mih, Action State Group, in person

Datatracker slides here

Abstract: As AI agents take real actions across organizations that
do not share a trust domain, current IETF/IRTF work addresses different
parts of the record: SCITT for transparency and anchoring, RATS for
attestation, the emerging DMSC / agent-gateway work for routing and
authorization, kernel-side enforcement proposals, and security-principal
binding for verifier boundaries. This lightning talk sketches how the
pieces compose: CAN (authorization), WHO (identity), WHAT (the action
record), and AUDIT (the transparency substrate) stay separate verifier
outputs, joined by a shared action digest. A passing capsule proves the
record binding; it does not by itself prove authority, provenance truth,
physical completion, independent observation, or relying-party policy
acceptance. The interop target is one action digest, one capsule, and
separate verifier rows — including a deliberately failing row beside a
passing capsule-binding row — showing that composition preserves the
trust boundaries rather than collapsing them into one trust root. Four
minutes on the seams and where they don't yet meet; a cross-cutting
topic no single WG owns, raised for awareness and collaborators, not to
relitigate WG work.

Looking for:
collaborators and implementers across SCITT, RATS, the DMSC gateway
work, agent authorization, and security-principal binding — to align the
seams; and reviewers for the composition draft.

Related I-Ds:
draft-mih-sato-agent-accountability-composition;
draft-mih-scitt-agent-action-capsule;
draft-mih-agent-bilateral-attestation; draft-sato-soos-*;
draft-bu-agentproto-security-principal-binding;
draft-schrock-ep-authorization-receipts.

Coordinates: Steven Mih, spec@actionstate.ai
Reference implementation: https://github.com/action-state-group/agent-action-capsule
Evidence: an open reference verifier plus independent cross-verification
(SCITT/COSE receipts and a Microsoft CCF receipt), with positive and
deliberately-failing rows — current implementation evidence, not
complete interop coverage.


14. HTTP Events Query

Presenter, Affiliation: Rahul Gupta, presenting remotely

Datatracker slides here

Abstract: Events Query is a minimal protocol built on top of HTTP
that allows user agents to receive event notifications directly from any
resource of interest. It uses the QUERY method [RFC10008] to request
(optionally) the current representation and subsequent event
notifications within a single response. The Events Query Protocol (EQP)
is predicated on the idea that the most intuitive source for event
notifications is the resource itself.

Looking for:

Related I-Ds:
https://datatracker.ietf.org/doc/draft-gupta-httpapi-events-query/

Coordinates: Rahul Gupta, cxres+ietf@protonmail.com


15. Confidential Computing and Digital Sovereignty

Presenter, Affiliation: Muhammad Usama Sardar, TU Dresden and GA4GH, in person

Datatracker slides here

Abstract: Recent research on Confidential Computing has shown its
core trust mechanisms to be broken. For instance, research [0] has led
to the CVE-2026-33697 [1] of CVSS 7.5, which is the highest-scored
public vulnerability in more than decade of history of Confidential
Computing.

This indicates that further research is required on this topic and
motivates the need of forming a RG before forming a WG. We will invite
interested folks to a side meeting to discuss potential topics of
research and scope for the proposed charter of RG.

Looking for:
Collaborators who can help in:
* drafting charter and
* setting a reasonable scope

for the proposed RG.

Researchers who are interested in related topics.

Related I-Ds:
* Intra-handshake attestation: https://datatracker.ietf.org/doc/draft-fossati-tls-attestation/
* Post-handshake attestation: https://datatracker.ietf.org/doc/draft-fossati-seat-expat/
* Intra vs. post: https://datatracker.ietf.org/doc/draft-usama-seat-intra-vs-post/
* Hybrid https://datatracker.ietf.org/doc/draft-fossati-seat-early-attestation/
* Security considerations of remote attestation: https://datatracker.ietf.org/doc/draft-rats-sardar-sec-cons/

Coordinates: Muhammad Usama Sardar, muhammad_usama.sardar@tu-dresden.de

Side meeting: 11:00-12:00 (CEST) · Park Suite 4

Happy to make personal appointments during the week.

Wiki page
https://github.com/EuroProofNet/ProgramVerification/wiki/AttestedTLS

RATS and SEAT have some drafts on specific bits and pieces of
confidential computing. The scope of proposed RG is cross-cutting and
much wider than both.

Papers

Code

Slides and other material

Some recent slides and videos at https://github.com/CCC-Attestation/formal-spec-KBS


16. KIRA – Routing for Resilient Control Planes

Presenter, Affiliation: Roland Bless, KIT (Karlsruhe Institute of Technology), Germany, in person

Datatracker slides here

Abstract: KIRA is a scalable, zero-touch routing architecture that
offers resilient IPv6 connectivity without any configuration. It uses
ID-based addressing and scales to hundreds of thousands of nodes in a
single network. KIRA works well in various network topologies (even
mobile ad-hoc networks or LEO Satellite networks) and was designed to
offer resilient control plane connectivity that does not depend on
configuration nor any other services. A built-in DHT offers support for
service registration and discovery, thereby helping to realize autonomic
network management and control and zero-touch deployments.

In the age of AI governed networks, KIRA offers a reliable basis for
setting networks back to a clean state.

Looking for: collaborators, early implementers

Related I-Ds: https://datatracker.ietf.org/doc/draft-bless-rtgwg-kira/

Coordinates: Roland Bless, roland.bless@kit.edu


17. Problem Statement and Roadmap for BGP FlowSpec Monitoring

Presenter, Affiliation: Yujia Gao, Zhongguancun Lab, presenting in person

Datatracker slides here

Abstract: This talk introduces the operational challenges of
monitoring BGP FlowSpec deployments and presents a roadmap covering
route lifecycle, policy realization, enforcement status,
traffic-treatment outcomes, and correlation across different telemetry
mechanisms.

Looking for: We are looking for collaborators and early
implementers, and welcome feedback from network operators and equipment
vendors.

Coordinates: Yujia Gao, gaoyj@zgclab.edu.cn


18. Who Authorizes the Request? Intent Admission in Agentic Systems

Presenter, Affiliation: Yuning Jiang, Singapore Huawei Research Centre, in person

Datatracker slides here

Abstract: When an AI agent acts on a user's behalf, a valid token
may identify the caller and its general scope, but does not necessarily
prove that the specific request was authorized, permitted by policy, or
supported by required user consent. The agent may therefore attempt
actions beyond what was actually approved. We propose a signed admission
assertion issued before execution. An admission point authenticates the
request originator, evaluates applicable permission and consent
requirements, and binds the admitted request to the permitted action,
authorized presenter, and intended execution endpoint. The endpoint
verifies these bindings before acting, enabling request substitution and
unauthorized scope expansion to be detected and rejected.

Looking for: Collaborators.

Related I-Ds:
* draft-jiang-intent-security — threats, security requirements, and a reference model for admission and multi-hop intent processing: https://datatracker.ietf.org/doc/draft-jiang-intent-security/
* draft-jiang-oauth-intent-admission — a concrete realization using an Intent Admission Assertion: https://datatracker.ietf.org/doc/draft-jiang-oauth-intent-admission/

Coordinates: Yuning Jiang, jiangyuning2=40h-partners.com@dmarc.ietf.org


19. Framework for Normalizing Multi-Vendor Network Inputs for LLM-Assisted Network Management

Presenter, Affiliation: Shailesh Prabhu Nokia, in person

Datatracker slides here

Abstract: This talk introduces a framework for normalizing
heterogeneous network management inputs into a vendor-agnostic schema
for LLM-assisted network management. The proposed approach aims to
improve interoperability, consistency, explainability, and accuracy
while reducing dependence on vendor-specific prompt engineering.

Looking for:
* Reviewers to my draft.
* Feedback on the proposed architecture and schema design.
* Collaborators interested in standardizing reusable prompt schemas and
structured network inputs for AI-assisted operations.

Related I-Ds:
* Primary draft:
* https://datatracker.ietf.org/doc/draft-prabhu-nmrg-prompt-schema-llm/
* Related work:
* https://datatracker.ietf.org/doc/draft-yang-nmrg-mcp-nm/ (MCP for Network Management)
* https://datatracker.ietf.org/doc/draft-yang-nmrg-a2a-nm/ (Applicability of A2A to Network Management)

Coordinates:
* Shailesh Prabhu, shailesh.prabhu@nokia.com
* NMRG: https://datatracker.ietf.org/rg/nmrg/about/
* Feedback is welcome via the NMRG mailing list and during the NMRG
session at IETF 126.


20. Application-Centric Deployment across the IoT Computing Continuum

Presenter, Affiliation: Carlos Kamienski, Federal University of ABC (UFABC), Brazil, in person

Datatracker slides here

Abstract:
The edge-cloud computing continuum has been extensively studied from
infrastructure and communication perspectives (e.g., RFC 9556), but
comparatively little attention has been paid to the application side:
how smart applications are decomposed, placed, coordinated, and
managed across constrained devices, edge nodes, cloudlets, and clouds.
This talk presents an Internet-Draft in preparation
(draft-kamienski-t2trg-iot-deployment) that proposes a shared
vocabulary and a conceptual model for application-centric deployment
across the IoT computing continuum, treating placement, coordination
styles (orchestration and choreography), and lifecycle management as
interrelated concerns. The draft is grounded in two scenarios — an
operational smart irrigation deployment and a drone-based delivery
application — and outlines open research directions, including the
role of AI agents (and emerging protocols such as MCP and A2A) in
continuum deployment. This is an early-stage, cross-cutting topic
seeking broader awareness and fresh input from the IETF/IRTF community
before formal submission.

Looking for: Gauging community interest in application-centric
deployment as a research topic; collaborators, reviewers, and potential
co-authors for the Internet-Draft; implementers and providers of
additional use cases; and input on the most appropriate venue for this
discussion within the IRTF.

Related I-Ds:
- draft-kamienski-t2trg-iot-deployment (Kamienski, Santos, Di Felice) — preliminary version available on request
- Related background: RFC 9556; "IoTinuum: The IoT Computing Continuum" (DCOSS-IoT 2024); "IoTDeploy: Deployment of IoT Smart Applications over the Computing Continuum" (Internet of Things, Elsevier, 2024)

Coordinates:
- Email: carlos.kamienski@ufabc.edu.br
- A preliminary version of the draft is available on request to anyone interested.
- I will be on-site throughout the meeting week for hallway conversations.

21. From Fragmented Benchmarks to an Interoperable Format for Evaluating Network LLM Agents

Presenter, Affiliation: Chenguang Du, Beijing Zhongguancun Laboratory, in person

Datatracker slides here

Abstract: A growing number of projects are developing benchmarks for
LLMs and autonomous agents in networking. However, their task formats,
test cases, validation methods, and evaluation metrics are often
incompatible, making scenarios difficult to reuse and results difficult
to compare.

This talk introduces draft-cui-nmrg-llm-benchmark-02, which explores a
common, extensible format for describing network-agent evaluation tasks.
The proposed format combines network intents, topologies, initial
states, valid solutions, and executable test cases, with functional
outcomes serving as the primary measure of success. We hope to start a
broader discussion on interoperable datasets, reusable test cases, and
common evaluation conventions for the AI-for-networking community.

Looking for:
* feedback on whether a common network-agent benchmark exchange format
would be useful;
* contributors of task datasets, network scenarios, and executable test
cases;
* guidance on which aspects are suitable for IRTF research and which may
eventually benefit from IETF standardization.

Related I-Ds: draft-cui-nmrg-llm-benchmark-02

Coordinates:
Chenguang Du, du_chenguang@outlook.com
Source code and issue tracker:
https://github.com/nobrowning/draft_llm_conf_benchmark
Discussion mailing list:
NMRG — nmrg@irtf.org
https://www.ietf.org/mailman/listinfo/nmrg/