Agent Runtime Telemetry System
draft-wang-agent-runtime-telemetry-system-00
This document is an Internet-Draft (I-D).
Anyone may submit an I-D to the IETF.
This I-D is not endorsed by the IETF and has no formal standing in the
IETF standards process.
| Document | Type | Active Internet-Draft (individual) | |
|---|---|---|---|
| Authors | Xianghua Wang , Xuesen Liu , 李哲 , Mengjia Zhao | ||
| Last updated | 2026-07-05 | ||
| RFC stream | (None) | ||
| Intended RFC status | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-wang-agent-runtime-telemetry-system-00
Network Working Group X. Wang
Internet-Draft X. Liu
Intended status: Informational Z. Li
Expires: 7 January 2027 M. Zhao
CAICT
6 July 2026
Agent Runtime Telemetry System
draft-wang-agent-runtime-telemetry-system-00
Abstract
Large language model (LLM)-driven agents are increasingly deployed in
large-scale production environments, where multi-agent collaboration,
chained tool invocation, and long-context reasoning have become
standard execution patterns. However, existing distributed systems
telemetry frameworks primarily focus on infrastructure-level
telemetry, such as network traffic, host metrics, and service health,
and provide limited visibility into internal agent reasoning and
execution semantics.This results in several recurring challenges in
production environments, including inconsistent metric definitions
across platforms, lack of end-to-end execution traceability,
difficulty in root cause analysis, and limited interoperability among
heterogeneous telemetry systems.
This document defines a unified capability framework for Agent
Runtime Telemetry Systems. It standardizes two telemetry collection
planes: Network-side collection at gateways and traffic ingress
layers, and Agent-side instrumentation within the agent runtime
environment. It further specifies structured telemetry models,
cross-layer trace correlation mechanisms, runtime metrics, and
anomaly detection and remediation workflows.
The framework enables interoperable telemetry across heterogeneous
agent systems and deployment environments. It supports unified
telemetry ingestion, end-to-end execution trace reconstruction,
behavioral telemetry, and closed-loop anomaly analysis, providing a
standardized foundation for agent operations, reliability
engineering, and compliance auditing.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Wang, et al. Expires 7 January 2027 [Page 1]
Internet-Draft Agent Runtime Telemetry System July 2026
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 7 January 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions and Definitions . . . . . . . . . . . . . . . . . 4
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Overview of Agent Runtime Telemetry System . . . . . . . . . 4
5. Agent Data Collection . . . . . . . . . . . . . . . . . . . . 5
5.1. Network-side Collection . . . . . . . . . . . . . . . . . 6
5.2. Agent-side Collection . . . . . . . . . . . . . . . . . . 7
6. Runtime Monitoring . . . . . . . . . . . . . . . . . . . . . 8
6.1. Token Usage Monitoring . . . . . . . . . . . . . . . . . 8
6.2. Model Monitoring . . . . . . . . . . . . . . . . . . . . 9
6.3. Service Quality Monitoring . . . . . . . . . . . . . . . 11
6.4. Multi-Agent Collaboration Monitoring . . . . . . . . . . 12
7. Behavior Tracking . . . . . . . . . . . . . . . . . . . . . . 14
7.1. Decision Trace Tracking . . . . . . . . . . . . . . . . . 14
7.2. Tool Invocation Observation . . . . . . . . . . . . . . . 15
7.3. Session Interaction Observability . . . . . . . . . . . . 16
8. Anomaly Analysis & Remediation . . . . . . . . . . . . . . . 17
8.1. Baseline Deviation Analysis . . . . . . . . . . . . . . . 17
8.2. Root Cause Analysis . . . . . . . . . . . . . . . . . . . 18
8.3. Anomaly Alerting . . . . . . . . . . . . . . . . . . . . 18
Wang, et al. Expires 7 January 2027 [Page 2]
Internet-Draft Agent Runtime Telemetry System July 2026
8.4. Anomaly-Based Threat Library . . . . . . . . . . . . . . 19
9. Security Considerations . . . . . . . . . . . . . . . . . . . 20
10. Privacy Considerations . . . . . . . . . . . . . . . . . . . 20
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
12. Normative References . . . . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21
1. Introduction
With the rapid advancement of large language models, AI agents are
increasingly deployed in enterprise production systems. These
systems have evolved from isolated applications into complex
distributed architectures composed of multiple agents, model
services, tool execution services, and external system dependencies.
This evolution introduces new challenges in runtime telemetry,
behavior tracing, failure localization, and unified operational
management.
Compared to traditional distributed applications, agent execution
workflows exhibit fundamentally different runtime characteristics. A
single task execution may involve multiple dynamically composed
stages, including model inference, tool invocation, context
management, long-term memory access, and multi-agent collaboration.
These execution paths are stateful, non-linear, and tightly coupled
across system components, making them difficult to observe using
conventional infrastructure-centric telemetry approaches.
Existing telemetry systems for agent-based workloads are insufficient
in several key aspects. First, most systems focus on coarse-grained
runtime metrics such as service availability, request throughput, and
system resource utilization. They lack visibility into fine-grained
execution semantics, including reasoning steps, tool invocation
chains, and failure causality, limiting their effectiveness for
debugging and operational optimization. Second, current telemetry
models are primarily designed for traditional distributed systems.
They emphasize network-level requests and service metrics but do not
provide a unified representation for agent-native behaviors, such as
inference traces, context evolution, memory operations, and inter-
agent communication. As a result, they fail to capture the semantic
structure of agent execution workflows. Third, the ecosystem remains
fragmented, with no widely adopted standard for agent telemetry
collection, trace representation, or cross-platform telemetry
interfaces. This fragmentation leads to poor interoperability across
frameworks and vendors, making it difficult to implement unified
telemetry in large-scale multi-agent deployments.
Wang, et al. Expires 7 January 2027 [Page 3]
Internet-Draft Agent Runtime Telemetry System July 2026
This document defines a standardized capability framework for Agent
Runtime Telemetry System. It specifies requirements across four
dimensions: Agent Data Collection, Runtime monitoring, Behavior
Tracking, Anomaly Analysis & Remediation.
The framework is designed to support heterogeneous agent frameworks
and multi-deployment environments. It enables consistent telemetry
collection, unified metric semantics, end-to-end trace
reconstruction, and closed-loop anomaly analysis, thereby providing a
foundation for telemetry-driven agent operations, reliability
engineering, and compliance governance.
2. Conventions and Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
3. Terminology
The following terms are defined in this document:
* Runtime Telemetry: Continuous structured monitoring capability for
agent runtime behaviors, resource consumption and interaction
chains, outputting standardized telemetry signals for consumption
by operation, governance and audit systems.
* Behavior Trace: A complete execution path record covering the full
lifecycle of a single task processed by an agent from reception to
completion, including causal sequences of inference steps, tool
invocations, intermediate decisions and final outputs.
* Service Level Agreement(SLA): A formal agreement between a service
provider and a service consumer that defines measurable service
quality objectives, including availability, performance, and
reliability guarantees, as well as remediation or penalty
mechanisms in case of non-compliance.
4. Overview of Agent Runtime Telemetry System
This document defines four layers of standardized telemetry technical
capabilities with hierarchical dependencies and mutual coordination,
following a complete technical pipeline: data collection, real-time
metric monitoring, post-hoc behavior tracing, and anomaly analysis.
Wang, et al. Expires 7 January 2027 [Page 4]
Internet-Draft Agent Runtime Telemetry System July 2026
This document defines the capability framework of the Agent Runtime
Telemetry System, covering four dimensions: Agent Data Collection,
Runtime Monitoring, Behavior Tracking, and Anomaly Analysis &
Remediation.
Agent Data Collection serves as the foundational telemetry layer of
the telemetry system. It standardizes two data collection planes--
Network-side and Agent-side instrumentation--and defines unified
constraints for identifiers, transmission mechanisms, and base data
structures.This layer aligns with the global identification system
defined in Chapter 7, including trace_id, span_id, and session_id,
providing a unified data foundation for higher-level monitoring,
tracing, and analytical capabilities.
The Runtime Monitoring captures real-time operational state and
resource consumption during agent task execution, providing online
situational awareness of system behavior.It consists of four
subdomains: Token Usage Monitoring, Model Monitoring, Service Quality
Monitoring, and Multi-Agent Collaboration Monitoring.
The Behavior Tracing reconstructs complete execution traces after
task completion in a structured manner, enabling interpretability,
failure reproduction, and compliance auditing.It achieves full
lifecycle reconstruction of agent reasoning logic, invocation
behavior, and interaction processes through decision trace tracking,
tool call tracing, and session interaction observability.
The Anomaly Analysis & Remediation provides a closed-loop analytical
system based on telemetry from the previous three layers, enabling
anomaly detection, root cause analysis, and coordinated mitigation
actions.This capability enables continuous detection and governance
of anomalous agent behavior through baseline deviation analysis, root
cause localization, anomaly alerting, and an anomaly-based threat
library.
5. Agent Data Collection
The Agent Data Collection focuses on standardized collection
mechanisms for telemetry data throughout the Agent lifecycle, forming
the data foundation of the Agent Runtime Telemetry System. Its
objective is to acquire various types of telemetry signals required
for agent operation through unified data collection modes.
Unlike traditional distributed systems, agent systems are typically
driven by large language models and include complex behavioral paths
such as inference generation, tool invocation, context management,
and multi-agent collaboration. The telemetry system MUST NOT only
capture basic runtime information but also reflect key events,
Wang, et al. Expires 7 January 2027 [Page 5]
Internet-Draft Agent Runtime Telemetry System July 2026
context changes and cross-component interaction relationships
generated during agent task execution, thereby supporting runtime
metrics (Chapter 6), behavior tracing (Chapter 7), and anomaly
analysis and remediation (Chapter 8).
This document defines two standardized data collection modes:
Network-side Collection and Agent-side Collection.
5.1. Network-side Collection
Network-side Collection observes the full lifecycle of agent requests
within communication channels, covering key stages including request
initiation, routing and forwarding, model invocation, tool
invocation, response return, and security policy enforcement.
This mode is deployed at gateways or traffic ingress layers without
requiring intrusion into or modification of the agent runtime
environment, providing low-intrusion deployment, broad coverage, and
strong centralized governance capabilities.
Network-side Collection MUST satisfy the following requirements:
* MUST generate or propagate a globally unified trace_id at gateway
ingress and egress for cross-system correlation;
* MUST record full request lifecycle timestamps, including
request_start_timestamp and request_end_timestamp;
* MUST structurally record token_input_total, token_output_total,
and token_cost_estimate for cost and resource analysis;
* MUST record request runtime status using predefined enumerations;
* MUST attach basic identifiers such as tenant_id, agent_id, and
model_id to all telemetry data;
* MUST use asynchronous or non-blocking reporting mechanisms without
impacting the request path;
* MUST support configurable sampling strategies by tenant and
traffic ratio.
Network-side Collection SHOULD provide the following capabilities:
* SHOULD maintain mapping relationships between trace_id and agent-
side span_id for cross-layer trace reconstruction;
* SHOULD provide tenant- and role-based access control;
Wang, et al. Expires 7 January 2027 [Page 6]
Internet-Draft Agent Runtime Telemetry System July 2026
* SHOULD support short-term caching of trace relationships for
efficient backtracking;
* SHOULD support configurable data masking for sensitive inputs and
outputs.
Network-side Collection MAY include:
* MAY collect network-layer metrics such as TCP state, packet loss,
and bandwidth usage;
* MAY support custom business tags for enhanced analytics.
5.2. Agent-side Collection
Agent-side Collection is a mechanism within the agent runtime
environment that structurally captures execution processes via
embedded telemetry mechanisms (such as SDK instrumentation, runtime
instrumentation, or proxy forwarding). Its scope includes inference
behavior, tool invocation, memory access, and context evolution.This
mode reconstructs the complete internal execution path of the agent
and correlates it with network-side telemetry to enable end-to-end
telemetry.
Agent-side Collection MUST satisfy the following requirements:
* MUST generate or inherit trace_id consistent with network-side
rules;
* MUST ensure trace_id is unique and non-reusable across the full
task lifecycle, and support propagation across agents;
* MUST assign a unique span_id to each atomic execution unit;
* MUST construct hierarchical execution structures (Execution Tree)
using parent_span_id;
* MUST structurally record span inputs, states, outputs, and
millisecond-level UTC timestamps;
* MUST record full tool invocation lifecycle, including parameters,
results, status, and retry counts;
* MUST ensure trace_id consistency with network-side telemetry for
cross-layer fusion.
Agent-side Collection SHOULD provide the following capabilities:
Wang, et al. Expires 7 January 2027 [Page 7]
Internet-Draft Agent Runtime Telemetry System July 2026
* SHOULD fully record memory operation lifecycle (create, read,
update, delete);
* SHOULD record complete temporal sequences of tool invocations;
* SHOULD support multiple integration methods including embedded
instrumentation, runtime instrumentation, and proxy forwarding;
* SHOULD support sampling strategies based on full, proportional, or
tenant-level configurations;
* SHOULD support masking of sensitive information in inputs and
outputs;
* SHOULD include standardized metadata fields (agent_id, session_id,
tenant_id, model_version).
Agent-side Collection MAY include:
* MAY support cross-sub-agent trace stitching in multi-agent
scenarios;
* MAY collect runtime resource metrics such as CPU, GPU, and memory
usage;
* MAY support local caching and compensatory reporting after network
recovery.
6. Runtime Monitoring
The Runtime Monitoring focuses on telemetry of underlying Agent
infrastructure and runtime workload. It serves as the fundamental
data source for consumers to assess Agent availability and for
operators to perform capacity planning and fault localization.
6.1. Token Usage Monitoring
Token Usage Monitoring is a core dimension for measuring Agent cost
efficiency and resource consumption. It quantifies computational
resources consumed by input and output text during large language
model execution and provides the basis for billing, rate limiting,
and performance optimization.
Token Usage Monitoring SHOULD record the following signals:
* token_input_total measures total input token consumption within
the monitoring window. It includes user input, system prompts,
historical context, and tool outputs re-injected into the model,
Wang, et al. Expires 7 January 2027 [Page 8]
Internet-Draft Agent Runtime Telemetry System July 2026
excluding model-generated output tokens.In implementation, the
system MUST support aggregation across tenant, agent, and model
dimensions and remain consistent with billing metrics, SHOULD
support time-granularity trend analysis to detect input complexity
changes, and MAY support business label extension for cost
attribution.
* token_output_total measures the cumulative number of output tokens
generated by the model within the monitoring time window,
reflecting model generation load and response complexity.
Semantically, it only counts tokens produced by the model
output.In implementation, this metric MUST use consistent time
windows and statistical boundaries with token_input_total to avoid
double counting or bias, SHOULD support model version comparison,
and MAY support output cost allocation modeling.
* token_cost_estimate represents the estimated token cost calculated
based on model pricing rules, used for cost accounting and billing
analysis. It is derived from token_input_total and
token_output_total according to model unit pricing and does not
represent the actual billing amount.In implementation, the
computation MUST be traceable and consistent with the billing
system.It SHOULD support versioned pricing across models, and MAY
support multi-currency conversion.
* token_per_request_avg represents the average token consumption per
request within the monitoring time window, reflecting request
complexity and interaction density. It is a derived metric
defined as the ratio of total tokens to total request count.It
SHOULD support breakdown by session and agent dimensions.It MAY be
used to identify abnormally high-consumption requests.
6.2. Model Monitoring
Model Monitoring is a dedicated telemetry dimension for large
language model execution, used to describe how inference latency,
throughput, hardware load, and version differences affect system
behavior, and serves as one of the core telemetry capabilities
distinguishing agents from traditional systems.
Model Monitoring SHOULD record the following signals:
* inference_latency_ms measures the average end-to-end time required
for the model to complete an inference request within the
monitoring window, reflecting overall response efficiency. It
includes the full execution duration from request ingress to final
output completion without distinguishing internal stages.In
implementation, the metric MUST support aggregation by model_id
Wang, et al. Expires 7 January 2027 [Page 9]
Internet-Draft Agent Runtime Telemetry System July 2026
and version.It SHOULD support distributed sampling to reduce
overhead under high concurrency.It MAY support automatic marking
of abnormal latency for subsequent root cause analysis.
* time_to_first_token_ms measures the average time required for the
model to generate the first token within the monitoring window,
reflecting model startup and context processing efficiency. It is
defined as the duration from request ingress into the model to the
generation of the first output token.In implementation, it MUST
use a consistent statistical definition with
inference_latency_ms.It SHOULD support distinguishing the impact
of different context lengths.It MAY be used for streaming
experience optimization analysis.
* tokens_per_second measures the average rate of token generation
within the monitoring window, reflecting model inference
throughput. It is defined as the average number of output tokens
per unit time.In implementation, it SHOULD support dynamic window
calculation and MUST remain consistent with output token
statistics.It MAY be used for performance comparison across
different hardware environments.
* gpu_utilization and cpu_utilization measure GPU and CPU
utilization ratios during model execution within the monitoring
window, reflecting compute intensity and resource bottlenecks.
They represent average or sampled utilization values over the
monitoring period.In implementation, sampling frequency MUST be
configurable, and multi-instance aggregation SHOULD be
supported.It MAY be used for capacity planning and elastic
scheduling.
* model_error_rate represents the error rate of model service
invocations within the monitoring window, reflecting service
stability. It is calculated as the number of failed requests
divided by the total number of requests, covering timeouts,
inference failures, and internal service exceptions.In
implementation, it SHOULD support categorized error statistics and
MAY be used to identify abnormal model versions.
* context_window_usage measures the context window utilization ratio
within the monitoring window, reflecting context pressure and
long-context handling capability. It is defined as the ratio of
used tokens to maximum context length.In implementation, it MUST
align with the model's actual context limit and SHOULD support
cross-model version comparison.It MAY be used for evaluation of
long-context optimization strategies.
Wang, et al. Expires 7 January 2027 [Page 10]
Internet-Draft Agent Runtime Telemetry System July 2026
6.3. Service Quality Monitoring
The Service Quality Monitoring capability domain focuses on telemetry
of external service-level performance of agents and serves as the
core basis for determining whether service requirements are met.
Service Quality Monitoring SHOULD record the following signals:
* request_success_rate measures the proportion of successful
requests within the monitoring window, reflecting service
availability and stability. Successful requests are those that
return valid business results, excluding errors and timeouts.
* request_error_rate measures the proportion of failed requests
within the monitoring window, reflecting system anomaly levels.
It includes model errors, system errors, and business-level
failures.
* request_timeout_rate measures the proportion of timed-out requests
within the monitoring window, reflecting insufficient system
responsiveness. It is defined as the proportion of requests
exceeding the maximum response time threshold.In implementation,
it MUST use a unified time baseline with latency metrics and
SHOULD support configurable timeout thresholds.
* latency_p50 / p95 / p99 measure latency distribution within the
monitoring window, representing the 50th, 95th, and 99th
percentiles of request latency. p50 reflects typical experience,
p95 reflects common user experience, and p99 reflects worst-case
performance.In implementation, p99 MUST be used as the SLA
threshold and alerting baseline, and hierarchical aggregation
SHOULD be supported.
* availability_rate measures service availability within the
monitoring window, reflecting the proportion of time the system
can respond normally. It is calculated as healthy runtime divided
by total window duration.In implementation, it MUST support multi-
instance aggregation and SHOULD align with SLA systems.
* requests_per_second (RPS) measures system throughput within the
monitoring window, defined as the average number of requests
processed per second.In implementation, it SHOULD support dynamic
time window calculation.
Wang, et al. Expires 7 January 2027 [Page 11]
Internet-Draft Agent Runtime Telemetry System July 2026
6.4. Multi-Agent Collaboration Monitoring
Multi-Agent Collaboration Monitoring characterizes communication
behavior and collaboration efficiency among multiple agents during
cooperative task execution. It serves as a key telemetry dimension
for evaluating system-level autonomy and coordination capability.
This dimension focuses on inter-agent message passing, task
distribution, and collaborative execution outcomes.
Communication behaviors between agents within multi-agent systems
SHOULD record the following signals:
* inter_agent_message_total measures the total number of inter-agent
communication messages within the monitoring time window,
reflecting the level of collaborative activity in the system. The
metric is defined as the cumulative count of all explicit inter-
agent communication events within the window, where explicit
communication refers to business interaction messages actively
transmitted between agents with clearly defined sender and
receiver identities. It excludes implicit interactions such as
in-process memory access or local function invocations that do not
involve cross-agent message transmission. Each message MUST be
assigned a unique message_id and MUST include sender_agent_id and
receiver_agent_id. In implementation, this metric MUST align with
the trace system to ensure each message can be mapped to a unique
trace_id or span_id. It MUST guarantee deduplication consistency
so that the same message_id is not counted more than once. It
SHOULD support aggregation by agent_pair (sender -> receiver) for
communication topology analysis, and MAY support task-level
(task_id) aggregation for communication density analysis.
* inter_agent_latency_ms measures the average communication latency
between agents within the monitoring time window, reflecting
collaboration efficiency and message delivery performance. It is
defined as the time difference between a message_send event and a
message_ack or response_received event for a single message. In
implementation, the system MUST ensure a unified clock source
across all nodes to avoid time skew. The metric MUST be bound to
message_id to ensure per-message traceability. It SHOULD support
segmented latency analysis, including network delay, queue delay,
and processing delay. It MAY support aggregated statistical
distributions (p50/p95/p99) by agent_pair or task_id.
* inter_agent_error_rate measures the proportion of failed inter-
agent communication events within the monitoring time window,
reflecting the stability of multi-agent collaboration. The metric
is defined as the number of failed communication events divided by
the total number of inter-agent communication events in the same
Wang, et al. Expires 7 January 2027 [Page 12]
Internet-Draft Agent Runtime Telemetry System July 2026
window. Failure events include message loss, message rejection,
communication timeout, serialization errors, and routing failures.
In implementation, this metric MUST use the same message_id scope
as inter_agent_message_total to ensure consistent counting. It
MUST explicitly classify error sources (transport / routing /
timeout / protocol / agent rejection). It SHOULD support
breakdown by error type, and MAY support aggregation by agent_pair
or network path for failure distribution analysis.
* collaboration_success_rate measures the proportion of successfully
completed collaborative tasks within the monitoring time window,
reflecting the overall coordination capability of the multi-agent
system. The metric is defined as the number of tasks that satisfy
predefined completion criteria divided by the total number of
collaborative tasks that have been initiated and fully executed
within the window. Only tasks that have completed execution
within the observation window are included, and success
determination MUST strictly follow predefined task completion
conditions. In implementation, this metric MUST align with the
task_id identification system to ensure deduplication at the task
level. It MUST explicitly define success criteria, including both
subtask completion status and final output validation results. It
SHOULD support analysis by task_type or agent composition, and MAY
incorporate evaluator or judge models as part of success
determination.
* distributed_task_completion_time measures the average end-to-end
duration of distributed collaborative tasks from start to
completion within the monitoring time window, reflecting system
scheduling and coordination efficiency. It is defined as the time
difference between task_start_event and task_complete_event for
the same task_id. In implementation, this metric MUST align with
both trace_id and task_id to ensure cross-agent execution paths
can be reconstructed. It MUST ensure semantic consistency of
start and end events (local span timestamps MUST NOT be mixed with
global task timing). It SHOULD support phase decomposition,
including planning, execution, and coordination stages. It MAY
support bucketing analysis by task complexity or number of
participating agents.
* agent_active_count measures the number of agents participating in
execution within the monitoring time window, reflecting the scale
and activity level of system collaboration. It is defined as the
deduplicated count of agent_id values that have participated in at
least one message exchange or task execution during the window.
In implementation, this metric MUST rely on a globally unique
agent_id system for deduplication. It MUST clearly define the
boundary of "active" (at least one message sent, received, or
Wang, et al. Expires 7 January 2027 [Page 13]
Internet-Draft Agent Runtime Telemetry System July 2026
participation in task execution). It SHOULD support dynamic
aggregation by task or time window, and MAY distinguish between
active participation (sender) and passive participation
(receiver).
7. Behavior Tracking
The Behavior Tracking focuses on the telemetry of agent decision-
making processes, tool invocation behaviors, and interaction content.
It provides structured reconstruction capabilities for understanding
what an agent did, how it was done, and why it was performed during
execution.
This dimension achieves explainability, post-hoc reproducibility, and
compliance auditing of agent behavior through unified modeling of
reasoning traces, tool invocation chains, session interactions, and
end-to-end execution paths.
7.1. Decision Trace Tracking
Decision Trace Tracking records the complete reasoning path of an
agent from task reception to final output generation. It serves as
the core telemetry mechanism for understanding agent decision logic.
Each agent inference invocation SHOULD generate a standardized
reasoning step record containing the following fields:
* trace_id is the globally unique identifier for a single
intelligent task execution, spanning the full lifecycle of model
inference, multi-round tool invocation, and session-level
interaction. Semantically, a single trace_id corresponds to the
complete lifecycle of one task, and all telemetry events,
reasoning steps, and tool invocation records under the same task
MUST share the same trace_id for correlation. In implementation,
the system MUST guarantee global uniqueness and non-reusability of
trace_id. It SHOULD support deterministic generation based on
external business seeds. The generation rules MUST be consistent
across network-side and agent-side instrumentation to enable
cross-layer trace correlation. The system SHOULD support trace_id
propagation across sub-agents in multi-agent scenarios to fully
reconstruct distributed execution paths.
* span_id is the unique identifier for the smallest execution unit
within a trace, representing atomic operations such as model
inference, tool invocation, or sub-agent execution. All spans
under the same trace_id form a hierarchical execution tree via
parent_span_id, where the root span has no parent. In
implementation, span_id MUST be globally unique and MUST NOT be
Wang, et al. Expires 7 January 2027 [Page 14]
Internet-Draft Agent Runtime Telemetry System July 2026
reused. It MUST be propagated across network-side, agent-side,
and sub-agent boundaries, and child spans MUST inherit the same
trace_id while maintaining correct parent-child relationships.
The system SHOULD support attaching custom attributes and event
annotations to spans for fine-grained performance and anomaly
tracking.
* reasoning_action identifies the type of reasoning behavior
performed in the current step, used to describe the decision
semantics of the agent. Valid values include plan, think,
observe, decide. In implementation, this field MUST use a unified
enumeration and SHOULD maintain consistency across models.
* input_context records the input context provided to the model at
the current reasoning step, representing the full set of
information used for inference. In implementation, it SHOULD
support desensitization mechanisms and MUST be bound to span_id.
* decision_output records the output of the current reasoning step,
representing the decision result of the current span. In
implementation, it MUST be correlated with the inputs of
subsequent spans to ensure trace continuity.
* timestamp records the time at which the observed event occurs
during execution, ensuring consistent ordering across spans, tool
invocations, and sessions. It MUST use a unified UTC time
baseline and MUST ensure cross-system time alignment.
7.2. Tool Invocation Observation
The Tool Invocation Observation records the complete lifecycle of
external tool, API, or service invocations performed by an agent. It
is a critical capability for identifying functional failures and
performance bottlenecks.
Tool invocation observations SHOULD record the following fields:
* tool_call_id is the unique identifier for a single tool invocation
event, representing a complete tool execution lifecycle. In
implementation, this identifier MUST be globally unique.
* tool_name identifies the name of the invoked tool, representing
the external capability or service dependency. In implementation,
it SHOULD support version binding.
* input_parameters records the input parameters of the tool
invocation, used for reproducing execution behavior. In
implementation, it MUST support desensitization mechanisms.
Wang, et al. Expires 7 January 2027 [Page 15]
Internet-Draft Agent Runtime Telemetry System July 2026
* output_result records the output returned by the tool, used for
evaluating execution correctness and effectiveness. In
implementation, it SHOULD support size limitation and compression.
* call_status records the execution status of the tool invocation,
including success, failure, timeout, and similar states. In
implementation, it MUST be analyzed jointly with retry_count.
* retry_count records the number of retries for the same
tool_call_id, used to analyze unstable dependencies. In
implementation, it MUST remain consistent with call_status.
* error_message records the failure reason of the tool invocation,
used for fault diagnosis. In implementation, it SHOULD support
error classification.
7.3. Session Interaction Observability
Session Interaction Observability targets the telemetry of dialogue
behavior between agents and callers, covering the full interaction
lifecycle from session establishment to termination.
The Session Interaction Observability capability domain SHOULD cover
the following telemetry fields:
* session_id is the unique identifier for a single agent interaction
session. All inference requests and tool invocations within the
same session SHOULD share this identifier. A parent_session_id
MAY be used to represent nested subtask hierarchies. In
implementation, session_id MUST be transmitted via standard
request headers, SHOULD NOT be reused after session termination,
and SHOULD support session affinity-based routing. The system
SHOULD correlate session_id with trace_id for end-to-end telemetry
and trace reconstruction.
* conversation_turn_id identifies the sequential order of
interaction turns within a session, used to reconstruct dialogue
ordering. Semantically, it increases monotonically over time. In
implementation, ordering consistency MUST be guaranteed across
distributed systems.
* user_input records the input content provided by the user, used
for intent analysis and interaction behavior understanding.
Semantically, it represents the raw input text. In
implementation, it SHOULD support desensitization processing.
Wang, et al. Expires 7 January 2027 [Page 16]
Internet-Draft Agent Runtime Telemetry System July 2026
* agent_response records the output generated by the agent, used for
evaluating interaction quality and system behavior. Semantically,
it represents model-generated response content. In
implementation, it SHOULD support mixed structured and
unstructured representations.
8. Anomaly Analysis & Remediation
The Anomaly Analysis & Remediation builds a closed-loop telemetry
capability for agent runtime systems based on telemetry data
collected from Agent Data Collection (Chapter 5), Runtime Monitoring
(Chapter 6), and Behavior Tracking (Chapter 7).
It uses trace_id, span_id, and session_id as unified correlation keys
to jointly analyze runtime metrics and behavior traces within
consistent time windows, enabling anomaly detection, localization,
attribution, and remediation.
8.1. Baseline Deviation Analysis
Baseline Deviation Analysis is used to identify deviations of agent
runtime behavior relative to historical normal behavioral patterns
and serves as the fundamental mechanism for anomaly detection.
This mechanism is built upon the complete runtime metric system
defined in Chapter 6. The system performs joint observation of the
above metrics within a time window and compares them against
historical baseline distributions to determine whether the current
runtime state exhibits significant deviation. A unified deviation
score is generated to represent the overall anomaly severity. This
score does not depend on any single metric but reflects the
aggregated variation trends of multi-dimensional metrics within the
same observation window.
In implementation, baseline models MUST support hierarchical modeling
across tenant, agent, and model dimensions to accommodate differences
in workload patterns and model behaviors. Meanwhile, baseline models
MUST support versioned management to ensure consistency and
traceability during model iteration and system evolution.Baseline
models MUST also support versioned management to ensure consistency
and traceability across model iterations and system evolution.When
performing deviation detection, the system MUST perform joint
verification using both runtime metrics and behavior traces to reduce
false positives caused by numerical fluctuations alone, and SHOULD
support continuous baseline updates over time to adapt to long-term
environmental drift.
Wang, et al. Expires 7 January 2027 [Page 17]
Internet-Draft Agent Runtime Telemetry System July 2026
8.2. Root Cause Analysis
Root Cause Analysis is used to systematically attribute anomalies to
their origin points, propagation paths, and impact scopes after
anomaly occurrence, thereby establishing causal relationships between
metric anomalies and behavior traces.
This capability relies on the trace/span execution tree structure
defined in Chapter 7, combined with time-aligned changes in runtime
metrics defined in Chapter 6, to perform cross-layer correlation
analysis of anomalies.
During analysis, the system first identifies key metrics that exhibit
significant deviations within the anomaly time window, such as
increased latency, abnormal Token consumption, elevated error rates,
or expanded multi-agent communication delays. These abnormal signals
are then correlated with corresponding trace_id values to locate the
affected execution traces.
The system then performs stepwise backtracking along the execution
structure constructed via span_id to identify the earliest node where
abnormal signals appear. It further analyzes whether the node
belongs to the model inference stage, tool invocation stage, or
multi-agent collaboration stage, thereby classifying the anomaly
source as one of the following types: model anomaly, tool anomaly,
system anomaly, or collaboration anomaly.
The system SHOULD also analyze the propagation path of anomalies
across the execution trace to determine the scope and diffusion
pattern of the impact.
The final output of Root Cause Analysis SHOULD provide a complete
description of the anomaly origin node, propagation path, and
affected scope, and SHOULD support cross-session and cross-agent
correlation to enable interpretable reconstruction of complex agent
system behaviors.
8.3. Anomaly Alerting
Anomaly Alerting converts anomalies detected through runtime metrics
and baseline deviation analysis into real-time actionable alert
events, enabling rapid response and risk control during system
operation.
Wang, et al. Expires 7 January 2027 [Page 18]
Internet-Draft Agent Runtime Telemetry System July 2026
Alert triggers originate from three categories of signals: baseline
deviation detection results, critical runtime metric anomalies, and
behavior trace anomalies. When the system determines that anomaly
severity exceeds predefined thresholds, a standardized alert event is
generated.
Each alert event MUST maintain strong correlation with trace_id,
session_id, and agent_id, and MUST explicitly include the key metric
source that triggered the alert to ensure full contextual
interpretability.
The system SHOULD support multi-level alerting mechanisms, including
informational, warning, and critical levels, with dynamic severity
classification based on anomaly intensity.
In implementation, the system MUST support alert deduplication and
aggregation to prevent alert storms under high-frequency fluctuation
scenarios. It SHALL also support alert lifecycle management,
including state transitions such as triggered, acknowledged, in-
progress, and resolved/closed.
8.4. Anomaly-Based Threat Library
The Anomaly-Based Threat Library is used to structurally store
historical anomaly events, enabling reusable anomaly pattern
recognition and continuous improvement of system detection and
analysis capabilities.
This capability is built by integrating historical anomaly events and
root cause analysis results, and jointly abstracting them with
runtime metric patterns (Chapter 6) and behavior trace patterns
(Chapter 7), forming a semantically consistent set of anomaly
patterns.
These anomaly patterns include both metric-side features and
behavior-side features, and establish associations with corresponding
root cause types and remediation strategies, forming complete
knowledge units.
During system evolution, newly observed anomaly events MAY be
continuously fed back into the threat library to extend or refine
existing anomaly patterns.The threat library MUST support versioned
management to ensure compatibility and stability of detection rules
during iterative evolution.Additionally, the system MAY support
similarity-based anomaly retrieval over historical patterns to
facilitate rapid identification of known issues, and MAY enable
cross-system sharing and reuse of anomaly patterns.
Wang, et al. Expires 7 January 2027 [Page 19]
Internet-Draft Agent Runtime Telemetry System July 2026
9. Security Considerations
This document defines an agent runtime telemetry framework covering
data collection, metrics monitoring, behavior tracing, and anomaly
analysis. Security properties depend on the deployment environment
and the trust assumptions of the telemetry pipeline.
The system introduces several security considerations, including but
not limited to agent identity spoofing, telemetry data manipulation,
trace and span falsification, cross-agent message forgery, and
potential information leakage through telemetry data exposure.
Telemetry data SHOULD be treated as attested runtime telemetry rather
than ground truth about system behavior or external execution
reality.In deployments using a trusted gateway, enforcement and
validation of telemetry data MAY be performed at the gateway
boundary. In SDK-based deployments without external enforcement, the
correctness of telemetry data is conditional on agent and runtime
integrity.
This document does not address runtime compromise scenarios or
provide guarantees under fully compromised execution environments.
10. Privacy Considerations
This framework may collect sensitive runtime information, including
user inputs, model outputs, tool invocation parameters, and inter-
agent communication data. Implementations SHOULD apply data
minimization and access control to prevent unauthorized
exposure.Sensitive data SHOULD be masked or anonymized when stored,
transmitted, or used for analysis. Multi-tenant systems MUST enforce
strict isolation of telemetry data across tenants.
11. IANA Considerations
This document has no IANA actions.
12. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
Wang, et al. Expires 7 January 2027 [Page 20]
Internet-Draft Agent Runtime Telemetry System July 2026
Authors' Addresses
Xianghua Wang
CAICT
Huayuan North Road, HaiDian District
Beijing
Beijing, 100191
China
Email: wangxianghua@caict.ac.cn
Xuesen Liu
CAICT
Huayuan North Road, HaiDian District
Beijing
Beijing, 100191
China
Email: liuxuesen@caict.ac.cn
Zhe Li
CAICT
Huayuan North Road, HaiDian District
Beijing
Beijing, 100191
China
Email: lizhe@caict.ac.cn
Mengjia Zhao
CAICT
Huayuan North Road, HaiDian District
Beijing
Beijing, 100191
China
Email: zhao_mengjia@foxmail.com
Wang, et al. Expires 7 January 2027 [Page 21]