Semantic Inference Routing Protocol (SIRP)

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 3 April 2026.¶

Copyright Notice

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

AI inference services are frequently deployed behind gateways, routers, or service meshes that mediate traffic. In many deployments, routing is guided by client-supplied metadata (e.g., headers, query parameters, tags). Such metadata can be manipulated, diverge across providers, or fail to capture the semantic intent of a request.¶

The Semantic Inference Routing Protocol (SIRP) introduces a standardized, model-agnostic, content-driven approach for classification and routing prior to backend invocation. Building upon established semantic routing principles [I-D.FARREL-SEMANTIC-ROUTING], SIRP defines: (1) classification axes and representation, (2) interoperable signaling via standardized header fields (or protocol-native equivalents), and (3) a pluggable pipeline of value-added routing (VAR) modules.¶

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

2. Conventions and Terminology

SIRP:

Semantic Inference Routing Protocol.¶

Content-Level Classification:

Machine learning-based analysis of the request payload (text or multimodal) to yield category, sensitivity, and complexity labels.¶

Semantic Routing:

Routing decisions informed by classification results rather than untrusted metadata alone.¶

Value-Added Routing (VAR):

Optional modules that refine routing along cost, urgency, domain specialization, and privacy dimensions.¶

Routing Decision:

Final selection of backend target and parameterization emitted by the router.¶

3. Problem Statement and Motivation

Conventional inference routing suffers from: (1) manipulable metadata, (2) heterogeneous vendor flags and model parameters, and (3) inefficiency when queries are misrouted to unsuitable backends. By incorporating classification of the actual content into the routing plane, SIRP improves robustness, policy enforcement, and performance portability.¶

4. Requirements

SIRP introduces the following requirements:¶

1. Transparency: Classification outputs MUST be available to downstream components and SHOULD be optionally exposed to clients.¶
2. Security and Integrity: Classifiers MUST detect and mitigate adversarial inputs; logs MUST be protected against leakage.¶
3. Extensibility: The routing pipeline MUST allow composable modules (e.g., category to urgency to privacy).¶
4. Interoperability: SIRP MUST integrate with existing gateway ecosystems (e.g., Envoy External Processing, Kubernetes Gateway API) following HTTP protocol building best practices [RFC9205].¶
5. Efficiency: Classification and routing overhead SHOULD be bounded to preserve latency SLOs.¶
6. Backward Compatibility: Clients lacking SIRP support MUST be served via conservative default routing.¶

5. Protocol Overview

Figure 1 illustrates a canonical SIRP-capable deployment.¶

+--------+    (1) Inference Request    +-----------------+
| Client | --------------------------> | SIRP Router/    |
+--------+                             | Gateway/Proxy   |
                                       +---------+-------+
                                                 |
            (2a) Content classification         |
            (2b) Populate SIRP headers          |
                                                 v
                                       +---------+-------+
                                       | Routing Pipeline|
                                       | Core+VAR Modules|
                                       +---------+-------+
                                                 |
                                    (4) Forward Decision
                                                 |
                                       +---------v-------+
                                       | Backend         |
                                       | Inference Model |
                                       +---------+-------+
                                                 |
+--------+         (5) Response ----------------+
| Client | <-----------------------------------+
+--------+

Routers may additionally maintain semantic caches (e.g., embedding-based or canonicalized text keys) to short-circuit repeated queries. A reference implementation demonstrating these concepts is available in [VLLM-SEMANTIC-ROUTER].¶

6. Message Format and Header Definitions

SIRP defines interoperable message annotations conveyed via HTTP header fields (or semantically equivalent fields in non-HTTP transports) as specified in [RFC9110]. The header field format follows structured field values as defined in [RFC9651] where applicable. Implementations MUST preserve these fields end-to-end within the routing plane. Table 1 lists the base header set.¶

X-SIRP-Decision:

The decision field MUST uniquely identify the chosen backend target and parameterization. It MAY be encoded as an opaque token, JSON object, or signed structure (e.g., JWS) when tamper-evidence is needed.¶

Extensibility:

Additional fields MAY be defined under the X-SIRP- namespace. New fields SHOULD be registered per Section 12.¶

7. Routing Logic and Decision Flow

SIRP decomposes routing into ordered modules, similar to service function chaining architectures [RFC7665] but applied to AI inference services. A reference flow is shown in Figure 2.¶

+-------+   +----------+   +---------+   +-----+
| Idle  |-->| Classify |-->|CoreRoute|-->| VAR |
+-------+   +----------+   +---------+   +-----+
                           |               |
                           v               v
                        [candidates]  [refinements]
                             \           /
                              \         /
                               +-> EmitDecision -> Forward

Classification Module:

Input: request content. Output: X-SIRP-Category, X-SIRP-Sensitivity, X-SIRP-Complexity.¶

Core Routing Module:

Select candidate backends and default parameter templates.¶

VAR Pipeline:

Optional modules refine or override the decision (cost, urgency, specialization, privacy).¶

Emit Decision:

Produce X-SIRP-Decision and forward the request.¶

8. Value-Added Routing (VAR) Modules

VAR modules are OPTIONAL but RECOMMENDED for advanced behavior. Similar to how Network Service Headers [RFC8300] enable service function chaining with metadata, VAR modules use classification metadata to enhance routing decisions:¶

Cost Optimization:

When classification confidence is high and complexity is low, the router SHOULD prefer lower-cost models; otherwise it SHOULD escalate.¶

Urgency Prioritization:

For time-critical requests, the router MAY favor low-latency backends, potentially at higher cost.¶

Domain Specialization:

Category-specific backends (e.g., math, code, biomedical) SHOULD be preferred when available.¶

Privacy-Aware Handling:

For medium/high sensitivity, the router MUST enforce stricter controls (e.g., sandboxed clusters, masking, or blocking).¶

9. Examples and Use Cases

This section presents detailed examples demonstrating SIRP's classification and routing behavior across various scenarios.¶

10. Experimental and Evaluation Methodology

Implementers SHOULD evaluate SIRP using public QA/reasoning datasets (e.g., MMLU, ARC, TruthfulQA, GPQA, HellaSwag, CommonsenseQA), including:¶

• Comparisons: metadata-only routing vs. SIRP-enabled routing.¶
• Ablations: disabling individual VAR modules.¶
• OOD/Adversarial: robustness to jailbreaks and unseen domains.¶
• Metrics: accuracy, latency, cost reduction, compliance/SLOs.¶

11. Security Considerations

Classification and routing artifacts may contain sensitive content and MUST be access-controlled and logged with least privilege. Models SHOULD be hardened with adversarial examples. Privacy modules MUST comply with applicable regulations. Implementations SHOULD bound classification cost and rate-limit to mitigate denial-of-service.¶

12. IANA Considerations

This document requests creation of a new IANA registry entitled “SIRP Header Fields” within the “Message Headers” category. Initial registrations are:¶

• X-SIRP-Category¶
• X-SIRP-Sensitivity¶
• X-SIRP-Complexity¶
• X-SIRP-Decision¶
• X-SIRP-Policy¶

Future extensions SHOULD follow the "Specification Required" policy as defined in [RFC8126].¶

Authors' Addresses