Skip to main content

Speculative Decoding Coordination Protocol (SDCP) for Distributed Inference with Multi-Level Caches
draft-li-mlc-inference-acceleration-00

Document Type Active Internet-Draft (individual)
Authors Zhiqiang Li , Zongpeng Du , Junjie Wang , Wei Cheng , Ying Zhao , Chunhao Zhao , Xun Sun
Last updated 2026-11-02
RFC stream (None)
Intended RFC status (None)
Formats
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-li-mlc-inference-acceleration-00
Network Working Group                                              Z. Li
Internet-Draft                                                     Z. Du
Intended status: Standards Track                            China Mobile
Expires: 6 May 2027                                              J. Wang
                                                                W. Cheng
                                                                 Y. Zhao
                                                                  Centec
                                                                 C. Zhao
                                                                    SAIA
                                                                  X. Sun
                                                                   Inesa
                                                         2 November 2026

   Speculative Decoding Coordination Protocol (SDCP) for Distributed
                   Inference with Multi-Level Caches
                 draft-li-mlc-inference-acceleration-00

Abstract

   Speculative decoding accelerates language model inference: a small
   draft model proposes several tokens and the large target model
   verifies them in one forward pass.  In large deployments, draft
   models and the target model run on separate, independently scaled
   instances that exchange state through a multi-level cache.  This
   document specifies the Speculative Decoding Coordination Protocol
   (SDCP), with which a router assigns requests to draft instances by
   load, draft instances submit proposals that reference draft state in
   the cache, and a verifier, the only party that commits output, checks
   proposals under a registered acceptance rule, returns the accepted
   tokens and the next-round speculation limit, and escalates
   persistently rejected sessions to a reduced speculation length,
   another draft model, or decoding by the target model itself depending
   on its load.  SDCP defines a session contract, a committed-prefix
   digest chain, CBOR messages carried over HTTP, idempotent rounds, and
   epoch fencing for recovery.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

Li, et al.                 Expires 6 May 2027                   [Page 1]
Internet-Draft                    SDCP                     November 2026

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 6 May 2027.

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Background  . . . . . . . . . . . . . . . . . . . . . . .   3
     1.2.  Problem Statement . . . . . . . . . . . . . . . . . . . .   3
     1.3.  Protocol Overview . . . . . . . . . . . . . . . . . . . .   4
     1.4.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . .   5
   2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   5
   3.  Architecture  . . . . . . . . . . . . . . . . . . . . . . . .   6
   4.  Session Model . . . . . . . . . . . . . . . . . . . . . . . .   6
     4.1.  Session Contract  . . . . . . . . . . . . . . . . . . . .   6
     4.2.  Committed-Prefix Digest Chain . . . . . . . . . . . . . .   8
   5.  Messages and Resources  . . . . . . . . . . . . . . . . . . .   8
     5.1.  HTTP Usage  . . . . . . . . . . . . . . . . . . . . . . .   8
     5.2.  Message Definitions . . . . . . . . . . . . . . . . . . .   9
   6.  Procedures  . . . . . . . . . . . . . . . . . . . . . . . . .  11
     6.1.  Session Creation and Assignment . . . . . . . . . . . . .  11
     6.2.  Drafting and Proposal . . . . . . . . . . . . . . . . . .  11
     6.3.  Verification and Commit . . . . . . . . . . . . . . . . .  12
     6.4.  Acceptance Rules  . . . . . . . . . . . . . . . . . . . .  13
     6.5.  Speculation Control and Escalation  . . . . . . . . . . .  14
       6.5.1.  Speculation Limit . . . . . . . . . . . . . . . . . .  14
       6.5.2.  Low Rounds and Escalation . . . . . . . . . . . . . .  14
     6.6.  Sampling Policy Updates . . . . . . . . . . . . . . . . .  15
     6.7.  Output Delivery . . . . . . . . . . . . . . . . . . . . .  15
     6.8.  Commit Log  . . . . . . . . . . . . . . . . . . . . . . .  16
     6.9.  Failure Handling and Recovery . . . . . . . . . . . . . .  16

Li, et al.                 Expires 6 May 2027                   [Page 2]
Internet-Draft                    SDCP                     November 2026

   7.  Error Handling  . . . . . . . . . . . . . . . . . . . . . . .  16
   8.  Operational Considerations  . . . . . . . . . . . . . . . . .  17
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .  18
   10. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  18
     10.1.  Media Type . . . . . . . . . . . . . . . . . . . . . . .  18
     10.2.  SDCP Registries  . . . . . . . . . . . . . . . . . . . .  19
       10.2.1.  SDCP Problem Types . . . . . . . . . . . . . . . . .  19
       10.2.2.  SDCP Acceptance Rules  . . . . . . . . . . . . . . .  19
       10.2.3.  SDCP Actions . . . . . . . . . . . . . . . . . . . .  19
   11. References  . . . . . . . . . . . . . . . . . . . . . . . . .  19
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  19
     11.2.  Informative References . . . . . . . . . . . . . . . . .  21
   Appendix A.  Test Vectors . . . . . . . . . . . . . . . . . . . .  21
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  22
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  22

1.  Introduction

1.1.  Background

   Autoregressive decoding produces one token per forward pass of the
   target model.  Each pass reads all model weights and the whole KV
   cache, so decoding is limited by memory bandwidth and leaves compute
   idle.  Speculative decoding [LEVIATHAN] [CHEN] lets a cheaper draft
   model propose M tokens, which the target model then scores in a
   single pass.  The longest acceptable prefix of the proposal is kept,
   and the target model contributes one more token, so each target pass
   yields between 1 and M+1 tokens.  With the speculative sampling rule,
   the output follows exactly the distribution of the target model.
   Some models are trained with built-in multi-token prediction heads
   that serve as drafters [DEEPSEEKV3].

1.2.  Problem Statement

   When drafting and verification run in one process, the coordination
   is internal.  Large deployments separate them: several draft model
   instances, possibly of different sizes, serve many concurrent
   requests and feed one or more target-model verifier instances, and
   both exchange token state and KV caches through a multi-level cache
   spanning accelerator memory, host memory, and networked storage
   [I-D.li-kvcache-efficient-storage].  In that setting the following
   must be agreed between independently implemented components:

   *  how requests are assigned to draft instances according to their
      load, so that the requests of one batch can use different
      drafters;

Li, et al.                 Expires 6 May 2027                   [Page 3]
Internet-Draft                    SDCP                     November 2026

   *  how many tokens a drafter proposes, given its history of accepted
      proposals, its load, and the bandwidth available to the cache;

   *  how a proposal refers to draft state in the cache instead of
      carrying it, so that the verifier can assemble large verification
      batches from many drafters;

   *  which acceptance rule the verifier applies, and how the rule can
      follow user preferences;

   *  what happens when proposals keep being rejected: reduce the
      speculation length, switch to another drafter, let the target
      model decode by itself if it has spare capacity, or fail; and

   *  how one authoritative output is maintained across retries,
      concurrent proposals, and instance failures.

1.3.  Protocol Overview

   SDCP addresses these needs with the following elements:

   *  an immutable Session Contract that fixes the models, the
      acceptance rule, limits, and the escalation policy (Section 4.1);

   *  a committed-prefix digest chain that every proposal must extend,
      which makes stale or forked proposals detectable (Section 4.2);

   *  assignment, proposal, verdict, and output messages in CBOR
      [RFC8949], described in CDDL [RFC8610] and exchanged over HTTP
      (Section 5);

   *  a verification procedure with registered acceptance rules
      (Section 6.3); and

   *  speculation-length control and load-aware escalation
      (Section 6.5).

   The verifier is the sole commit authority.  Drafters, routers, and
   caches can fail or misbehave without changing the committed output,
   except as discussed in Section 9.

Li, et al.                 Expires 6 May 2027                   [Page 4]
Internet-Draft                    SDCP                     November 2026

1.4.  Scope

   SDCP does not define model architectures, sampling kernels, batching
   algorithms, or client-facing inference APIs.  It assumes a multi-
   level cache that stores and serves opaque objects named by URI and
   supports leases, such as KVSP [I-D.li-kvcache-efficient-storage].
   Draft-model training, tree-structured speculation, and multiple
   verifiers per session are out of scope for this version.

2.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   Target model:  The model whose output distribution the service
      promises.

   Draft model:  A cheaper model, or auxiliary prediction head, that
      proposes tokens.

   Router:  The component that admits requests, creates sessions, and
      assigns them to drafters.

   Drafter:  An instance that runs a draft model.

   Verifier:  An instance that runs the target model and commits output.

   Multi-Level Cache (MLC):  Storage for prompts, KV caches, and draft
      artifacts across memory and storage tiers, reachable by all
      instances.

   Session:  One generation task, identified by a 16-octet identifier.

   Committed prefix:  The prompt followed by all output tokens committed
      so far; its length in output tokens is n.

   Round:  One proposal and its verdict.  Rounds of a session are
      numbered from 1.

   Speculation length (M):  The number of tokens in a proposal.

   Acceptance rule:  The algorithm that decides how many proposed tokens
      are kept (Section 6.4).

   Epoch:  A counter that identifies the current verifier authority of a

Li, et al.                 Expires 6 May 2027                   [Page 5]
Internet-Draft                    SDCP                     November 2026

      session (Section 6.9).

3.  Architecture

               requests                        output tokens
     Client -----------> +--------+          +-------------> Client
                         | Router |          |
                         +--------+          |
           assignment  /    |     \  events  |
                      v     |      \         |
             +-----------+  |  +-----------+-+---+
             | Drafter 1 |  |  |    Verifier     |
             +-----------+  |  |  (target model) |
             +-----------+  |  +--------+--------+
             | Drafter 2 |--+--proposal->|verdict
             +-----+-----+     <---------+
                   |                     |
                   v  draft KV, q rows   v  target KV, commits
             +-------------------------------------+
             |          Multi-Level Cache          |
             +-------------------------------------+

                            Figure 1: SDCP Roles

   The router creates a session at a verifier, which computes or loads
   the KV cache of the prompt, and assigns the session to a drafter.
   The drafter proposes tokens to the verifier.  The verifier answers
   each proposal with a verdict that both commits output and steers the
   next round.  Draft state that is large, such as the draft model's KV
   cache and probability rows, stays in the MLC and is referenced by
   URI.

4.  Session Model

4.1.  Session Contract

   The Session Contract is fixed when the session is created:

Li, et al.                 Expires 6 May 2027                   [Page 6]
Internet-Draft                    SDCP                     November 2026

   contract = {
     target:      hash32,          ; SHA-256 of target model weights
     tokenizer:   hash32,          ; SHA-256 of tokenizer definition
     ? drafters:  [+ hash32],      ; permitted draft models
     rule:        uint,            ; acceptance rule (registry), fixed
     ? sampling:  sampling,
     m_max:       uint .le 64,     ; upper bound on M
     max_new:     uint,            ; maximum output tokens
     stop:        [* uint],        ; stop tokens
     low_rounds:  uint .ge 1,      ; R, escalation period
     ? low_ratio: uint .le 100,    ; percent, default 0
     fallback:    [+ ladder],      ; escalation ladder
     ? self_load: uint .le 100,    ; max verifier load for self-decode
     ? self_max:  uint,            ; max tokens per self-decode period
     ? scope:     bstr .size 16    ; KVSP Scope ID of cache objects
   }
   sampling = {
     ? temperature_milli: uint,    ; temperature x 1000
     ? top_k:             uint,
     ? top_p_milli:       uint .le 1000,
     ? seed:              uint
   }
   hash32 = bstr .size 32
   action = a-continue / a-reduce / a-switch /
            a-self-decode / a-finish / a-fail
   ladder = a-reduce / a-switch / a-self-decode / a-fail
   a-continue    = 0
   a-reduce      = 1
   a-switch      = 2
   a-self-decode = 3
   a-finish      = 4
   a-fail        = 5

   The contract digest is the SHA-256 of the contract encoded with the
   core deterministic encoding of Section 4.2.1 of [RFC8949], prefixed
   with a domain separator:

      contract_digest = SHA-256( "SDCP-CONTRACT-1" || 0x00 ||
                                 deterministic_cbor(contract) )

   The sampling profile defines the target distribution p used by the
   acceptance rule: p is the target model's next-token distribution
   after applying temperature, top-k, and top-p in that order; a
   temperature of zero selects the most probable token (ties broken in
   favor of the smallest token identifier).  An absent "drafters" member
   permits any draft model that the router's local policy allows.  The
   router can change the sampling profile at a round boundary
   (Section 6.6); such updates are carried separately from the contract,

Li, et al.                 Expires 6 May 2027                   [Page 7]
Internet-Draft                    SDCP                     November 2026

   which, including the acceptance rule, is never modified.  The
   contract digest is computed once, over the contract as created.  A
   contract that contains a key not defined by this document MUST be
   rejected with problem type "sdcp-bad-contract", so that all parties
   compute the contract digest over the same content.

4.2.  Committed-Prefix Digest Chain

   The verifier maintains a digest over the committed prefix.  For a
   prompt of P tokens t[0..P-1] and committed output tokens o[1..n]:

      d[0] = SHA-256( "SDCP-PREFIX-1" || 0x00 ||
                      contract_digest || uint32(P) ||
                      uint32(t[0]) || ... || uint32(t[P-1]) )
      d[k] = SHA-256( d[k-1] || uint32(o[k]) )   for k = 1..n

   uint32() is four octets in network byte order and SHA-256 is defined
   in [RFC6234].  A proposal states the length n and digest d[n] of the
   prefix it extends.  Because d[n] depends on every committed token, a
   proposal built on a different or outdated prefix is detected without
   comparing token lists.

5.  Messages and Resources

5.1.  HTTP Usage

   SDCP is an HTTP application following BCP 56 [RFC9205].  Each role
   has a base URI configured by the operator; SDCP defines no well-known
   URI.  HTTP/2 [RFC9113] or HTTP/3 [RFC9114] over TLS 1.3 [RFC8446]
   with mutual certificate authentication is REQUIRED.  Message bodies
   use the media type "application/sdcp+cbor".  Errors use Problem
   Details [RFC9457]; clients handle unlisted status codes by class
   [RFC9110].

Li, et al.                 Expires 6 May 2027                   [Page 8]
Internet-Draft                    SDCP                     November 2026

    +========+=============================+============+=============+
    |Role    | Method and Path             |Body        |Response     |
    +========+=============================+============+=============+
    |Verifier| POST sessions               |session-open|session-state|
    +--------+-----------------------------+------------+-------------+
    |Verifier| PUT sessions/{id}/drafter   |assignment  |session-state|
    +--------+-----------------------------+------------+-------------+
    |Verifier| POST proposals              |proposal    |verdict      |
    +--------+-----------------------------+------------+-------------+
    |Verifier| GET                         |none        |output       |
    |        | sessions/{id}/output?from=n |            |             |
    +--------+-----------------------------+------------+-------------+
    |Verifier| POST sessions/{id}/policy   |policy      |session-state|
    +--------+-----------------------------+------------+-------------+
    |Drafter | POST assignments            |assignment  |none (204)   |
    +--------+-----------------------------+------------+-------------+
    |Router  | PUT instances/{id}/status   |status      |none (204)   |
    +--------+-----------------------------+------------+-------------+
    |Router  | POST events                 |event       |none (204)   |
    +--------+-----------------------------+------------+-------------+

                          Table 1: SDCP Resources

   {id} in session paths is the base64url encoding without padding
   [RFC4648] of the session identifier.

5.2.  Message Definitions

   session-id = bstr .size 16
   cache-ref  = tstr                       ; URI of an MLC object

   session-open = {
     session: session-id, epoch: uint, contract: contract,
     prompt: [* uint] / cache-ref,         ; tokens or MLC reference
     ? prompt_kv: cache-ref,               ; target KV of the prompt
     commit_log: cache-ref,                ; durable commit records
     ? resume: {                           ; present on failover
       round: uint, committed: uint, digest: hash32,
       committed_tokens: cache-ref,
       ? target_kv: cache-ref
     }
   }
   session-state = {
     session: session-id, epoch: uint, round: uint,
     committed: uint, digest: hash32, ? state: uint
   }
   assignment = {
     session: session-id, epoch: uint, contract: contract,

Li, et al.                 Expires 6 May 2027                   [Page 9]
Internet-Draft                    SDCP                     November 2026

     verifier: tstr,                       ; verifier base URI
     drafter: tstr,                        ; drafter base URI
     prompt: [* uint] / cache-ref,
     round: uint,                          ; last round with a verdict
     ? sampling: sampling,                 ; current profile, if updated
     committed: uint, digest: hash32,
     ? committed_tokens: cache-ref,        ; committed output so far
     m_init: uint
   }
   proposal = {
     session: session-id, epoch: uint, round: uint,
     drafter: tstr,                        ; drafter base URI
     base: uint, base_digest: hash32,
     tokens: [+ uint],                     ; M proposed tokens
     ? q_rows: cache-ref,                  ; draft rows, for rule 2
     ? draft_kv: cache-ref,
     status: status
   }
   verdict = {
     session: session-id, epoch: uint, round: uint,
     accepted: uint,                       ; a, 0 <= a <= M
     emitted: [* uint],                    ; tokens committed this round
     committed: uint, digest: hash32,      ; new n and d[n]
     action: action, m_cap: uint,
     low: uint,                            ; consecutive low rounds
     load: uint .le 100                    ; verifier load, percent
   }
   output = {
     session: session-id, epoch: uint, from: uint, tokens: [* uint],
     final: bool, ? finish: action
   }
   status = {
     instance: tstr, role: 1 / 2,          ; 1 drafter, 2 verifier
     load: uint .le 100, queue: uint,
     ? accept_milli: uint .le 1000,        ; EWMA of a/M x 1000
     ? cache_mbps: uint,                   ; available MLC bandwidth
     ttl_ms: uint
   }
   event = {
     session: session-id, epoch: uint,
     kind: action, committed: uint, digest: hash32
   }
   policy = { session: session-id, from_round: uint,
              sampling: sampling }

Li, et al.                 Expires 6 May 2027                  [Page 10]
Internet-Draft                    SDCP                     November 2026

   The contract, sampling, hash32, and action types are defined in
   Section 4.1.  Unknown map keys MUST be ignored, except in the
   contract (Section 4.1).  A receiver that encounters an unknown action
   value MUST treat it as "fail".

   The "q_rows" object holds, for each proposed position, the draft
   distribution from which the proposed token was sampled, as a row of V
   IEEE 754 binary32 values in little-endian order, where V is the
   vocabulary size; rows are concatenated in position order.  The
   drafter MUST sample each proposed token from exactly the row it
   publishes.

6.  Procedures

6.1.  Session Creation and Assignment

   The router selects a verifier and sends session-open with epoch 1 and
   a commit_log reference in the MLC.  The verifier validates the
   contract (it MUST refuse a contract whose target digest does not
   match its model), computes or loads the KV cache of the prompt,
   computes d[0], and answers with session-state (round 0, committed 0).

   The router then selects a drafter whose model is permitted by the
   contract.  It first informs the verifier with PUT
   sessions/{id}/drafter, and then sends the assignment, carrying the
   current round, committed length, and digest, to the drafter.  The
   verifier accepts proposals only from the drafter named in the latest
   assignment it has received.  Drafter selection is a local policy
   based on the drafters' status reports, for example preferring
   drafters with low load and queue and high recent acceptance.  The
   router MUST NOT assign sessions to an instance whose status has
   expired.  Each instance reports status at intervals shorter than its
   ttl_ms.

6.2.  Drafting and Proposal

   A drafter holding an assignment prepares its own KV cache for the
   committed prefix (by computation or by loading from the MLC) and
   generates M tokens, with 1 <= M <= min(m_max, m_cap), where m_cap is
   the limit from the last verdict (m_init before the first).  It writes
   the draft artifacts the verifier may need to the MLC and holds leases
   on them until it receives the verdict of the round.  It then sends a
   proposal with base and base_digest equal to the committed length and
   digest it last learned, and round equal to the last verdict's round
   plus one.

Li, et al.                 Expires 6 May 2027                  [Page 11]
Internet-Draft                    SDCP                     November 2026

   A drafter MUST NOT have more than one outstanding proposal per
   session.  It MAY prepare the next proposal speculatively, assuming
   full acceptance, but sends it only after the verdict confirms that
   assumption.

   Choice of M is local to the drafter.  The following rule is
   RECOMMENDED.  Let alpha be the drafter's exponentially weighted
   estimate of the per-token acceptance probability for the session, t_d
   the time to draft one token (including writing its artifacts at the
   cache bandwidth currently available), and t_v the expected
   verification time.  The expected number of tokens committed per round
   is (1 - alpha^(M+1)) / (1 - alpha) for alpha < 1, and M + 1 for alpha
   = 1 [LEVIATHAN]; the drafter chooses the M within its limits that
   maximizes that quantity divided by (M * t_d + t_v).  A loaded drafter
   thereby proposes fewer tokens, and a drafter whose cache bandwidth is
   scarce does the same.

6.3.  Verification and Commit

   On receiving a proposal, the verifier performs these steps:

   1.  Authorization: the authenticated client MUST be the drafter named
       in the latest assignment the verifier received for the session;
       otherwise respond 403 with problem type "sdcp-not-assigned".

   2.  Fencing: if epoch differs from the session epoch, respond 409
       with problem type "sdcp-stale-epoch".

   3.  Idempotency: if a verdict for this round already exists, return
       it unchanged; if the round is being verified for an identical
       earlier request, wait for that verdict and return it.

   4.  Base check: if base is not the committed length or base_digest is
       not d[base], respond 409 with problem type "sdcp-stale-base" and
       the current session-state, so that the drafter can resynchronize.
       If round is not the last round plus one, respond 409 likewise.

   5.  Preconditions of the rule: if the acceptance rule needs draft
       probabilities, the verifier MUST have obtained the rows for every
       proposed position, and checked that q[i](x[i]) is not zero for
       every position, before drawing any random value.  If it cannot
       obtain them, it rejects the round with problem type "sdcp-
       q-unavailable" and commits nothing.  Obtaining them first is
       necessary because abandoning a round only after a rejection was
       observed would bias the output distribution.  The verifier MAY
       fetch the rows concurrently with scoring.

Li, et al.                 Expires 6 May 2027                  [Page 12]
Internet-Draft                    SDCP                     November 2026

   6.  Scoring: the verifier computes the target distributions p[1..M+1]
       for the positions following the committed prefix, normally by a
       single forward pass over the proposed tokens using the target KV
       cache of the committed prefix.  It MAY batch proposals of many
       sessions into one pass.

   7.  Acceptance: it applies the contract's acceptance rule
       (Section 6.4), yielding a accepted tokens and one additional
       token.  It truncates the emitted tokens after the first stop
       token and at max_new.

   8.  Commit: it appends the emitted tokens to the committed prefix,
       updates n and d[n], and appends a commit record (epoch, round,
       emitted tokens, n, and d[n]) to the session's commit_log
       (Section 6.8); only after that append succeeds does it send the
       verdict and make the tokens available as output.  If the append
       fails because the log has been fenced, the verifier MUST discard
       the round, release no output, and stop serving the session.  The
       commit and the recording of the verdict MUST be atomic with
       respect to concurrent proposals for the session.

   9.  Control: it determines action and m_cap as specified in
       Section 6.5.

   After the verdict, the drafter MUST discard its draft state for
   proposed positions beyond the accepted ones and MAY release its
   leases.  Its KV cache for the accepted tokens remains valid.  The
   verifier MAY publish the target KV cache of committed tokens to the
   MLC so that another verifier can take over the session.

6.4.  Acceptance Rules

   Rules are registered (Section 10.2.2).  This document defines two.
   In both, p[i] is the target distribution at proposed position i, x[i]
   the proposed token, and positions are evaluated in order.

   Rule 1, greedy:
      x[i] is accepted if it equals argmax p[i], where ties are broken
      in favor of the smallest token identifier.  Evaluation stops at
      the first position that is not accepted.  The additional token is
      argmax p[a+1].  The output is identical to greedy decoding of the
      target model as computed by the verifier's scoring passes,
      whatever the drafter proposes.

   Rule 2, speculative sampling:
      For each position, draw r uniformly from [0, 1).  Accept x[i] if r
      < min(1, p[i](x[i]) / q[i](x[i])), where q[i] is the row of q_rows
      for position i, that is, the draft distribution from which x[i]

Li, et al.                 Expires 6 May 2027                  [Page 13]
Internet-Draft                    SDCP                     November 2026

      was sampled.  At the first rejected position, the additional token
      is drawn from the normalized residual distribution max(0, p[i] -
      q[i]).  If all M tokens are accepted, the additional token is
      drawn from p[M+1].  With exact q values, the output has the
      distribution of the target model [LEVIATHAN] [CHEN].  The proposal
      MUST carry q_rows; a proposal for which q[i](x[i]) is zero is
      rejected, before any random draw, with problem type "sdcp-bad-
      proposal".

      The verifier's random values MUST be independent of those the
      drafter used.  When the sampling profile contains a seed, the
      verifier derives each value from HMAC-SHA-256 [RFC2104] keyed with
      the seed (as eight octets in network byte order) over "SDCP-
      VERIFY-1" || 0x00 || session || uint32(round) || uint32(i) ||
      uint8(purpose), where purpose is 0 for the acceptance test and 1
      for drawing the additional token, taking the first 53 bits of the
      result divided by 2^53.  Drafters MUST NOT use this derivation for
      their own sampling.  Without a seed, the verifier uses a
      cryptographically secure random number generator.

6.5.  Speculation Control and Escalation

6.5.1.  Speculation Limit

   Each verdict carries m_cap, the largest M the drafter may use in the
   next round, with 1 <= m_cap <= m_max.  The verifier lowers m_cap when
   its own load makes long proposals costly, and as specified for the
   "reduce" action below.  It SHOULD restore m_cap gradually (for
   example by one per round) when rounds are no longer low and its load
   permits.

6.5.2.  Low Rounds and Escalation

   A round is low if 100 * a < low_ratio * M, or, when low_ratio is zero
   or absent, if a = 0.  The verifier keeps three counters per session,
   all starting at zero: c, the number of consecutive low rounds; g, the
   number of consecutive rounds that are not low; and j, the position in
   the contract's fallback list.  After a low round, c is incremented
   and g reset; after a round that is not low, c is reset and g
   incremented, and when g reaches R, where R is low_rounds, j is reset
   to zero.  When c reaches R, the verifier applies entry j of the
   fallback list (skipping entries that are not permitted, as described
   for self_decode, and advancing j past every entry it applies or
   skips), then resets c to zero.  If no entry remains, it applies
   "fail".  Otherwise it applies "continue".  The fallback list MUST
   contain only the actions reduce, switch, self_decode, and fail; j
   persists across a change of drafter and across a self-decoding
   period.  The actions are:

Li, et al.                 Expires 6 May 2027                  [Page 14]
Internet-Draft                    SDCP                     November 2026

   continue (0):  The drafter proceeds with the next round.

   reduce (1):  The verifier sets m_cap to max(1, floor(m_cap / 2)) and
      the drafter proceeds.

   switch (2):  The verifier sends an event to the router, which assigns
      the session to another drafter with a permitted model; the current
      drafter MUST stop proposing for the session.

   self_decode (3):  The verifier decodes the session itself with the
      target model.  This action is permitted only if the contract
      contains self_load and the verifier's current load does not exceed
      it; otherwise the verifier skips it and applies the next entry of
      the fallback list.  Each token committed during self-decoding is
      recorded as a round of its own, with no proposal, so round numbers
      stay consecutive.  During self-decoding the verifier commits
      tokens one at a time, publishes them as output, and stops after
      self_max tokens (if present) or when its load exceeds self_load;
      it then sends an event so that the router assigns a drafter again.

   finish (4):  Generation has ended because of a stop token or max_new.

   fail (5):  The verifier ends the session with an error.

   Because self_decode is conditional on the verifier's load, the same
   fallback list leads to self-decoding on a lightly loaded verifier and
   to the next entry, such as switch, on a busy one.

6.6.  Sampling Policy Updates

   The router can change the sampling profile of a session, for example
   to follow a preference stated by the user.  It sends a policy message
   with from_round greater than the current round.  The verifier applies
   the new profile to rounds numbered from_round and later, and the
   router includes it in the "sampling" member of subsequent
   assignments; the contract in those assignments is unchanged.  The
   acceptance rule cannot be changed.  Committed output is never changed
   by a policy update.

6.7.  Output Delivery

   Committed tokens are delivered with GET output?from=n, which returns
   the tokens committed at output indices n and later, waiting up to a
   server-chosen time if none are available.  Only tokens whose commit
   record has been written are returned.  Indices are stable and every
   output message carries the session epoch, so a front end that retries
   never duplicates or loses tokens, and a front end that receives
   output with a lower epoch than one it has seen discards it.  The

Li, et al.                 Expires 6 May 2027                  [Page 15]
Internet-Draft                    SDCP                     November 2026

   final output message has final set to true and finish set to "finish"
   or "fail".

6.8.  Commit Log

   The commit_log is an MLC object to which records are appended with a
   conditional operation, for example an HTTP PUT with If-Match on the
   current entity tag [RFC9110], so that an append succeeds only if no
   other append has happened since the writer last read the log.  Each
   record carries the epoch of its writer.  A commit record appended by
   a verifier MUST carry the session's current epoch and follow the last
   record it wrote.  A fence record carries only an epoch; after a fence
   record with epoch E+1, appends carrying an epoch lower than E+1 MUST
   fail.  If the MLC cannot enforce this rule, the verifier MUST read
   the log after its conditional append and treat its append as failed
   if a fence record with a higher epoch precedes its own record.

6.9.  Failure Handling and Recovery

   *  Drafter failure: when a drafter's status expires, the router
      reassigns its sessions.  The new drafter starts from the committed
      prefix in the assignment.  Proposals from the failed drafter are
      rejected because it is no longer the assigned drafter.

   *  Verifier failure: the router first appends a fence record with the
      new epoch to the commit_log, so that the old verifier, even if it
      is only partitioned, can no longer commit or release output; it
      then reads the last commit record and opens the session at a new
      verifier with that epoch and the "resume" member carrying that
      round, committed length, digest, and the target KV cache from the
      MLC if available.  It then informs the verifier of the drafter and
      sends a new assignment.  Proposals carrying an older epoch are
      rejected, and front ends discard output carrying an older epoch,
      so a recovered old verifier cannot extend the session.

   *  Lost messages: proposals and their verdicts are idempotent per
      round, so a drafter retries a proposal whose verdict it did not
      receive.

7.  Error Handling

      +=====================+========+==============================+
      | Type suffix         | Status | Meaning                      |
      +=====================+========+==============================+
      | sdcp-stale-epoch    | 409    | Epoch is not current         |
      +---------------------+--------+------------------------------+
      | sdcp-stale-base     | 409    | Proposal does not extend the |
      |                     |        | committed prefix             |

Li, et al.                 Expires 6 May 2027                  [Page 16]
Internet-Draft                    SDCP                     November 2026

      +---------------------+--------+------------------------------+
      | sdcp-q-unavailable  | 409    | Draft probabilities required |
      |                     |        | by the rule are unavailable  |
      +---------------------+--------+------------------------------+
      | sdcp-bad-contract   | 422    | Contract invalid or not      |
      |                     |        | supported                    |
      +---------------------+--------+------------------------------+
      | sdcp-limit          | 422    | Proposal violates m_cap or   |
      |                     |        | m_max                        |
      +---------------------+--------+------------------------------+
      | sdcp-session-closed | 410    | Session has finished or      |
      |                     |        | failed                       |
      +---------------------+--------+------------------------------+
      | sdcp-bad-proposal   | 422    | Proposal inconsistent with   |
      |                     |        | its draft distributions      |
      +---------------------+--------+------------------------------+
      | sdcp-not-assigned   | 403    | Drafter is not assigned to   |
      |                     |        | the session                  |
      +---------------------+--------+------------------------------+

                        Table 2: SDCP Problem Types

   The type URI is "https://iana.org/assignments/sdcp#" followed by the
   suffix.  These types are specific to SDCP and are therefore
   registered in an SDCP registry rather than in the "HTTP Problem
   Types" registry (Section 4.2 of [RFC9457]).  Problem details for
   "sdcp-stale-base" include the member "session_state" carrying the
   current committed length and digest.

8.  Operational Considerations

   *  Batch windows: a verifier that waits to fill a batch adds that
      wait to every round.  Windows of a fraction of the target model's
      forward-pass time are typical.

   *  Cache placement: draft artifacts are short-lived and SHOULD be
      placed in the fastest tier that both drafter and verifier can
      reach.

   *  Monitoring: implementations SHOULD report the mean accepted
      length, the distribution of M, escalation counts by action, stale-
      base rejections, and self-decoded tokens.

   *  Incremental deployment: a verifier can serve sessions without
      drafters by decoding itself, so SDCP can be introduced one drafter
      pool at a time.

Li, et al.                 Expires 6 May 2027                  [Page 17]
Internet-Draft                    SDCP                     November 2026

9.  Security Considerations

   All exchanges use mutual TLS, and every role authorizes requests by
   authenticated identity: only the router opens sessions and assigns
   drafters, and only the assigned drafter submits proposals.

   Integrity of output.  Under the greedy rule, the committed output
   depends only on the target model and the committed prefix; a
   malicious drafter can waste verifier capacity but cannot change the
   output.  Under speculative sampling, the drafter supplies q, and a
   drafter that reports false probabilities can bias the output (for
   example, by reporting a small q for tokens it wants accepted).
   Speculative sampling MUST therefore be used only with drafters that
   are trusted to the same degree as the verifier; with other drafters
   the greedy rule MUST be used.

   Denial of service.  Proposals cost a forward pass.  Verifiers MUST
   enforce m_cap and SHOULD rate-limit rounds per session and per
   drafter.  The escalation ladder also bounds the work a poorly matched
   or hostile drafter can cause.

   Cache objects.  References in proposals point to the MLC; the
   verifier MUST fetch them only from caches it trusts for the session's
   scope, so that a proposal cannot direct the verifier to arbitrary
   URIs.

   Privacy.  Prompts, KV caches, and outputs are user data.  The privacy
   considerations of [I-D.li-kvcache-efficient-storage] apply to all MLC
   objects of a session.

10.  IANA Considerations

10.1.  Media Type

   IANA is requested to register the media type "application/sdcp+cbor"
   in the "Media Types" registry [RFC6838]: Type name: application;
   Subtype name: sdcp+cbor; Required parameters: N/A; Optional
   parameters: N/A; Encoding considerations: binary; Security
   considerations: see Section 9 of RFC XXXX; Interoperability
   considerations: none; Published specification: RFC XXXX; Applications
   that use this media type: distributed inference systems using
   speculative decoding; Fragment identifier considerations: the syntax
   and semantics of fragment identifiers specified for "application/
   cbor" apply; Additional information: Deprecated alias names for this
   type: N/A, Magic number(s): N/A, File extension(s): N/A, Macintosh
   file type code(s): N/A; Person and email address to contact: IETF
   <iesg@ietf.org>; Intended usage: COMMON; Restrictions on usage: none;
   Author: IETF; Change controller: IETF.

Li, et al.                 Expires 6 May 2027                  [Page 18]
Internet-Draft                    SDCP                     November 2026

10.2.  SDCP Registries

   IANA is requested to create a registry group named "Speculative
   Decoding Coordination Protocol (SDCP)" at
   "https://www.iana.org/assignments/sdcp" with the registries below.
   For the numeric registries, values 0-191 are assigned by
   Specification Required [RFC8126] and values 192-255 are reserved for
   Private Use.  The designated experts should check that a new
   acceptance rule states whether it preserves the target distribution
   and what trust it requires in drafters.

10.2.1.  SDCP Problem Types

   Columns: Type Suffix, Title, Recommended HTTP Status Code, Reference.
   The type URI is "https://iana.org/assignments/sdcp#" followed by the
   suffix.  Initial contents: the entries of Table 2, with reference RFC
   XXXX.  Policy: Specification Required.

10.2.2.  SDCP Acceptance Rules

   Initial contents: 0 Reserved; 1 greedy; 2 speculative-sampling; each
   with reference RFC XXXX.  A registration MUST specify the acceptance
   test, the additional token, and the information a proposal must
   carry.

10.2.3.  SDCP Actions

   Initial contents: 0 continue; 1 reduce; 2 switch; 3 self_decode; 4
   finish; 5 fail.

11.  References

11.1.  Normative References

   [RFC2104]  Krawczyk, H., Bellare, M., and R. Canetti, "HMAC: Keyed-
              Hashing for Message Authentication", RFC 2104,
              DOI 10.17487/RFC2104, February 1997,
              <https://www.rfc-editor.org/info/rfc2104>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4648]  Josefsson, S., "The Base16, Base32, and Base64 Data
              Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006,
              <https://www.rfc-editor.org/info/rfc4648>.

Li, et al.                 Expires 6 May 2027                  [Page 19]
Internet-Draft                    SDCP                     November 2026

   [RFC6234]  Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms
              (SHA and SHA-based HMAC and HKDF)", RFC 6234,
              DOI 10.17487/RFC6234, May 2011,
              <https://www.rfc-editor.org/info/rfc6234>.

   [RFC6838]  Freed, N., Klensin, J., and T. Hansen, "Media Type
              Specifications and Registration Procedures", BCP 13,
              RFC 6838, DOI 10.17487/RFC6838, January 2013,
              <https://www.rfc-editor.org/info/rfc6838>.

   [RFC8126]  Cotton, M., Leiba, B., and T. Narten, "Guidelines for
              Writing an IANA Considerations Section in RFCs", BCP 26,
              RFC 8126, DOI 10.17487/RFC8126, June 2017,
              <https://www.rfc-editor.org/info/rfc8126>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8446]  Rescorla, E., "The Transport Layer Security (TLS) Protocol
              Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
              <https://www.rfc-editor.org/info/rfc8446>.

   [RFC8610]  Birkholz, H., Vigano, C., and C. Bormann, "Concise Data
              Definition Language (CDDL): A Notational Convention to
              Express Concise Binary Object Representation (CBOR) and
              JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610,
              June 2019, <https://www.rfc-editor.org/info/rfc8610>.

   [RFC8949]  Bormann, C. and P. Hoffman, "Concise Binary Object
              Representation (CBOR)", STD 94, RFC 8949,
              DOI 10.17487/RFC8949, December 2020,
              <https://www.rfc-editor.org/info/rfc8949>.

   [RFC9110]  Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke,
              Ed., "HTTP Semantics", STD 97, RFC 9110,
              DOI 10.17487/RFC9110, June 2022,
              <https://www.rfc-editor.org/info/rfc9110>.

   [RFC9113]  Thomson, M., Ed. and C. Benfield, Ed., "HTTP/2", RFC 9113,
              DOI 10.17487/RFC9113, June 2022,
              <https://www.rfc-editor.org/info/rfc9113>.

   [RFC9114]  Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114,
              June 2022, <https://www.rfc-editor.org/info/rfc9114>.

Li, et al.                 Expires 6 May 2027                  [Page 20]
Internet-Draft                    SDCP                     November 2026

   [RFC9205]  Nottingham, M., "Building Protocols with HTTP", BCP 56,
              RFC 9205, DOI 10.17487/RFC9205, June 2022,
              <https://www.rfc-editor.org/info/rfc9205>.

   [RFC9457]  Nottingham, M., Wilde, E., and S. Dalal, "Problem Details
              for HTTP APIs", RFC 9457, DOI 10.17487/RFC9457, July 2023,
              <https://www.rfc-editor.org/info/rfc9457>.

11.2.  Informative References

   [CHEN]     Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre,
              L., and J. Jumper, "Accelerating Large Language Model
              Decoding with Speculative Sampling", arXiv:2302.01318,
              February 2023, <https://arxiv.org/abs/2302.01318>.

   [DEEPSEEKV3]
              DeepSeek-AI, "DeepSeek-V3 Technical Report",
              arXiv:2412.19437, December 2024,
              <https://arxiv.org/abs/2412.19437>.

   [I-D.li-kvcache-efficient-storage]
              Li, Z., Du, Z., Wang, J., Cheng, W., Zhao, Y., Zhao, C.,
              and X. Sun, "Key-Value (KV) Cache Storage Protocol (KVSP):
              Indexing, Placement, Prefetch, and Migration for
              Distributed Inference", Work in Progress, Internet-Draft,
              draft-li-kvcache-efficient-storage-00, 2 November 2026,
              <https://datatracker.ietf.org/doc/html/draft-li-kvcache-
              efficient-storage-00>.

   [LEVIATHAN]
              Leviathan, Y., Kalman, M., and Y. Matias, "Fast Inference
              from Transformers via Speculative Decoding", Proceedings
              of the 40th International Conference on Machine Learning
              (ICML), PMLR 202, July 2023,
              <https://proceedings.mlr.press/v202/leviathan23a.html>.

Appendix A.  Test Vectors

   A contract with greedy rule, m_max 8, max_new 256, stop token 2,
   low_rounds 3, and fallback ladder [reduce, switch, self_decode,
   fail], with target and tokenizer digests equal to the KVSP test
   vector digests, has the following deterministic CBOR encoding (143
   octets):

Li, et al.                 Expires 6 May 2027                  [Page 21]
Internet-Draft                    SDCP                     November 2026

   a8 64 72 75 6c 65 01 64 73 74 6f 70 81 02 65 6d
   5f 6d 61 78 08 66 74 61 72 67 65 74 58 20 25 4e
   52 8c 66 9a 20 84 c9 72 72 98 ca cb 83 e8 85 4e
   0a 6f f8 0e 2b f1 80 96 b9 dc e0 f7 eb da 67 6d
   61 78 5f 6e 65 77 19 01 00 68 66 61 6c 6c 62 61
   63 6b 84 01 02 03 05 69 74 6f 6b 65 6e 69 7a 65
   72 58 20 9f 42 a7 a2 9d ca 30 63 ad 26 d7 be 7f
   e6 a9 30 fa 38 af b1 a7 03 e4 f7 ff 53 8d b4 e1
   33 b1 9c 6a 6c 6f 77 5f 72 6f 75 6e 64 73 03

   For the prompt [1, 2, 3, 4] and committed output tokens 17 and 42:

   contract_digest = bcae60a92c622b03fa1c229bac9f5654
                     076957e2f1a8bedcb3549ce4a6328e1f
   d[0]            = 186b373698d5baf27716830811ffa641
                     4887dd0a4f9d13db6c00e19bdb57b76e
   d[1]            = 3372e0fd91b77611c3502da5b0a5c8a5
                     1540e055d7abdd0d1bcf91110e142c1c
   d[2]            = 469bbe77ceb5b1540a4d61d826a3117e
                     357e097c5ac20271306633559fccdf1f

   Example round under the greedy rule: committed n = 2; the drafter
   proposes [7, 9, 11] (M = 3); the target argmax tokens at the three
   positions are 7, 9, and 13, and at the fourth position 5.  Then a =
   2, the additional token is 13 (the correction at the first mismatch),
   emitted = [7, 9, 13], and the new committed length is 5.  The round
   is not low, so c is reset and the action is "continue".

Acknowledgments

   The authors thank colleagues working on disaggregated inference
   serving for discussions on separating drafting from verification.

Authors' Addresses

   Zhiqiang Li
   China Mobile
   Beijing
   100053
   China
   Email: lizhiqiangyjy@chinamobile.com

   Zongpeng Du
   China Mobile
   Beijing
   100053
   China

Li, et al.                 Expires 6 May 2027                  [Page 22]
Internet-Draft                    SDCP                     November 2026

   Email: duzongpeng@chinamobile.com

   Junjie Wang
   Centec
   Shanghai
   201203
   China
   Email: wangjj@centec.com

   Wei Cheng
   Centec
   Shanghai
   201203
   China
   Email: chengw@centec.com

   Ying Zhao
   Centec
   Shanghai
   200000
   China
   Email: zhaoying@centec.com

   Chunhao Zhao
   SAIA
   Shanghai
   200125
   China
   Email: chunhao.zhao@sh-aia.com

   Xun Sun
   Inesa
   Shanghai
   200030
   China
   Email: sunxun@inesa.com

Li, et al.                 Expires 6 May 2027                  [Page 23]