The Signed-Agent-Metadata HTTP Header Field for Web Bot Auth

1.1. Problem Statement: Shared Fate on Multi-Tenant Platforms

Web Bot Auth ([I-D.ietf-webbotauth-httpsig-protocol]) allows automated HTTP clients to cryptographically sign requests using HTTP Message Signatures ([RFC9421]) and advertise their key discovery URI in the Signature-Agent header field.¶

This model maps cleanly when a single organization operates a single crawler or bot with robust abuse controls. However, cloud browser providers, hosted AI agent platforms, and enterprise gateways run thousands of independent customer workloads behind shared egress infrastructure. Consider a fictional Acme Corp (platform.acme.com), which has pivoted from mail-order industrial products into hosting cloud AI agents. When Acme signs outbound requests with its platform domain key, all traffic from its customers shares one Signature-Agent identity:¶

# NOTE: '\' line wrapping per RFC 8792

Signature-Agent: "https://platform.acme.com/.well-known/\
  http-message-signatures-directory"

This creates a shared-fate problem for both platforms and verifiers (origins, CDNs, and security services). Suppose one tenant runs an aggressive scraping workload against shopping.com, while hundreds of other tenants run benign research and shopping agents on the same platform. The verifier at shopping.com observes malicious traffic authenticated only by platform.acme.com. Without sub-identifiers, the verifier must choose between two poor outcomes:¶

1. Penalize the entire platform: Apply rate limits, challenges, or blocks to all requests from platform.acme.com, which disrupts legitimate tenants sharing the platform.¶
2. Permit platform-level abuse: Allow traffic from platform.acme.com to bypass controls, which leaves shopping.com exposed to the abusive tenant.¶

Platforms and verifiers need a lightweight mechanism to disambiguate sub-identities and convey per-request operational metadata under a shared Signature-Agent key without exposing personally identifiable information (PII) on the wire.¶

1.2. Relationship to Web Bot Auth and Signature Agent Cards

The Web Bot Auth ecosystem separates cryptographic key discovery, static agent description, and dynamic per-request metadata:¶

• Base Protocol ([I-D.ietf-webbotauth-httpsig-protocol]): Defines how an automated client signs HTTP requests and points verifiers to its public keys via Signature-Agent.¶
• Signature Agent Card ([I-D.meunier-webbotauth-registry]): Defines a hosted JSON metadata document where an agent operator advertises static, operator-level properties, including client_name, contacts, and a web_bot_auth object (purpose, trigger, rate-expectation, rfc9309-product-token, and rfc9309-compliance).¶
• Per-Request Metadata (Signed-Agent-Metadata, this document): Defines an in-band HTTP header field that carries dynamic, pseudonymous sub-identifiers (product, project, session) covered by the per-request HTTP Message Signature.¶

Signed-Agent-Metadata (SAM) is additive to the Signature Agent Card. Static properties that describe the agent as a whole (such as purpose or trigger) belong in the hosted Signature Agent Card and are intentionally omitted from Signed-Agent-Metadata to avoid redundancy. Similarly, if an operator already publishes a separate Signature-Agent URI and Signature Agent Card for each product, the operator does not need to include the product member in Signed-Agent-Metadata.¶

1.3. Related Approaches and Use Cases

Several existing or proposed mechanisms address adjacent bot authentication problems, each targeting a different operational model:¶

Signed-Agent-Metadata specifically addresses multi-tenant platforms that sign traffic under a common origin and need low-overhead sub-identity or specific behavior isolation.¶

3.1. Syntax

The Signed-Agent-Metadata HTTP header field is a Dictionary Structured Field as defined in Section 3.2 of [RFC9651]. Its ABNF syntax follows the sf-dictionary production rule:¶

Signed-Agent-Metadata = sf-dictionary

Each dictionary member key is a lowercase token (key in [RFC9651]). Each member value defined in this specification MUST be a String (sf-string in [RFC9651]). Parameters on dictionary members are not defined by this specification and MUST be ignored by verifiers.¶

An example Signed-Agent-Metadata header field with all three members present:¶

Signed-Agent-Metadata: product="Acme-Research", project="prj_89a0f",
  session="sess_88b1"

3.2. Dictionary Members

Sending the Signed-Agent-Metadata header field is OPTIONAL, and all dictionary members within it are OPTIONAL. A platform MAY include any subset of members that fits its architecture and tenancy model to reduce shared-fate penalties across its workloads.¶

Verifiers MUST ignore unrecognized member keys in the Signed-Agent-Metadata dictionary to allow backward-compatible extensions.¶

3.3. Hierarchical Scoping and Chaining

Identifiers carried in Signed-Agent-Metadata are scoped namespaces, not globally unique identifiers. They form a strict hierarchy under the signing Signature-Agent:¶

Signature-Agent  (Tier 0: Platform / Signing Origin)
  +-- product    (Tier 1: Application / Product)
        +-- project  (Tier 2: Tenant / Workspace)
              +-- session  (Tier 3: Task / Session)

When evaluating Signed-Agent-Metadata, verifiers MUST apply the following scoping rules:¶

1. Chained Tuple Evaluation: A narrower identifier MUST always be evaluated within the namespace of the verified Signature-Agent and all broader Signed-Agent-Metadata identifier members present in the request. For example, session="sess_1" under project="prj_a" and session="sess_1" under project="prj_b" MUST be treated as distinct sessions.¶
2. Omitted Intermediate Tiers: If a platform omits an intermediate member (for example, sending product and session without project), the scoping chain consists of the present members in their defined hierarchical order: (Signature-Agent, product, session).¶

3.4. Identifier Generation, Stability, and Origin Scoping

To make Signed-Agent-Metadata effective for isolating noisy neighbors and avoiding shared-fate penalties, platforms SHOULD follow these generation practices:¶

1. Platform Attestation and Tenant Isolation: The platform SHOULD populate Signed-Agent-Metadata members from authenticated platform state rather than allowing tenant workloads to supply arbitrary unverified values for project (or for product if shared across tenants). If a platform passes through tenant-supplied session identifiers, the platform SHOULD also include an authenticated project member (or namespace the session identifier per tenant) so that one tenant cannot collide with or spoof identifiers belonging to another tenant.¶

2. Identifier Stability:¶

   • product and project values SHOULD remain stable across requests and across signing key rotations for the lifetime of the product or tenant workspace. Generating a new random project value on every HTTP request prevents reputation accrual and reduces the utility of the field.¶
   • session values SHOULD remain constant across HTTP requests belonging to a single bounded task execution, workflow run, or browser context lifecycle, and SHOULD change when a new task or session starts. If a platform rotates session on every request, verifiers cannot link requests within a task and will evaluate traffic at a broader tier.¶
3. Origin Scoping for session: To prevent unrelated websites from correlating a session across origins, platforms SHOULD derive session values per target origin (or per registrable domain). For example, a platform can compute an origin-scoped identifier using a keyed hash over the internal session identifier and the target authority:¶

session_token = base64url(
  HMAC_SHA256(
    platform_secret,
    internal_session_id + ":" + target_origin
  )
)

With origin scoping, a verifier at https://shopping.com observes a consistent session identifier across requests to its own origin, while https://content.org and https://news.net observe different, uncorrelated values.¶

4.1. Signing Requests

When a platform chooses to send Signed-Agent-Metadata on a signed Web Bot Auth request ([I-D.ietf-webbotauth-httpsig-protocol]), it binds the header field to the HTTP request so verifiers can authenticate it:¶

1. Construct the Header: The platform sets the Signed-Agent-Metadata header field with the applicable dictionary members.¶
2. Include in Signature-Input: The platform adds "signed-agent-metadata" to the list of covered components in the Signature-Input header field alongside "@authority" and "signature-agent".¶
3. Compute the Signature: The platform computes the HTTP Message Signature over the signature base as specified in [RFC9421].¶

4.2. Verifier Processing

When receiving an HTTP request containing Signed-Agent-Metadata, a verifier performs the following steps:¶

1. Verify Signature Coverage: Check that the Signature-Input header field for the Web Bot Auth signature (tag="web-bot-auth") includes "signed-agent-metadata" in its covered component list. If "signed-agent-metadata" is not covered by the signature, the verifier MUST ignore the Signed-Agent-Metadata header field.¶
2. Validate the Signature: Verify the HTTP Message Signature using the public key discovered via Signature-Agent, as specified in [I-D.ietf-webbotauth-httpsig-protocol]. If signature verification fails, the verifier MUST NOT trust the Signed-Agent-Metadata header field.¶
3. Parse and Scope: Parse Signed-Agent-Metadata as an [RFC9651] Dictionary. If parsing fails, ignore the header field. Otherwise, extract the recognized string members (product, project, session) and construct the Effective Identity tuple scoped under the verified Signature-Agent.¶

4.3. Complete HTTP Request Example

The following example shows an HTTP GET request to www.shopping.com, signed by Acme's agent platform (https://platform.acme.com) on behalf of its Acme-Research product, including product, project, and session in Signed-Agent-Metadata (lines folded for readability):¶

# NOTE: '\' line wrapping per RFC 8792

GET /catalog/anvils HTTP/1.1
Host: www.shopping.com
Signature-Agent: "https://platform.acme.com/.well-known/\
  http-message-signatures-directory"
Signed-Agent-Metadata:
  product="Acme-Research", project="prj_104", session="sess_3a2b1c"
Signature-Input:
  sig1=("@authority" "signature-agent" "signed-agent-metadata");\
  created=1772400000;expires=1772400300;\
  keyid="TtgsnS5fiN3jhav0LuKROGskI1rrkxZmeKndLaD8_Fg";\
  tag="web-bot-auth"
Signature:
  sig1=:OwviNIOl6eMJ4Vn006OoVEvgwglY3A6FRfjQgXYKPqBK62/0l2q3bfYiEOf+\
  wwcf3mM0UP7P99ebmYjb/xylCw==:

5.1. Reputation Isolation and Fallback

Verifiers can maintain reputation and rate-limiting state at each level of the Effective Identity tuple:¶

• Targeted Enforcement: If a single session (session="sess_3a2b1c") under project="prj_104" exhibits abusive behavior on shopping.com (such as hoarding rocket-powered roller skates or aggressive price scraping), the verifier can rate-limit, challenge, or block that specific session. Other sessions under prj_104 and other projects on platform.acme.com keep their normal standing.¶
• Hierarchical Rollup: If a tenant rotates session identifiers rapidly to evade per-session limits on content.org, the verifier can aggregate anomaly signals at the parent project tier (project="prj_104") and restrict the entire project without penalizing the rest of platform.acme.com.¶
• Graceful Fallback: If a platform signs only product="Acme-Labs" and project="prj_104", the verifier evaluates traffic at the project tier. If a request has no signed Signed-Agent-Metadata header, the verifier falls back to Tier 0 (evaluating all traffic under the platform.acme.com Signature-Agent URI).¶

5.2. Abuse Reporting to Platform Operators

When a platform publishes contact endpoints in its Signature Agent Card (contacts in [I-D.meunier-webbotauth-registry]), a verifier that detects severe abuse can report the incident to the platform by providing:¶

• The Signature-Agent URI and keyid¶
• The exact signed Signed-Agent-Metadata header value¶
• The target origin (@authority) and request timestamp (created)¶

Because the platform generated the pseudonymous project and session identifiers (or derived session via HMAC with the target origin), the platform can map the reported tuple back to the responsible internal tenant account and take corrective action.¶

6.1. Unsigned or Stripped Headers

An attacker or intermediary could inject a forged Signed-Agent-Metadata header onto an unsigned request or append Signed-Agent-Metadata to a request whose Signature-Input does not cover "signed-agent-metadata". As required in Section 4.2, verifiers MUST ignore Signed-Agent-Metadata unless "signed-agent-metadata" is included in the verified Signature-Input component list.¶

An intermediary could also strip Signed-Agent-Metadata only if "signed-agent-metadata" was not included in Signature-Input. Once "signed-agent-metadata" is listed in Signature-Input, removing or modifying the header invalidates the HTTP Message Signature.¶

6.2. Cross-Tenant Identifier Spoofing

If a multi-tenant platform allows customer code to set raw Signed-Agent-Metadata headers directly before the platform's signing proxy signs the outbound request, a malicious customer could set product or project values belonging to another customer. For example, a tenant operating product="Acme-Crimes" could claim product="Acme-Research", project="prj_104" on requests to news.net, either to inherit a trusted customer's reputation or to send abusive traffic that degrades a competitor's standing.¶

To protect its own tenants from cross-tenant spoofing, the signing platform SHOULD overwrite, namespace, or validate Signed-Agent-Metadata at its signing boundary using authenticated tenant metadata (Section 3.4).¶

6.3. Sybil Attacks and Identifier Churn

A malicious tenant may attempt to reset a poor reputation by cycling through newly created session or project identifiers on every request. Verifiers SHOULD treat newly observed identifiers as having neutral (unestablished) standing rather than high trust, and SHOULD track the rate of new sub-identifiers created under each parent project, product, or Signature-Agent. High sub-identifier churn under a parent tier is itself a signal to apply policy at the parent tier.¶

7.1. Prohibition of Personally Identifiable Information

Signed-Agent-Metadata is transmitted in cleartext HTTP headers to origin servers and intermediaries. Platforms MUST NOT place personally identifiable information (PII)—such as email addresses, legal names, usernames, phone numbers, IP addresses, or billing account names—into project or session. All values for these fields MUST be opaque tokens, UUIDs, or cryptographic hashes.¶

7.2. Cross-Origin Correlation

As documented in the Privacy Considerations of [I-D.ietf-webbotauth-httpsig-protocol], persistent identifiers in signed HTTP requests allow visited origins to correlate client activity across websites.¶

In Signed-Agent-Metadata, product and project identify commercial software or tenant workspaces, whereas session represents an individual task or browsing context. Scoping session identifiers to the target origin (Section 3.4) prevents two collaborating origins from linking a session's activity across sites, while still giving each origin a stable handle for local rate limiting and abuse defense.¶

8.1. HTTP Field Name Registration

IANA is requested to register the following entry in the "Hypertext Transfer Protocol (HTTP) Field Name Registry" defined in [RFC9110]:¶

Field Name:

Signed-Agent-Metadata¶

Status:

provisional (or permanent if adopted on Standards Track)¶

Structured Type:

Dictionary¶

Reference:

This document (Section 3)¶