Internet-Draft Registry and Signature Agent card for We June 2026
Guerreiro, et al. Expires 28 December 2026 [Page]
Workgroup:
Web Bot Auth
Internet-Draft:
draft-meunier-webbotauth-registry-03
Published:
Intended Status:
Informational
Expires:
Authors:
M. Guerreiro
Cloudflare
U. Kirazci
Amazon
T. Meunier
Cloudflare

Registry and Signature Agent card for Web bot auth

Abstract

This document defines the "Signature Agent Card", a JSON metadata document that a signature agent using [DIRECTORY] publishes to describe itself: its identity, purpose, rate expectations, and cryptographic keys. Its parameters are drawn from the OAuth Dynamic Client Registration Metadata registry [DCR], the same namespace used by [CIMD], extended with a single web_bot_auth object. This document registers that object with IANA and establishes a registry for its members.

About This Document

This note is to be removed before publishing as an RFC.

The latest revision of this draft can be found at https://thibmeu.github.io/http-message-signatures-directory/draft-meunier-webbotauth-registry.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-meunier-webbotauth-registry/.

Discussion of this document takes place on the Web Bot Auth Working Group mailing list (mailto:web-bot-auth@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/web-bot-auth/. Subscribe at https://www.ietf.org/mailman/listinfo/web-bot-auth/.

Source for this draft and an issue tracker can be found at https://github.com/thibmeu/http-message-signatures-directory.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 28 December 2026.

▲

Table of Contents

1. Introduction

Signature Agents are entities that originate or forward signed HTTP requests on behalf of users or services. They include bots developers, platforms providers, and other intermediaries using [DIRECTORY]. These agents often need to identify themselves, and establish trust with origin servers.

Today, the mechanisms for doing so are inconsistent: some rely on User-Agent strings (e.g. MyCompanyBot/1.0), others on IP address lists hosted on file servers (e.g. https://curated-bots.com/ips.json), and still others on out-of-band definitions (e.g. documentation on docs.example.com/mybot). This diversity makes it difficult for operators and origin servers to reliably discover and share a Signature Agent’s purpose, contact information, or rate expectations.

The OAuth Client ID Metadata Document [CIMD] already defines a resolvable identity: a client_id that is an HTTPS URL dereferencing to a JSON metadata document. This document reuses it. A Signature Agent Card takes the client metadata parameters that [CIMD] draws from [DCR], and adds bot-specific information in a single web_bot_auth object. An existing Client ID Metadata Document is therefore a valid Signature Agent Card.

2. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

3. Signature Agent Card

A Signature Agent Card is a JSON object describing a Signature Agent. Its parameters are drawn from the OAuth Dynamic Client Registration Metadata registry [DCR], the same parameter namespace used by [CIMD]. Web bot auth-specific information is carried in a single web_bot_auth object, defined in Section 3.2.

Because the card reuses the [CIMD] parameter namespace, an existing Client ID Metadata Document is a valid Signature Agent Card. A client that does not understand the web_bot_auth object ignores it.

The Signature-Agent header is defined in Section 4.1 of [DIRECTORY]. A Signature Agent Card can be referenced from that header using type=cimd, as described in Section 4.1.

{
  "client_id": "https://example.com/bot",
  "client_name": "Example Bot",
  "client_uri": "https://example.com/bot/about.html",
  "logo_uri": "https://example.com/logo.png",
  "contacts": ["mailto:bot-support@example.com"],
  "jwks_uri": "https://example.com/.well-known/http-message-signatures-directory",
  "web_bot_auth": {
    "expected-user-agent": "Mozilla/5.0 ExampleBot",
    "rfc9309-product-token": "ExampleBot",
    "rfc9309-compliance": ["User-Agent", "Allow", "Disallow", "Content-Usage"],
    "trigger": "fetcher",
    "purpose": "tdm",
    "targeted-content": "Cat pictures",
    "rate-control": "429",
    "rate-expectation": "avg=10rps;max=100rps",
    "known-urls": ["/", "/robots.txt", "*.png"],
    "ips_uri": "https://example.com/ips.json"
  }
}

Unless otherwise specified, all parameters in this document are OPTIONAL, except that a card resolved through its client_id MUST carry client_id (see Section 3.1.1). There MUST be at least one parameter set.

Parameters for which the value is unknown MUST be ignored. All string values are UTF-8.

3.1. Client Metadata Parameters

The following parameters are OAuth client metadata parameters defined in [DCR] and used as in [CIMD]. They keep the semantics defined there; this section only notes how they are used by a Signature Agent.

client_id

A resolvable identifier for the Signature Agent. See Section 3.1.1.

client_name

A friendly name for the Signature Agent, e.g. ExampleBot.

client_uri

A URL of a web page describing the Signature Agent: what it does, and how it handles the data it fetches, e.g. https://example.com/bot/about.html.

logo_uri

A URL referencing a logo for the Signature Agent, for quick visual identification, e.g. https://example.com/logo.png.

contacts

An array of URIs providing reliable communication channels, typically email addresses, e.g. ["mailto:bot-support@example.com"].

jwks_uri

See Section 3.1.2.

jwks

A JWK Set as defined in Section 5 of [JWK]. See Section 3.1.3.

3.1.1. Client ID

The client_id parameter is the resolvable identity of the Signature Agent, as defined in [CIMD]. It is an HTTPS URL that, when dereferenced, returns the Signature Agent Card. A card retrieved through its client_id MUST include the client_id parameter.

A client retrieving a card from a URL MUST use the GET method, MUST treat any status code other than 200 (OK) as an error, and MUST NOT follow HTTP redirects. The client_id value in the returned document MUST be identical to the URL used to retrieve it, compared as described in Section 6.2.1 of [RFC3986] (simple string comparison).

Example

  • https://example.com/bot

3.1.2. JWKS URI

The jwks_uri parameter provides the URL of a JWK Set as defined in Section 5 of [JWK], following the [DCR] semantics. [DCR] does not mandate a media type for the resource at jwks_uri.

When present, this parameter separates key material discovery from metadata discovery. Clients that need key material SHOULD fetch the key set at the given URL rather than relying on the jwks parameter. This separation allows registry operators to host metadata and key material on different endpoints, supporting deployment scenarios where the registry endpoint itself contains signature agent card metadata but key material is hosted elsewhere.

A Signature Agent Card MUST NOT contain both jwks_uri and jwks. Clients SHOULD reject a card that contains both parameters.

The resource at jwks_uri SHOULD be signed using [HTTP-MESSAGE-SIGNATURES]. Clients SHOULD validate the signature and ignore keys that do not carry a corresponding valid signature.

The URI scheme MUST be https.

Example

  • https://example.com/.well-known/http-message-signatures-directory

3.1.3. JWKS

The jwks parameter contains a JWK Set as defined in Section 5 of [JWK]. It is the inline client metadata parameter defined in [DCR].

If jwks is present, it is RECOMMENDED that the card is signed using [HTTP-MESSAGE-SIGNATURES]. Content-Digest header MUST be included in the covered components.

TODO: describe signature, CWS keys.

3.2. Web Bot Auth Extension

The web_bot_auth parameter is a JSON object carrying bot-specific metadata. It is registered in the OAuth Dynamic Client Registration Metadata registry [DCR] (see Section 7), so that it may appear in any Client ID Metadata Document [CIMD]. Its members are governed by the "Web Bot Auth Metadata Parameters" registry defined in Section 7. Members for which the value is unknown MUST be ignored.

The members defined by this document are below.

3.2.1. Expected user agent

The expected-user-agent parameter specifies one or more User-Agent strings as defined in Section 10.1.5 of [HTTP] or prefix matches. Prefixes MAY use * as a wildcard.

Example

  • Mozilla/5.0 ExampleBot

3.2.2. robots.txt product token

The rfc9309-product-token parameter specifies the product token used for robots.txt directives per Section 2.2.1 of [ROBOTSTXT].

Example

  • ExampleBot

3.2.3. robots.txt compliance

The rfc9309-compliance parameter lists directives from robots.txt that the agent implements.

Example

  • ["User-Agent", "Disallow"]

  • ["User-Agent", "Disallow", "CrawlDelay"]

3.2.4. Trigger

The trigger parameter indicates the operational mode of the agent.

Valid values:

  1. fetcher - request initiated by the user

  2. crawler - autonomous scanning

3.2.5. Purpose

The purpose parameter describes the intended use of collected data. Values SHOULD be drawn from a controlled vocabulary, such as [AIPREF-VOCAB].

Example

3.2.6. Targeted content

The targeted-content parameter specifies the type of data the agent seeks. Its format is arbitrary UTF-8 encoded string.

Example

  • SEO analysis

  • Vulnerability scanning

  • Ads verification

3.2.7. Rate control

The rate-control parameter indicates how origins can influence the agent’s request rate.

TODO: specify a format

Example

3.2.8. Rate expectation

The rate-expectation parameter specifies anticipated request volume or burstiness.

TODO: consider a format such as avg=10rps;max=100rps

Example

  • 500 rps

  • Spikes during reindexing

3.2.9. Known URLs

The known-urls parameter lists predictable endpoints accessed by the agent.

These URLs may be absolute URLs like https://example.com/index.html. They could be relative path like /ads.txt. Or they can use * as wildcard such as *.png.

Example

  • ["/"]

  • ["/ads.txt"]

  • ["/favicon.ico"]

  • ["/index.html"]

3.2.10. IP Address List URI

The ips_uri parameter provides the URL of the signature agent's IP address list as defined in [JAFAR].

The URI scheme MUST be https.

Example

  • https://example.com/ips.json

4. Discovery

A Signature Agent Card is discovered by dereferencing its client_id, as described in Section 4.1, or from a registry.

A registry is a list of URLs, each refering to a signature agent card. An entry MAY be the client_id of a Signature Agent Card.

The URI scheme MUST be one of:

  • https: Points to an HTTPS resource serving a signature agent card

  • data: Contains an inline signature agent card

4.1. CIMD-based discovery

A Signature Agent advertises its identity through a resolvable client_id URL, as defined in [CIMD]. A consumer discovers the agent's metadata and keys as follows:

  1. Obtain a client_id URL, from a Signature-Agent header with type=cimd (Section 4.5), a registry entry, or out of band.

  2. Dereference the client_id with a GET request. The response MUST be 200 (OK), redirects MUST NOT be followed, and the returned client_id MUST match the requested URL (see Section 3.1.1). The response is the Signature Agent Card.

  3. Obtain key material. If the card has a jwks_uri, fetch the JWK Set at that URL; otherwise use the inline jwks parameter.

The metadata document and the key set are distinct resources reached at different hops, so no content negotiation between them is required.

GET /bot HTTP/1.1
Host: example.com

HTTP/1.1 200 OK
Content-Type: application/json

{
  "client_id": "https://example.com/bot",
  "client_name": "Example Bot",
  "jwks_uri": "https://example.com/.well-known/http-message-signatures-directory",
  "web_bot_auth": {
    "trigger": "fetcher",
    "purpose": "tdm"
  }
}

Example

# An example list of bots
https://bot1.example.com/.well-known/http-message-signatures-directory
https://crawler2.example.com/.well-known/http-message-signatures-directory

# Now the list of platforms
https://zerotrust-gateway.example.com/v1/signature-agent-card

# Below is an inlined card with the data URL scheme
data:application/json,{"client_name":"Inline Bot","jwks_uri":"https://inline.example.com/.well-known/http-message-signatures-directory"}

4.2. Formal Syntax

Below is an Augmented Backus-Naur Form (ABNF) description, as described in [ABNF].

The below definition imports https-URI from [HTTP], and dataurl from [DATAURL].

registry = *(cardendpointline / emptyline)
cardendpointline = (
    https-URI /      ; As defined in Section 4.2.2 of RFC 9110
    dataurl          ; As defined in Section 3 of RFC 2397
) EOL


comment = "#" *(UTF8-char-noctl / WS / "#")
emptyline = EOL
EOL = *WS [comment] NL ; end-of-line may have
                       ; optional trailing comment
NL = %x0D / %x0A / %x0D.0A
WS = %x20 / %x09

; UTF8 derived from RFC 3629, but excluding control characters

UTF8-char-noctl = UTF8-1-noctl / UTF8-2 / UTF8-3 / UTF8-4
UTF8-1-noctl = %x21 / %x22 / %x24-7F ; excluding control, space, "#"
UTF8-2 = %xC2-DF UTF8-tail
UTF8-3 = %xE0 %xA0-BF UTF8-tail / %xE1-EC 2UTF8-tail /
         %xED %x80-9F UTF8-tail / %xEE-EF 2UTF8-tail
UTF8-4 = %xF0 %x90-BF 2UTF8-tail / %xF1-F3 3UTF8-tail /
         %xF4 %x80-8F 2UTF8-tail

UTF8-tail = %x80-BF

4.3. Out-of-band communication between client and origin

A signature agent MAY submit their signature agent card to an origin, or the origin MAY manually add them to their local registry.

4.4. Registry Endpoint

A registry MAY be provided via a GitHub repository, a public file server, or a dedicated endpoint.

The registry SHOULD be served over HTTPS.

A client application SHOULD validate the registry format and reject malformed entries.

4.4.1. Authentication

A registry endpoint MAY require authentication to restrict access to authorized clients. This allows a registry operator to expose registries without revealing their contents publicly.

No specific authentication mechanism is mandated. Implementations MAY use pre-shared keys as mentioned in [PSK-TLS], bearer tokens as defined in [OAUTH-BEARER], or HTTP Message Signatures as defined in [HTTP-MESSAGE-SIGNATURES]. HTTP Message Signatures are RECOMMENDED when key rotation without out-of-band coordination is desired.

4.4.2. Efficient Polling with Conditional Requests

Registry servers SHOULD include ETag and Last-Modified response header fields as defined in Section 8.8 of [HTTP].

Clients SHOULD send If-None-Match or If-Modified-Since precondition header fields as defined in Section 13.1 of [HTTP] on subsequent requests. A server SHOULD respond with 304 (Not Modified) when the registry has not changed, avoiding redundant data transfer.

4.4.3. Caching

Registry servers SHOULD include a Cache-Control response header field as defined in [HTTP-CACHE] to communicate the intended freshness lifetime of the registry content. Clients SHOULD respect these cache directives and SHOULD NOT poll more frequently than indicated.

4.5. Signature-Agent header

When used for HTTP Message Signatures, a Signature Agent Card MAY be discovered via a Signature-Agent header member with type=cimd. The URI carried in that member is the client_id of a Signature Agent Card resolved as described in Section 4.1.

4.6. Composable discovery with Signature-Key

The discovery in this document is anchored on the client_id URL and is independent of the header that carries it. [SIGNATURE-KEY] defines a composable Signature-Key header with an extensible registry of key-distribution schemes. A future scheme could carry a client_id URL, letting a verifier resolve a Signature Agent Card per-signature alongside the keying material. This document does not define such a scheme; it is noted as a possible composable carrier.

4.7. Change Notification

Pull-based consumption with conditional requests is sufficient for most deployments. When lower notification latency is required (e.g., to promptly act on entry removal), a registry operator MAY implement a push-based change notification mechanism.

4.7.1. Advertising the Notification Endpoint

A registry operator that supports change notifications SHOULD advertise its notification endpoint in the registry HTTP response using a Link header field as defined in [WEB-LINKING]:

Link: <https://registry.example/v1/registry-changes>; rel="registry-changes"

The registry-changes link relation identifies an endpoint to which clients may register callbacks out of band.

TODO: Register the registry-changes link relation with IANA, or replace it with an extension relation URI.

4.7.2. Callback Registration

Registration of a client callback URL with the registry operator is performed out of band. No specific registration protocol is defined by this document. An unsubscribe mechanism SHOULD be considered, and MAY also be out of band.

4.7.3. Notification Requests

When an entry is added to or removed from the registry, the registry operator MUST send an HTTP request to each registered callback URL.

The format in [CDDL] is as follows:

Action = "put" / "delete"

Notification = {
  action: Action,
  signature-agent: tstr
}

TODO: should we use application/json, a new media-type, permit binary encoding? TODO: should signature-agent be an array?

action denotes the operation performed on the registry: 1. put when a new entry has been added, 2. delete when an entry has been removed.

The request MUST use POST. It MUST be signed using [HTTP-MESSAGE-SIGNATURES] with a key present in the registry operator's existing signature agent card. This is the signature agent card associated with the registry operator during out-of-band callback registration.

The Content-Type SHOULD be application/json.

Example notification for an added entry:

POST /registry-callback HTTP/1.1
Host: origin.example
Content-Type: application/json
Signature-Input: sig1=("@method" "@authority" "@path" "content-digest"); \
  created=1741046400; keyid="NFcWBst6DXG-N35nHdzMrioWntdzNZghQSkjHNMMSjw"
Signature: sig1=:base64signature:
Content-Digest: sha-256=:base64hash:

{
  "action": "put",
  "signature-agent": "https://abc123.registry.example/.well-known/signature-agent-card"
}

4.7.4. Notification Processing

Upon receiving a notification, the client MUST verify the HTTP Message Signature against the registry operator's key, discovered via the registry operator's signature-agent card. Notifications with missing or invalid signatures MUST be rejected.

A notification is advisory only. The registry endpoint remains the authoritative source of truth. After verifying a put notification, clients SHOULD re-fetch the affected signature agent card to obtain current metadata.

For delete notifications, clients SHOULD confirm the removal by re-fetching the full registry. This confirmation does not need to happen synchronously with notification processing.

Registry operators SHOULD retry delivery of failed notifications with exponential backoff. Clients that miss notifications will recover on their next conditional pull from the registry endpoint.

5. Security Considerations

Malicious actors may put properties which are not theirs in the registry. If signatures are present, clients MUST verify them. Clients SHOULD reject cards with invalid signatures.

5.1. Registry Integrity

When a registry is served over HTTPS, TLS provides channel integrity between the server and the client. To additionally bind the registry contents to the registry operator's cryptographic identity, registry servers SHOULD sign their HTTP responses using [HTTP-MESSAGE-SIGNATURES] with a key present in the registry operator's own signature agent card. Clients SHOULD verify such signatures when present.

5.2. Binding Delegated Keys to the Directory Authority

When a signature agent card delegates key discovery using jwks_uri, clients SHOULD validate the referenced directory response to ensure the authenticity and integrity of the delegated key material.

When a directory server provides key material over HTTP or HTTPS, it is RECOMMENDED that it include one HTTP Message Signature per key in the response, with each key used to provide one signature. The signature SHOULD cover @authority with the req flag set, and SHOULD include the created, expires, keyid, and tag signature parameters. The tag value MUST be http-message-signatures-directory.

Clients SHOULD validate these signatures using the keys provided by the directory. Clients SHOULD ignore keys from a directory response that do not have a corresponding valid signature binding the key material to the directory authority.

5.3. Private Registries

Registry endpoints that require authentication as described in Section 4.4.1 limit exposure of registry entries to authorized clients only. Registry operators SHOULD use authenticated endpoints when the enumeration of their registry entries is sensitive.

6. Privacy Considerations

TODO

6.1. Access Patterns

Registry servers SHOULD avoid logging personally identifiable information from client requests. Clients fetching a registry reveal their interest in its entries; registry servers SHOULD treat access logs as sensitive.

7. IANA Considerations

7.1. OAuth Dynamic Client Registration Metadata Registration

IANA is requested to register the following parameter in the "OAuth Dynamic Client Registration Metadata" registry established by [DCR].

Client Metadata Name:

web_bot_auth

Client Metadata Description:

A JSON object carrying bot-specific metadata for a Signature Agent. Its members are registered in the "Web Bot Auth Metadata Parameters" registry.

Change Controller:

IETF

Reference:

Section 3.2

The remaining parameters used by a Signature Agent Card (client_id, client_name, client_uri, logo_uri, contacts, jwks_uri, jwks) are already registered OAuth client metadata parameters and are not registered by this document.

7.2. Web Bot Auth Metadata Parameters Registry

IANA is requested to create the "Web Bot Auth Metadata Parameters" registry. This registry governs the members of the web_bot_auth object defined in Section 3.2.

7.2.1. Registration template

New registrations need to list the following attributes:

Parameter Name:

The name requested (e.g. "purpose"). This name is case sensitive. Names may not match other registered names in a case-insensitive manner unless the Designated Experts state that there is a compelling reason to allow an exception

Parameter Description:

Brief description of the parameter

Change Controller:

For Standards Track RFCs, list the "IESG". For others, give the name of the responsible party. Other details (e.g., postal address, email address, home page URI) may also be included.

Reference:

Where this parameter is defined

Notes:

Any notes associated with the entry

New entries in this registry are subject to the Specification Required registration policy ([RFC8126], Section 4.6). Designated experts can reject registrations on the basis that they do not meet the security and privacy requirements defined in TODO.

7.2.2. Initial Registry content

This section registers the web_bot_auth member names defined in Section 3.2 in this registry.

7.2.2.1. Expected User Agent Parameter
Parameter Name:

expected-user-agent

Parameter Description:

String or fragment patterns

Change Controller:

IETF

Reference:

Section 3.2.1

Notes:

N/A

7.2.2.2. RFC9309 Product Token Parameter
Parameter Name:

rfc9309-product-token

Parameter Description:

Robots.txt product token your signature-agent satisfies.

Change Controller:

IETF

Reference:

Section 3.2.2

Notes:

N/A

7.2.2.3. RFC9309 Compliance Parameter
Parameter Name:

rfc9309-compliance

Parameter Description:

Does your signature-agent respect robots.txt.

Change Controller:

IETF

Reference:

Section 3.2.3

Notes:

N/A

7.2.2.4. Trigger Parameter
Parameter Name:

trigger

Parameter Description:

Fetcher/Crawler

Change Controller:

IETF

Reference:

Section 3.2.4

Notes:

N/A

7.2.2.5. Purpose Parameter
Parameter Name:

purpose

Parameter Description:

Intended use for the collected data

Change Controller:

IETF

Reference:

Section 3.2.5

Notes:

N/A

7.2.2.6. Targeted Content Parameter
Parameter Name:

targeted-content

Parameter Description:

Type of data your agent seeks

Change Controller:

IETF

Reference:

Section 3.2.6

Notes:

N/A

7.2.2.7. Rate control Parameter
Parameter Name:

rate-control

Parameter Description:

How can an origin control your crawl rate

Change Controller:

IETF

Reference:

Section 3.2.7

Notes:

N/A

7.2.2.8. Rate expectation Parameter
Parameter Name:

rate-expectation

Parameter Description:

Expected traffic and intensity

Change Controller:

IETF

Reference:

Section 3.2.8

Notes:

N/A

7.2.2.9. Known URLs Parameter
Parameter Name:

known-urls

Parameter Description:

Predictable endpoint accessed

Change Controller:

IETF

Reference:

Section 3.2.9

Notes:

N/A

7.2.2.10. IP Address List URI Parameter
Parameter Name:

ips_uri

Parameter Description:

URL of the signature agent's IP address list in [JAFAR] format.

Change Controller:

IETF

Reference:

Section 3.2.10

Notes:

N/A

8. References

8.1. Normative References

[ABNF]
Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, , <https://www.rfc-editor.org/rfc/rfc5234>.
[AIPREF-VOCAB]
Keller, P. and M. Thomson, "A Vocabulary For Expressing AI Usage Preferences", Work in Progress, Internet-Draft, draft-ietf-aipref-vocab-06, , <https://datatracker.ietf.org/doc/html/draft-ietf-aipref-vocab-06>.
[CDDL]
Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, , <https://www.rfc-editor.org/rfc/rfc8610>.
[CIMD]
Parecki, A. and E. Smith, "OAuth Client ID Metadata Document", Work in Progress, Internet-Draft, draft-ietf-oauth-client-id-metadata-document-01, , <https://datatracker.ietf.org/doc/html/draft-ietf-oauth-client-id-metadata-document-01>.
[DCR]
Richer, J., Ed., Jones, M., Bradley, J., Machulak, M., and P. Hunt, "OAuth 2.0 Dynamic Client Registration Protocol", RFC 7591, DOI 10.17487/RFC7591, , <https://www.rfc-editor.org/rfc/rfc7591>.
[DIRECTORY]
Meunier, T. and S. Major, "HTTP Message Signatures Directory", Work in Progress, Internet-Draft, draft-meunier-webbotauth-httpsig-directory-00, , <https://datatracker.ietf.org/doc/html/draft-meunier-webbotauth-httpsig-directory-00>.
[HTTP]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, , <https://www.rfc-editor.org/rfc/rfc9110>.
[HTTP-CACHE]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Caching", STD 98, RFC 9111, DOI 10.17487/RFC9111, , <https://www.rfc-editor.org/rfc/rfc9111>.
[HTTP-MESSAGE-SIGNATURES]
Backman, A., Ed., Richer, J., Ed., and M. Sporny, "HTTP Message Signatures", RFC 9421, DOI 10.17487/RFC9421, , <https://www.rfc-editor.org/rfc/rfc9421>.
[HTTP-MORE-STATUS-CODE]
Nottingham, M. and R. Fielding, "Additional HTTP Status Codes", RFC 6585, DOI 10.17487/RFC6585, , <https://www.rfc-editor.org/rfc/rfc6585>.
[JAFAR]
Illyes, G., "A JSON-Based Format for Publishing IP Ranges of Automated HTTP Clients", Work in Progress, Internet-Draft, draft-illyes-webbotauth-jafar-00, , <https://datatracker.ietf.org/doc/html/draft-illyes-webbotauth-jafar-00>.
[JWK]
Jones, M., "JSON Web Key (JWK)", RFC 7517, DOI 10.17487/RFC7517, , <https://www.rfc-editor.org/rfc/rfc7517>.
[JWK-OKP]
Liusvaara, I., "CFRG Elliptic Curve Diffie-Hellman (ECDH) and Signatures in JSON Object Signing and Encryption (JOSE)", RFC 8037, DOI 10.17487/RFC8037, , <https://www.rfc-editor.org/rfc/rfc8037>.
[JWK-THUMBPRINT]
Jones, M. and N. Sakimura, "JSON Web Key (JWK) Thumbprint", RFC 7638, DOI 10.17487/RFC7638, , <https://www.rfc-editor.org/rfc/rfc7638>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC3986]
Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, , <https://www.rfc-editor.org/rfc/rfc3986>.
[RFC8126]
Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, , <https://www.rfc-editor.org/rfc/rfc8126>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[WEB-LINKING]
Nottingham, M., "Web Linking", RFC 8288, DOI 10.17487/RFC8288, , <https://www.rfc-editor.org/rfc/rfc8288>.

8.2. Informative References

[CBCP]
Illyes, G., Kühlewind, M., and K. Aj, "Crawler best practices", Work in Progress, Internet-Draft, draft-illyes-webbotauth-cbcp-00, , <https://datatracker.ietf.org/doc/html/draft-illyes-webbotauth-cbcp-00>.
[DATAURL]
Masinter, L., "The "data" URL scheme", RFC 2397, DOI 10.17487/RFC2397, , <https://www.rfc-editor.org/rfc/rfc2397>.
[OAUTH-BEARER]
Jones, M. and D. Hardt, "The OAuth 2.0 Authorization Framework: Bearer Token Usage", RFC 6750, DOI 10.17487/RFC6750, , <https://www.rfc-editor.org/rfc/rfc6750>.
[OPENID-CONNECT-DISCOVERY]
"OpenID Connect Discovery 1.0", n.d., <https://openid.net/specs/openid-connect-discovery-1_0.html>.
[PSK-TLS]
Housley, R., Hoyland, J., Sethi, M., and C. A. Wood, "Guidance for External Pre-Shared Key (PSK) Usage in TLS", RFC 9257, DOI 10.17487/RFC9257, , <https://www.rfc-editor.org/rfc/rfc9257>.
[RATELIMIT-HEADER]
Polli, R., Ruiz, A. M., and D. Miller, "RateLimit header fields for HTTP", Work in Progress, Internet-Draft, draft-ietf-httpapi-ratelimit-headers-11, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpapi-ratelimit-headers-11>.
[RFC8446]
Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, , <https://www.rfc-editor.org/rfc/rfc8446>.
[ROBOTSTXT]
Koster, M., Illyes, G., Zeller, H., and L. Sassman, "Robots Exclusion Protocol", RFC 9309, DOI 10.17487/RFC9309, , <https://www.rfc-editor.org/rfc/rfc9309>.
[SIGNATURE-KEY]
Hardt, D. and T. Meunier, "HTTP Signature Keys", Work in Progress, Internet-Draft, draft-hardt-httpbis-signature-key-05, , <https://datatracker.ietf.org/doc/html/draft-hardt-httpbis-signature-key-05>.
[UTF8]
Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, , <https://www.rfc-editor.org/rfc/rfc3629>.

Acknowledgments

TODO

The editor would also like to thank the following individuals (listed in alphabetical order) for feedback, insight, and implementation of this document -

Changelog

draft-meunier-webbotauth-registry-03

  • Make the Signature Agent Card an OAuth client metadata document with a web_bot_auth object.

  • Add resolvable client_id discovery: GET, 200 OK, no redirects, and an exact client_id match.

  • Allow Signature-Agent members with type=cimd and registry entries to point to a client_id URL.

  • Register web_bot_auth in the OAuth Dynamic Client Registration Metadata registry.

  • Move bot-specific fields into a new Web Bot Auth Metadata Parameters registry.

  • Stop redefining shared client metadata fields; reference RFC 7591 instead.

  • Drop the data:text/plain restriction on client_uri.

  • Rename inline keys from keys to jwks to match OAuth client metadata.

  • State that jwks_uri points to a JWK Set, without requiring a media type.

  • Require client_id in a card resolved through its client_id.

  • Require jwks and jwks_uri to be mutually exclusive.

  • Restrict registry entries and resolvable cards to https.

  • Say the jwks_uri resource SHOULD be signed using HTTP Message Signatures.

  • Note Signature-Key as a possible carrier.

draft-meunier-webbotauth-registry-02

  • Add ips_uri so a client can publish IP addresses.

  • Add jwks_uri to separate key material from metadata.

  • Fix the inline data URL example.

  • Rename "Public list" to "Registry Endpoint".

  • Add authentication guidance for private registry endpoints.

  • Add conditional GET guidance using ETag and If-Modified-Since.

  • Add caching guidance using HTTP-CACHE.

  • Add change notifications using signed PUT and DELETE webhooks.

  • Keep pull-based registry fetches as the source of truth.

  • Add registry integrity and private registry guidance to Security Considerations.

  • Add customer list exposure and access patterns to Privacy Considerations.

  • Add RFC 8288 as a normative reference.

draft-meunier-webbotauth-registry-01

  • Add contributors.

  • Align the registry draft with the OAuth Dynamic Client Registration Metadata registry.

  • Add an about URL field.

  • Add ABNF for registry discovery.

  • Clarify known URLs.

  • Add a placeholder for image size.

draft-meunier-webbotauth-registry-00

  • Initial draft.

Authors' Addresses

Maxime Guerreiro
Cloudflare
Ulas Kirazci
Amazon
Thibault Meunier
Cloudflare