Hosted Key Directories for Web Bot Auth
draft-singh-webbotauth-hosted-directories-00

Web Bot Auth                                                    S. Singh
Internet-Draft                                          Airlock Protocol
Intended status: Informational                              19 July 2026
Expires: 20 January 2027

                Hosted Key Directories for Web Bot Auth
              draft-singh-webbotauth-hosted-directories-00

Abstract

   Web Bot Auth authenticates automated clients to origins using HTTP
   Message Signatures, with verification keys published in a key
   directory at a well-known URI.  Current drafts assume that each agent
   operator hosts its own key directory.  A large and growing population
   of agents is operated by individuals and small organizations for whom
   hosting a directory is impractical, and whose legitimate traffic is
   otherwise indistinguishable from abusive automation.

   This document describes the hosted (multi-tenant) key directory
   deployment model, in which a registry operator publishes key
   directories on behalf of many independent agent operators while the
   operators retain exclusive custody of their private keys.  It
   analyzes principal mapping, proof of key possession, freshness, and
   accountability in this model; reports interoperability observations
   from an independent implementation; and offers operational guidance
   for hosted directory operators and for verifiers.  It is intended as
   input to the Web Bot Auth working group's operational and deployment
   guidance.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 20 January 2027.

Singh                    Expires 20 January 2027                [Page 1]
Internet-Draft           Hosted Key Directories                July 2026

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   4
   3.  Deployment Models for Key Directories . . . . . . . . . . . .   4
   4.  The Principal Mapping Problem . . . . . . . . . . . . . . . .   5
   5.  Proof of Key Possession . . . . . . . . . . . . . . . . . . .   6
   6.  Freshness, Caching, and Revocation  . . . . . . . . . . . . .   7
   7.  Interoperability Observations from an Independent
           Implementation  . . . . . . . . . . . . . . . . . . . . .   7
   8.  Operational Requirements for Hosted Directory Operators . . .   8
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   10. Privacy Considerations  . . . . . . . . . . . . . . . . . . .  10
   11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
   12. References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     12.1.  Normative References . . . . . . . . . . . . . . . . . .  10
     12.2.  Informative References . . . . . . . . . . . . . . . . .  11
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  11
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   Web Bot Auth applies HTTP Message Signatures [RFC9421] to automated
   traffic.  An agent signs requests with a private key; the
   corresponding public key is published in a key directory served from
   a well-known URI, and a verifier fetches the directory to validate
   the signature [I-D.meunier-webbotauth-httpsig-protocol]
   [I-D.meunier-webbotauth-httpsig-directory].

   The deployment model implicit in these documents is self-hosting: the
   entity that operates the agent also operates the HTTPS origin that
   serves its key directory.  This model fits large operators.  It does
   not fit the long tail of agents operated by individual developers,
   small businesses, and research groups.  Such operators frequently run

Singh                    Expires 20 January 2027                [Page 2]
Internet-Draft           Hosted Key Directories                July 2026

   agents from residential networks, ephemeral cloud workloads, or end-
   user devices; they may control no stable HTTPS origin at all.  As
   origins increasingly gate automated traffic on verifiable identity,
   agents without a practical path to publishing keys are treated as
   anonymous automation and denied access, regardless of their behavior
   or the legitimacy of their purpose.  The working group's use-case
   catalogue recognizes this asymmetry, observing that authenticating
   automated clients can "level the playing field for small bots"
   [I-D.nottingham-webbotauth-use-cases]; this document describes a
   deployment model that makes that levelling practical.

   Today this gap is filled in two ways, both with significant
   drawbacks:

   *  Verifier-curated allowlists of large, well-known agent services.
      These scale to tens of entries, not to hundreds of thousands of
      independent operators.

   *  Platform-mediated identity, in which an agent inherits trust by
      running on a particular hosting platform whose infrastructure
      signs its egress traffic.  This couples an agent's identity to its
      hosting choice: the identity, and any access arrangements origins
      have made for it, is lost when the agent moves.  It also
      concentrates accountability in the platform rather than in the
      party that actually directs the agent's behavior.

   This document describes a third model, the hosted key directory: a
   registry operator publishes key directories on behalf of many
   independent tenants, while each tenant generates and retains
   exclusive custody of its own private keys.  The model is directly
   analogous to established practice in other trust systems: most DKIM
   [RFC6376] signing domains publish their keys through a handful of
   mail providers; most TLS certificates are issued by shared
   certificate authorities rather than self-run ones; and most DNS zones
   are served by hosting providers rather than by their registrants.

Singh                    Expires 20 January 2027                [Page 3]
Internet-Draft           Hosted Key Directories                July 2026

   The registry document [I-D.meunier-webbotauth-registry] supplies
   discovery mechanics for this ecosystem: per-agent metadata cards, and
   registries that list them, which it explicitly contemplates being
   maintained by third parties.  It deliberately leaves trust and
   accreditation policy to implementers.  Hosted directories raise
   exactly those unaddressed operational questions: what principal a
   verifier should associate with a signature whose key lives in a
   shared directory; how possession of a key can be demonstrated when
   the directory host cannot sign with it; how quickly revocation must
   propagate through caches; and what a verifier is entitled to know
   about a directory operator's practices.  This document analyzes each
   and offers guidance, informed by an independent open-source
   implementation of the current drafts.

2.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   Agent:  An automated client that originates HTTP requests and signs
      them per the Web Bot Auth profile of [RFC9421].

   Operator (or Tenant):  The party that controls an agent's behavior
      and holds its private signing keys.  In a hosted directory, each
      operator is a tenant of the directory host.

   Hosted key directory:  A key directory served by a party other than
      the operators whose keys it advertises.

   Directory host (or Registry operator):  The party operating one or
      more hosted key directories and the enrollment process behind
      them.

   Verifier:  An origin, or infrastructure acting for origins, that
      validates Web Bot Auth signatures against a key directory.

   Principal:  The identity a verifier associates with a validated
      signature for the purpose of policy and rate limiting.

3.  Deployment Models for Key Directories

   Three deployment models are observed or anticipated in practice.
   They differ in who serves the directory, who holds private keys, and
   what happens to an agent's identity when it changes infrastructure.

Singh                    Expires 20 January 2027                [Page 4]
Internet-Draft           Hosted Key Directories                July 2026

    +=================+=============+=================+==============+
    | Property        | Self-hosted | Platform-hosted | Hosted       |
    |                 |             |                 | directory    |
    +=================+=============+=================+==============+
    | Directory       | Operator    | Hosting         | Registry     |
    | served by       |             | platform        | operator     |
    +-----------------+-------------+-----------------+--------------+
    | Private keys    | Operator    | Platform        | Operator     |
    | held by         |             |                 |              |
    +-----------------+-------------+-----------------+--------------+
    | Identity        | Yes         | No              | Yes          |
    | portable across |             |                 |              |
    | infrastructure  |             |                 |              |
    +-----------------+-------------+-----------------+--------------+
    | Operational     | High        | None            | Low          |
    | burden on       |             |                 |              |
    | operator        |             |                 |              |
    +-----------------+-------------+-----------------+--------------+
    | Accountable     | Operator    | Platform        | Registry     |
    | party visible   |             |                 | operator,    |
    | to verifier     |             |                 | and operator |
    |                 |             |                 | through it   |
    +-----------------+-------------+-----------------+--------------+

          Table 1: Properties of key directory deployment models

   The hosted model preserves a property essential to origin policy: the
   operator's identity is stable across changes of hosting, cloud
   provider, model provider, and network location, because the operator,
   not the infrastructure, holds the key.  Origins can therefore make
   per-operator access decisions that outlive any particular deployment.

4.  The Principal Mapping Problem

   The directory document [I-D.meunier-webbotauth-httpsig-directory]
   identifies a directory by the origin that serves it, and is silent on
   the relationship between a directory's keys and distinct agent
   identities.  At least one widely deployed verifier program has
   resolved that silence by treating the origin as the principal: every
   key advertised by a directory is attributed to a single logical agent
   identity, registered and reviewed as a unit.

   Applied unchanged to a hosted directory, this collapses all tenants
   into one principal.  The consequences are unacceptable in both
   directions: a single misbehaving tenant taints the shared identity
   for every other tenant (shared fate), and per-tenant policy and rate
   limits become impossible (lost granularity).

Singh                    Expires 20 January 2027                [Page 5]
Internet-Draft           Hosted Key Directories                July 2026

   Two remedies are available within the current protocol, and a third
   would benefit from working group attention:

   1.  Per-tenant directory authorities.  The directory host serves one
       directory per tenant under a distinct authority, for example
       tenant-label.agents.registry.example.  Because the Signature-
       Agent header carries the directory URI, each tenant presents a
       distinct principal to unmodified verifiers.  This works today;
       its costs are DNS and certificate automation at the host, which
       are routine at scale (DKIM's selector convention and shared CA
       issuance demonstrate the pattern at internet scale).  This is the
       RECOMMENDED deployment for hosted directories absent verifier
       support for key-level principals.

   2.  Key-level principals at the verifier.  A verifier MAY treat the
       pair (directory authority, JWK thumbprint [RFC7638]) as the
       principal rather than the authority alone.  This restores per-
       tenant granularity without protocol changes, at the cost of
       larger policy tables.

   3.  Per-tenant metadata via Signature Agent Cards.  The registry
       document [I-D.meunier-webbotauth-registry] defines a per-agent
       metadata card resolvable from a stable URL, with key material
       separable via jwks_uri.  Pairing a hosted directory with one card
       per tenant expresses tenancy explicitly without new protocol
       elements.  This document suggests the working group consider
       making that pairing normative guidance for shared directories.

5.  Proof of Key Possession

   The directory document RECOMMENDS that a directory prove possession
   of the advertised keys by signing directory responses with each key.
   A hosted directory cannot follow this recommendation, and this
   document takes the position that it MUST NOT be able to: the security
   value of the hosted model rests on the directory host never holding
   tenant private keys.  A host that can sign as its tenants is a single
   point of compromise for all of them.

   Two substitutes preserve the recommendation's intent in the hosted
   model:

   1.  Enrollment-time proof.  At enrollment, the host verifies
       possession by challenge: the prospective tenant signs a host-
       supplied nonce with the key being enrolled, binding the key
       thumbprint, the intended directory authority, and a timestamp.
       The host MUST perform such a challenge before first publication
       of a key and SHOULD repeat it at key rotation.

Singh                    Expires 20 January 2027                [Page 6]
Internet-Draft           Hosted Key Directories                July 2026

   2.  Tenant-produced directory assertions.  The tenant periodically
       signs, with its own key, a detached assertion binding its key
       thumbprint to the directory authority and a validity window.  The
       host publishes these assertions alongside the directory.  A
       verifier that wishes to check possession can validate the
       assertion without the host ever holding the key.  This
       construction is offered as a candidate for standardization; the
       author's implementation demonstrates enrollment-time proof today
       and can produce detached assertions if the working group
       standardizes a format.

   Interoperability note: no verifier observed by the author currently
   enforces directory response signatures.  The working group therefore
   has a window in which to specify a possession mechanism compatible
   with hosted directories before enforcement behavior ossifies.

6.  Freshness, Caching, and Revocation

   Directory caching directly bounds revocation latency: a verifier that
   cached a directory continues honoring a revoked key until the cache
   expires.  The example cache lifetime in current material (on the
   order of a day) is reasonable for a single-operator directory whose
   keys change rarely.  It is NOT RECOMMENDED for hosted directories,
   where tenant revocation is a routine, security-relevant event.

   A hosted directory SHOULD serve cache lifetimes on the order of
   minutes (the author's implementation defaults to 300 seconds), SHOULD
   support conditional requests so that unchanged directories cost
   little to revalidate, and MUST document its revocation propagation
   target (the maximum interval between a tenant revocation request and
   the key's disappearance from all served responses).  Push-style
   revocation signals between registries and high-volume verifiers are
   useful and complementary but are out of scope for this document.

7.  Interoperability Observations from an Independent Implementation

   The author operates an independent open-source implementation of the
   current protocol and directory drafts (signer, verifier, hosted
   directory, and enrollment).  Implementation surfaced four
   observations relevant to deployment guidance; they are reported here
   neutrally as running-code input to the working group.

Singh                    Expires 20 January 2027                [Page 7]
Internet-Draft           Hosted Key Directories                July 2026

   1.  Signature-Agent representation.  The protocol draft specifies a
       Structured Fields Dictionary form for the Signature-Agent header
       [RFC8941], while at least one major deployed verifier accepts
       only the earlier plain string form and rejects the Dictionary
       form.  Until this converges, signers interoperate most widely by
       emitting the string form, and verifiers SHOULD accept both.
       Explicit transition guidance in the working group's deliverables
       would prevent divergent deployments.

   2.  Covered components.  The drafts permit covering either @authority
       or @target-uri; at least one major deployed verifier requires
       @authority specifically.  Signers maximize interoperability by
       covering @authority; verifiers SHOULD accept either.

   3.  Nonce encoding.  Prose in current material describes the nonce as
       base64url while accompanying examples use standard base64 with
       padding.  The author's implementation follows the examples.  A
       one-line clarification would remove the ambiguity.

   4.  Nonce validation.  No verifier observed by the author validates
       nonces for replay; within its validity window a captured
       signature replays against the same authority.  Deployments SHOULD
       keep validity windows short (the author's implementation defaults
       to 60 seconds), and verifiers able to maintain per-authority
       nonce caches SHOULD do so.  Operational guidance here would
       materially improve the profile's replay posture.

8.  Operational Requirements for Hosted Directory Operators

   A verifier extending trust to a hosted directory is extending trust
   to the host's operational practices.  The following requirements make
   those practices legible.  They are offered as candidate material for
   the working group's operational guidance.

   A hosted directory operator:

   *  MUST NOT generate, receive, store, or otherwise possess tenant
      private keys;

   *  MUST verify possession of a key (per Section 5) before publishing
      it and upon rotation;

   *  MUST publish a description of its tenant verification practices,
      including any distinct assurance levels it offers (for example,
      possession-only; possession plus verified control of a domain;
      possession plus verified legal identity of the operator);

Singh                    Expires 20 January 2027                [Page 8]
Internet-Draft           Hosted Key Directories                July 2026

   *  MUST provide tenant-initiated and host-initiated revocation, and
      MUST document its revocation propagation target (Section 6);

   *  SHOULD serve each tenant under a distinct directory authority
      (Section 4);

   *  SHOULD maintain an append-only, auditable log of directory
      publication changes, so that unauthorized insertions or removals
      of keys are detectable after the fact;

   *  SHOULD publish availability information, since directory
      unavailability denies access to every tenant whose keys it serves.

9.  Security Considerations

   Key custody boundary.  The defining security property of the hosted
   model is that compromise of the directory host does not yield tenant
   signing capability.  A host that violates the custody requirement of
   Section 8 silently converts the model into platform-held identity
   with platform-sized blast radius.

   Publication integrity.  A compromised host can publish attacker keys
   under a tenant's authority or remove legitimate keys.  Auditable
   publication logs and tenant-produced assertions (Section 5) make such
   actions detectable; short cache lifetimes (Section 6) bound their
   persistence after correction.

   Enrollment integrity.  Possession challenges MUST use fresh,
   unpredictable nonces and MUST bind the intended directory authority,
   preventing an attacker from replaying an enrollment performed for one
   directory against another.

   Shared fate.  Where verifiers apply policy by directory authority
   rather than by key, one tenant's abuse degrades service for all
   tenants of a shared authority.  Per-tenant authorities and prompt
   host-initiated revocation are the mitigations; verifiers are
   encouraged toward key-level policy in any case.

   Availability.  A hosted directory is operationally an availability
   dependency for every tenant.  Hosts should treat directory serving
   with the availability discipline of authoritative DNS.

   All considerations of the underlying documents [RFC9421]
   [I-D.meunier-webbotauth-httpsig-protocol]
   [I-D.meunier-webbotauth-httpsig-directory] apply.

Singh                    Expires 20 January 2027                [Page 9]
Internet-Draft           Hosted Key Directories                July 2026

10.  Privacy Considerations

   A key directory is public; a hosted one enumerates, at minimum, the
   public keys of its tenants, and per-tenant authorities reveal tenant
   labels in DNS.  Hosts SHOULD support pseudonymous tenant labels for
   operators that have verified attributes with the host but do not wish
   to disclose them publicly, disclosing attributes to verifiers only
   through whatever assurance-level publication the host offers
   (Section 8).  The tension between operator accountability and
   operator privacy is real and merits working group discussion; this
   document does not resolve it.

11.  IANA Considerations

   This document has no IANA actions.

12.  References

12.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC9421]  Backman, A., Ed., Richer, J., Ed., and M. Sporny, "HTTP
              Message Signatures", RFC 9421, DOI 10.17487/RFC9421,
              February 2024, <https://www.rfc-editor.org/info/rfc9421>.

   [RFC8941]  Nottingham, M. and P-H. Kamp, "Structured Field Values for
              HTTP", RFC 8941, DOI 10.17487/RFC8941, February 2021,
              <https://www.rfc-editor.org/info/rfc8941>.

   [RFC7638]  Jones, M. and N. Sakimura, "JSON Web Key (JWK)
              Thumbprint", RFC 7638, DOI 10.17487/RFC7638, September
              2015, <https://www.rfc-editor.org/info/rfc7638>.

   [I-D.meunier-webbotauth-httpsig-protocol]
              Meunier, T., "HTTP Message Signatures for automated
              traffic: Protocol", Work in Progress, Internet-Draft,
              draft-meunier-webbotauth-httpsig-protocol-00, June 2026,
              <https://datatracker.ietf.org/doc/draft-meunier-
              webbotauth-httpsig-protocol/>.

Singh                    Expires 20 January 2027               [Page 10]
Internet-Draft           Hosted Key Directories                July 2026

   [I-D.meunier-webbotauth-httpsig-directory]
              Meunier, T., "HTTP Message Signatures Directory", Work in
              Progress, Internet-Draft, draft-meunier-webbotauth-
              httpsig-directory-00, June 2026,
              <https://datatracker.ietf.org/doc/draft-meunier-
              webbotauth-httpsig-directory/>.

12.2.  Informative References

   [I-D.meunier-webbotauth-registry]
              Meunier, T., "Registry and Signature Agent card for Web
              bot auth", Work in Progress, Internet-Draft, draft-
              meunier-webbotauth-registry-03, June 2026,
              <https://datatracker.ietf.org/doc/draft-meunier-
              webbotauth-registry/>.

   [I-D.nottingham-webbotauth-use-cases]
              Nottingham, M., "Use Cases for Authentication of Web
              Bots", Work in Progress, Internet-Draft, draft-nottingham-
              webbotauth-use-cases-02, April 2026,
              <https://datatracker.ietf.org/doc/draft-nottingham-
              webbotauth-use-cases/>.

   [RFC6376]  Crocker, D., Ed., Hansen, T., Ed., and M. Kucherawy, Ed.,
              "DomainKeys Identified Mail (DKIM) Signatures", STD 76,
              RFC 6376, DOI 10.17487/RFC6376, September 2011,
              <https://www.rfc-editor.org/info/rfc6376>.

Acknowledgments

   The observations in this document derive from implementing and
   operating the current Web Bot Auth drafts; the author thanks their
   authors and the deployed verifier operators whose public
   documentation made independent implementation possible.

Author's Address

   Shivdeep Singh
   Airlock Protocol
   Email: shivdeepsachdeva@gmail.com
   URI:   https://airlock.ing

Singh                    Expires 20 January 2027               [Page 11]