Automated Discovery Of Audit Reports (audits.json)
draft-dkg-audits-json-01

intarea                                                    D. K. Gillmor
Internet-Draft                                            M. K. Gerchick
Intended status: Informational                                      ACLU
Expires: 22 April 2027                                   19 October 2026

           Automated Discovery Of Audit Reports (audits.json)
                        draft-dkg-audits-json-01

Abstract

   This document describes a mechanism that an organization can use to
   enable automatic discovery of documents associated with regulatory
   compliance.  It is motivated by regulations that require, for
   example, publicly accessible audits of automated decision-making
   processes in hiring.

About This Document

   This note is to be removed before publishing as an RFC.

   The latest revision of this draft can be found at
   https://dkg.github.io/audits.json/.  Status information for this
   document may be found at https://datatracker.ietf.org/doc/draft-dkg-
   audits-json/.

   Source for this draft and an issue tracker can be found at
   https://github.com/dkg/audits.json/.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 22 April 2027.

Gillmor & Gerchick        Expires 22 April 2027                 [Page 1]
Internet-Draft                 audits.json                  October 2026

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
     1.2.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   3
     1.3.  Goals . . . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.4.  Non-Goals . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Locating The audits.json Summary  . . . . . . . . . . . . . .   4
   3.  audits.json Structure . . . . . . . . . . . . . . . . . . . .   4
     3.1.  audits Content  . . . . . . . . . . . . . . . . . . . . .   5
     3.2.  operations Content  . . . . . . . . . . . . . . . . . . .   5
   4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   5
   5.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   5
     5.1.  Normative References  . . . . . . . . . . . . . . . . . .   6
     5.2.  Informative References  . . . . . . . . . . . . . . . . .   6
   Appendix A.  Test Vectors . . . . . . . . . . . . . . . . . . . .   7
     A.1.  Example audits.json . . . . . . . . . . . . . . . . . . .   7
   Appendix B.  JSON Schema For audits.json  . . . . . . . . . . . .   8
   Appendix C.  Examples of Regulatory Regimes . . . . . . . . . . .  10
     C.1.  Example Regulatory Text . . . . . . . . . . . . . . . . .  11
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  11
   Document History  . . . . . . . . . . . . . . . . . . . . . . . .  11
     Changes between -00 and -01 . . . . . . . . . . . . . . . . . .  11
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   An increasing number of regulatory regimes require organizations
   involved in certain business practices to provide a degree of
   transparency in their business operations by posting reports of
   audits in an accessible location.  For example, New York City's 2021
   Local Law 144 ([NYC-LL144]) established auditing requirements for
   employers who use automated decision-making tools in the employment
   process, but it has proved challenging to even find these audits

Gillmor & Gerchick        Expires 22 April 2027                 [Page 2]
Internet-Draft                 audits.json                  October 2026

   effectively (see [Auditing-the-Audits]).

   For a business that has a website, the natural place to provide
   access to the audit is on that website.  This document describes a
   standard mechanism that can be used to point to any audit posted
   associated with regulatory compliance.

   The mechanism is an audits.json summary document, found at a well-
   known URL, which allows the hosting domain to refer to specific
   business operations, relevant regulatory regimes, and their
   associated audit reports.

   Adoption of this mechanism should make it easier for businesses to
   comply with these requirements, while also making it easier for
   researchers, analysts, and regulators to assess compliance and
   evaluate the overall effectiveness of the regulations.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

1.2.  Terminology

   *  An "audit report" is a document produced by an organization or an
      independent entity commissioned by the organization to describe --
      according to some kind of regulatory compliance -- a part of the
      organization's business operations.

1.3.  Goals

   *  Provide a standardized way for an organization to publicize audit
      reports associated with given regulatory requirements.

   *  Provide a standardized way for a reviewer of audit reports
      associated with a regulatory requirement to find those audit
      reports.

1.4.  Non-Goals

   Discovering an audit report is merely the first step in a process of
   having an effective regulatory regime based on audits.  This
   specification is focused merely on this satisfying this necessary but
   insufficient stage of larger work.

Gillmor & Gerchick        Expires 22 April 2027                 [Page 3]
Internet-Draft                 audits.json                  October 2026

   *  This specification makes no attempt to describe the syntax or
      semantics of any particular audit report.  The underlying
      assumption is that the audit reports for different regulations or
      compliance measures will have different requirements for content,
      structure, syntax, and so on.

   *  This specification also does not associate a real-world
      organization with a particular domain name.  If a researcher wants
      to find the audits associated with Example Corp, they need to
      associate Example Corp with the example.com domain name
      independently in order to use this specification.  Likewise, any
      regulation making use of this mechanism needs to explicitly
      describe the link between the regulated entity and the domain name
      or domain names on which it is expected to publish this summary.

   *  This specification does not describe a way for a casual visitor to
      a given website to find these audit reports.  A regulation that
      requires easy public accessibility of a report may need to offer
      additional user interface or user experience guidance in addition
      to requiring the use of this mechanism.

2.  Locating The audits.json Summary

   This specification uses the .well-known URL space defined by
   [RFC8615].

   A given domain hosts the audits.json summary in /.well-known/
   audits.json within the website operated by the organization.

   If Example Corporation operats https://example.com/, then the summary
   report would be found at https://example.com/.well-known/audits.json.

3.  audits.json Structure

   The object served from the URL described in Section 2 will have
   Content-Type: application/json, and will consist of a single
   dictionary object with at least two top-level keys: operations and
   audits.

   The overall object relationship looks like this:

   +------------+       +--------+
   | operations |   .-->| audits |
   +------------+  |    +--------+
   | urls       |  |    | title  |
   | regs       |  |    | urls   |
   | audits     +-'     +--------+
   +------------+

Gillmor & Gerchick        Expires 22 April 2027                 [Page 4]
Internet-Draft                 audits.json                  October 2026

                 Figure 1: Data Structure For `audits.json`

3.1.  audits Content

   The audits member is an object where each member is an object with:

   *  title, simple textual string describing the audit, and

   *  date, date object describing the calendar date (year, month, and
      day) of the audit's publication, and

   *  urls, an array of URLs that point to the relevant materials for
      the audit

3.2.  operations Content

   The operations member is a list of objects, each of which describes
   some set of business operations, via the following members:

   *  urls, an array of URLs, each of which describes a business
      operation covered by this object (for example, a job listing),

   *  regs, an array of URLs, each of which refers to a piece of
      regulatory guidance, and

   *  audits, an array of keys which can be used to point to specific
      audits

4.  IANA Considerations

   IANA should register audits.json in the "Well-Known URIs" registry,
   with the following values:

   *  URI suffix: audits.json

   *  Change controller: IETF (is this the right choice, if this is an
      informational draft?  is there a better group to control updates
      to this specification?)

   *  Specification document(s): This document

   *  Status: provisional (unless we see wider adoption, in which case
      we should ask for permanent)

   *  Related information: Any other examples we want to point to?

5.  References

Gillmor & Gerchick        Expires 22 April 2027                 [Page 5]
Internet-Draft                 audits.json                  October 2026

5.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/rfc/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

   [RFC8615]  Nottingham, M., "Well-Known Uniform Resource Identifiers
              (URIs)", RFC 8615, DOI 10.17487/RFC8615, May 2019,
              <https://www.rfc-editor.org/rfc/rfc8615>.

5.2.  Informative References

   [Auditing-the-Audits]
              Gerchick, M., Encarnación, R., Tanigawa-Lau, C.,
              Armstrong, L., Gutiérrez, A., and D. Metaxa, "Auditing the
              Audits: Lessons for Algorithmic Accountability from Local
              Law 144's Bias Audits", ACM, Proceedings of the 2025 ACM
              Conference on Fairness, Accountability, and
              Transparency pp. 29-44, DOI 10.1145/3715275.3732004, June
              2025, <https://doi.org/10.1145/3715275.3732004>.

   [CCPA]     "California Consumer Privacy Act Regulations", January
              2026, <https://cppa.ca.gov/regulations/pdf/
              ccpa_statute_eff_20260101.pdf>.

   [CO-SB205] "Colorado Senate Bill 24-205", May 2024,
              <https://leg.colorado.gov/sites/default/
              files/2024a_205_signed.pdf>.

   [I-D.bhutton-json-schema-01]
              Wright, A., Andrews, H. H., Hutton, B., and G. Dennis,
              "JSON Schema: A Media Type for Describing JSON Documents",
              Work in Progress, Internet-Draft, draft-bhutton-json-
              schema-01, 10 June 2022,
              <https://datatracker.ietf.org/doc/html/draft-bhutton-json-
              schema-01>.

   [JV]       Tekuri, S. K., "JSONSchema Validation using Go", n.d.,
              <https://github.com/santhosh-tekuri/jsonschema>.

Gillmor & Gerchick        Expires 22 April 2027                 [Page 6]
Internet-Draft                 audits.json                  October 2026

   [NYC-LL144]
              "Rules of the City of New York: Automated Employment
              Decision Tools", April 2023,
              <https://codelibrary.amlegal.com/codes/newyorkcity/latest/
              NYCrules/0-0-0-138391>.

Appendix A.  Test Vectors

A.1.  Example audits.json

Gillmor & Gerchick        Expires 22 April 2027                 [Page 7]
Internet-Draft                 audits.json                  October 2026

   {
     "operations": [
       {
         "urls": [
           "https://example.com/jobs/sysadmin",
           "https://example.com/jobs/clerk"
         ],
         "regs": [
           "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
         ],
         "audits": [ "fizz", "sparkle" ]
       },
       {
         "urls": [
           "https://example.com/jobs/cfo",
           "https://example.com/jobs/cto"
         ],
         "regs": [
           "https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-138391"
         ],
         "audits": [ "fizz" ]
       }
     ],
     "audits": {
       "fizz": {
         "title": "Fizz Resume Prioritization Service",
         "urls": [
           "https://fizz.example/bias_audits/aedt_audit_2024.pdf",
           "https://fizz.example/bias_audits/aedt_2024_data.tgz"
         ],
         "date": "2024-12-13"
       },
       "sparkle": {
         "title": "Sparkle Video Analytics Assessment",
         "urls": [
           "https://sparkle.example/video_analytics_2023_audit.pdf"
         ],
         "date": "2024-01-15"
       }
     }
   }

Appendix B.  JSON Schema For audits.json

   The following JSON Schema (see [I-D.bhutton-json-schema-01]) can be
   used to validate an audits.json summary file.

Gillmor & Gerchick        Expires 22 April 2027                 [Page 8]
Internet-Draft                 audits.json                  October 2026

   {
     "$schema": "https://json-schema.org/draft/2020-12/schema",
     "$id": "http://ietf.org/json_schemas/audit_schema.json",
     "title": "Audit Report Summary",
     "description": "Automated Discovery of Audit Reports",
     "type": "object",
     "properties": {
       "operations": {
         "type": "array",
         "items": {
           "type": "object",
           "properties": {
             "urls": {
               "type": "array",
               "items": {
                 "type": "string",
                 "format": "uri"
               },
               "uniqueItems": true,
               "minItems": 1
             },
             "regs": {
               "type": "array",
               "items": {
                 "type": "string",
                 "format": "uri"
               },
               "uniqueItems": true,
               "minItems": 1
             },
             "audits": {
               "type": "array",
               "items": {
                 "type": "string"
               },
               "description": "property of top-level 'audits' object",
               "uniqueItems": true,
               "minItems": 1
             }
           },
           "required": [
             "urls",
             "regs",
             "audits"
           ]
         }
       },
       "audits": {

Gillmor & Gerchick        Expires 22 April 2027                 [Page 9]
Internet-Draft                 audits.json                  October 2026

         "type": "object",
         "additionalProperties": {
           "type": "object",
           "properties": {
             "date": {
               "type": "string",
               "format": "date"
             },
             "title": {
               "type": "string"
             },
             "urls": {
               "type": "array",
               "items": {
                 "type": "string",
                 "format": "uri"
               },
               "uniqueItems": true,
               "minItems": 1
             }
           },
           "required": [
             "date",
             "title",
             "urls"
           ]
         }
       }
     },
     "required": [
       "operations",
       "audits"
     ]
   }

   For example, you can validate this using with [JV]:

   jv audits-schema.json audits.json

   Note that JSON Schema cannot represent foreign key constraints, so
   the fact that each /operations/*/audits must reference a named member
   of /audits needs to be enforced separately.

Appendix C.  Examples of Regulatory Regimes

   Please propose more pointers for this subsection!

Gillmor & Gerchick        Expires 22 April 2027                [Page 10]
Internet-Draft                 audits.json                  October 2026

   *  [NYC-LL144] established requirements for employers in New York
      City to publish audits of automated decision-making tools used for
      hiring

   *  Colorado's SB 205 [CO-SB205] requires developers of some
      artificial intelligence systems to publish reports about the
      design and deployment of such systems on their websites

   *  The California Consumer Privacy Act [CCPA] requires businesses to
      create "Risk Assessment Reports" about their use of Automated
      Decision-Making Technology (ADMT) as well as pre-use notices about
      ADMT use and regular cybersecurity audits.

C.1.  Example Regulatory Text

   A regulation or law that relies on this mechanism to point toward
   relevant audits can cite it using text similar to the following:

      Any Covered Organization that has a public website shall publicly
      point to the Required Audits associated with each position using a
      widely understood mechanism such as the audits.json specification.
      The Required Audits associated with this act shall refer to the
      act as https://springfield.example/citycouncil/acts/2025/114

   (note that this text presumes that the regulation has defined the
   terms "Covered Organization" and "Required Audits")

Acknowledgements

Document History

   This section is to be removed before publishing as an RFC.

Changes between -00 and -01

   This section is to be removed before publishing as an RFC.

   *  added references to CCPA

   *  align schema test for jv CLI API 0.7

Authors' Addresses

   Daniel Kahn Gillmor
   American Civil Liberties Union
   Email: dkg@fifthhorseman.net

Gillmor & Gerchick        Expires 22 April 2027                [Page 11]
Internet-Draft                 audits.json                  October 2026

   Marissa Kumar Gerchick
   American Civil Liberties Union
   Email: mgerchick@aclu.org

Gillmor & Gerchick        Expires 22 April 2027                [Page 12]