ba64: A Binary-to-Text Encoding That Is Never Larger Than Base64
draft-gaikwad-ba64-00

Network Working Group                                         M. Gaikwad
Internet-Draft                                               Independent
Intended status: Informational                            16 August 2026
Expires: 17 February 2027

    ba64: A Binary-to-Text Encoding That Is Never Larger Than Base64
                         draft-gaikwad-ba64-00

Abstract

   ba64 is a text encoding for binary data that is never larger than
   standard base64.  An encoder races DEFLATE compression against plain
   base64 and emits whichever final text is shorter.  Compressed output
   is marked by a leading "=" character, which is inside the base64
   alphabet, so it survives every base64-safe channel, yet can never
   begin a valid base64 string, so the two forms are unambiguous.  A
   CRC-32 over the decoded bytes guarantees that a ba64 decoder never
   silently returns wrong data.  The plain form is byte-identical to
   base64, so ba64 decoders are a drop-in replacement wherever base64 is
   read today.  An optional padding method decouples the emitted length
   from the compressibility of the input.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 17 February 2027.

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Gaikwad                 Expires 17 February 2027                [Page 1]
Internet-Draft                    ba64                       August 2026

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   3
   3.  Encoded Forms . . . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Frame Layout  . . . . . . . . . . . . . . . . . . . . . . . .   3
     4.1.  decoded_len (unsigned LEB128) . . . . . . . . . . . . . .   4
     4.2.  crc32 . . . . . . . . . . . . . . . . . . . . . . . . . .   4
     4.3.  Method Registry . . . . . . . . . . . . . . . . . . . . .   4
   5.  Method 0x01 - Raw DEFLATE . . . . . . . . . . . . . . . . . .   4
   6.  Method 0x02 - Raw DEFLATE With Padding  . . . . . . . . . . .   5
   7.  Decoding Algorithm  . . . . . . . . . . . . . . . . . . . . .   5
   8.  Error Taxonomy  . . . . . . . . . . . . . . . . . . . . . . .   6
   9.  Encoding Requirements . . . . . . . . . . . . . . . . . . . .   7
   10. The Padding Policy  . . . . . . . . . . . . . . . . . . . . .   8
   11. Limits and Resource Safety  . . . . . . . . . . . . . . . . .   9
   12. Security Considerations . . . . . . . . . . . . . . . . . . .   9
   13. Interoperability and Migration  . . . . . . . . . . . . . . .  10
   14. Non-Goals . . . . . . . . . . . . . . . . . . . . . . . . . .  10
   15. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
   16. Normative References  . . . . . . . . . . . . . . . . . . . .  10
   17. Informative References  . . . . . . . . . . . . . . . . . . .  11
   Appendix A.  Golden Examples  . . . . . . . . . . . . . . . . . .  11
   Appendix B.  Conformance Vectors  . . . . . . . . . . . . . . . .  12
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  12
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  12

1.  Introduction

   Base64 [RFC4648] expands binary data by 33%. Applications that embed
   binary in JSON, cookies, logs, or size-limited message channels pay
   that tax even when the underlying bytes are highly compressible.  Ad
   hoc solutions (gzip-then-base64) fork the wire format and lose base64
   compatibility.

   ba64 addresses this with a single rule: emit compressed output only
   when it is strictly shorter than plain base64.  The result is an
   encoding that is never larger than base64, is byte-identical to
   base64 in the common (incompressible) case, and is unambiguously
   self-describing.

Gaikwad                 Expires 17 February 2027                [Page 2]
Internet-Draft                    ba64                       August 2026

   The emitted length depends on the compressibility of the input, which
   exposes the compression side channel of [CRIME], summarized for TLS
   in [RFC7457], where an adversary controls part of an input carried
   alongside secret data.  Section 6 defines an OPTIONAL padding method
   that decouples the emitted length from compressibility, Section 10
   specifies the policy for selecting a pad, and Section 12 states the
   limits of the mitigation.

   This document specifies version 1 of the ba64 format.  Version 1 is
   intended to be frozen: evolution occurs only through new method
   identifiers assigned by a future revision.

2.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   Throughout, "base64" means canonical base64 as defined in [RFC4648],
   Section 4: the alphabet A-Z a-z 0-9 + /, "=" padding REQUIRED, no
   whitespace, no other characters, and the unused trailing bits of the
   final symbol set to zero.  There is no URL-safe variant of ba64
   (Section 14).

3.  Encoded Forms

   A ba64 text is an ASCII string in exactly one of two forms:

   plain form:       base64(input)        ; byte-identical to base64
   compressed form:  "=" + base64(frame)

   Dispatch is by the first character: "=" selects the compressed form;
   anything else (including the empty string, which decodes to zero
   bytes) selects the plain form.

4.  Frame Layout

   frame = version || method || decoded_len || crc32 || payload

   offset  size       field        value
   0       1 byte     version      0x01
   1       1 byte     method       see below
   2       1-9 bytes  decoded_len  unsigned LEB128
   ...     4 bytes    crc32        CRC-32 of DECODED bytes, LE
   ...     rest       payload      method-specific compressed data

Gaikwad                 Expires 17 February 2027                [Page 3]
Internet-Draft                    ba64                       August 2026

4.1.  decoded_len (unsigned LEB128)

   Little-endian base-128; each byte holds 7 value bits with the high
   bit as continuation.  It MUST be minimally encoded (if longer than
   one byte, the final byte MUST NOT be 0x00), and MUST NOT exceed 9
   bytes (so decoded_len < 2^63).  Violations yield E_HEADER.

4.2.  crc32

   CRC-32/ISO-HDLC, the algorithm of gzip [RFC1952], zlib, and PNG:
   reflected polynomial 0xEDB88320, initial value 0xFFFFFFFF, final XOR
   0xFFFFFFFF, reflected input and output.  Stored little-endian,
   computed over the decoded (original) bytes, so it detects both
   channel corruption and decompressor divergence end-to-end.  Anchors:
   CRC32("123456789") = 0xCBF43926, CRC32("") = 0.

4.3.  Method Registry

        +=============+===========================================+
        | ID          | Meaning                                   |
        +=============+===========================================+
        | 0x00        | Reserved, never valid                     |
        +-------------+-------------------------------------------+
        | 0x01        | DEFLATE, raw (Section 5)                  |
        +-------------+-------------------------------------------+
        | 0x02        | DEFLATE, raw, with padding (Section 6)    |
        +-------------+-------------------------------------------+
        | 0x03 - 0xEF | Reserved for future revisions -> E_METHOD |
        +-------------+-------------------------------------------+
        | 0xF0 - 0xFF | Private use -> E_METHOD unless configured |
        +-------------+-------------------------------------------+

                        Table 1: Method identifiers

5.  Method 0x01 - Raw DEFLATE

   The payload is exactly one complete raw DEFLATE stream per [RFC1951],
   with no zlib or gzip wrapper.  Normative outcomes:

   *  Inflater produces MORE than decoded_len bytes: abort at the cap,
      E_LENGTH_MISMATCH.

   *  Stream completes with FEWER than decoded_len bytes:
      E_LENGTH_MISMATCH.

   *  Payload exhausted before the final block completes, or malformed:
      E_PAYLOAD.

Gaikwad                 Expires 17 February 2027                [Page 4]
Internet-Draft                    ba64                       August 2026

   *  Bytes remain after the final block: E_PAYLOAD.

6.  Method 0x02 - Raw DEFLATE With Padding

   Method 0x02 is method 0x01 with one additional header field and a
   trailing pad.  It allows an encoder to select the emitted text length
   independently of the compressibility of the input (Section 10,
   Section 12).

   frame = version || 0x02 || decoded_len || crc32
                          || pad_len || payload || pad

   *  pad_len is an unsigned LEB128 under the same constraints as
      decoded_len: minimally encoded, at most 9 bytes; violations yield
      E_HEADER.

   *  pad is exactly pad_len bytes at the end of the frame.  If fewer
      than pad_len bytes remain after pad_len itself, the result is
      E_TRUNCATED.

   *  payload is everything between pad_len and the pad: one complete
      raw DEFLATE stream, judged by the Section 5 rules.  The pad is
      removed BEFORE inflating and is never fed to the inflater.

   *  Encoders MUST write zero bytes for the pad.  Decoders MUST NOT
      inspect the pad's content and MUST NOT reject a frame because of
      it.  Pad content is therefore not integrity-protected: the CRC
      covers the decoded bytes only.

   *  pad_len = 0 is valid, and is grammatically distinct from a method-
      0x01 frame carrying the same payload.

   A padded frame occupies the length its encoder selected.  Padding
   therefore trades the compression saving for length uniformity.

7.  Decoding Algorithm

   Input: string s, caller limit max_decoded_len (default 67,108,864).
   Checks MUST run in this order, so any invalid input yields one
   deterministic code:

Gaikwad                 Expires 17 February 2027                [Page 5]
Internet-Draft                    ba64                       August 2026

   1. s does not start with "=": canonical-base64-decode s
        (fail -> E_BASE64); return the bytes ("" -> empty bytes).
   2. canonical-base64-decode s[1:]                fail -> E_BASE64
   3. read version (1 byte)  missing -> E_TRUNCATED; !=0x01 -> E_VERSION
   4. read method  (1 byte)  missing -> E_TRUNCATED; unknown -> E_METHOD
   5. read decoded_len (LEB128)
        exhausted -> E_TRUNCATED; non-minimal or >9 bytes -> E_HEADER
   6. decoded_len > max_decoded_len  -> E_LIMIT_EXCEEDED
        (MUST precede any allocation proportional to decoded_len)
   7. read crc32 (4 bytes)    missing -> E_TRUNCATED
  7b. method 0x02 only: read pad_len (LEB128)
        exhausted -> E_TRUNCATED; non-minimal or >9 bytes -> E_HEADER
        fewer than pad_len bytes remain -> E_TRUNCATED
        trim the trailing pad_len bytes; the payload is what is left
   8. inflate payload with hard output cap = decoded_len
        (a cap of zero still caps)  -> E_PAYLOAD / E_LENGTH_MISMATCH
   9. CRC32(output) != crc32  -> E_CHECKSUM
  10. return output

   E_TRUNCATED refers only to the fixed header fields (steps 3-7b);
   truncation inside the DEFLATE stream is E_PAYLOAD.  A decoder MUST
   NOT reject a frame merely because a conforming encoder would have
   chosen the plain form: decoders validate the grammar, not encoder
   optimality.

8.  Error Taxonomy

   Conformance means the exact code, not merely "an error."

Gaikwad                 Expires 17 February 2027                [Page 6]
Internet-Draft                    ba64                       August 2026

    +===================+=============================================+
    | Code              | Fires when                                  |
    +===================+=============================================+
    | E_BASE64          | input or frame body is not canonical base64 |
    +-------------------+---------------------------------------------+
    | E_TRUNCATED       | frame ends inside the header, or is shorter |
    |                   | than its pad                                |
    +-------------------+---------------------------------------------+
    | E_HEADER          | LEB128 non-minimal or longer than 9 bytes   |
    +-------------------+---------------------------------------------+
    | E_VERSION         | version byte != 0x01                        |
    +-------------------+---------------------------------------------+
    | E_METHOD          | method unknown, reserved, or unconfigured   |
    +-------------------+---------------------------------------------+
    | E_LIMIT_EXCEEDED  | claimed decoded_len exceeds the caller's    |
    |                   | limit                                       |
    +-------------------+---------------------------------------------+
    | E_PAYLOAD         | DEFLATE stream malformed, short, or has     |
    |                   | trailing bytes                              |
    +-------------------+---------------------------------------------+
    | E_LENGTH_MISMATCH | inflated size != decoded_len                |
    +-------------------+---------------------------------------------+
    | E_CHECKSUM        | CRC-32 of output != stored value            |
    +-------------------+---------------------------------------------+

                            Table 2: Error codes

   Implementation note: the boundary between E_PAYLOAD and
   E_LENGTH_MISMATCH for a truncated DEFLATE stream depends on the
   underlying inflate library (some treat premature end of input as end
   of stream).  Both codes reject the frame and the CRC still prevents
   silently-wrong output; callers SHOULD treat the two as equivalent
   "corrupt frame" outcomes.

9.  Encoding Requirements

   1.  Output MUST be one of the two forms, using canonical base64
       throughout.

   2.  Floor rule: the compressed form MUST be emitted only if its final
       text length is strictly less than the plain form's; ties take
       plain.  Consequently len(ba64(x)) <= len(base64(x)) for every
       input.

   3.  decoded_len MUST equal the true input length, minimally encoded;
       crc32 MUST be over the input bytes.

Gaikwad                 Expires 17 February 2027                [Page 7]
Internet-Draft                    ba64                       August 2026

   4.  An encoder MAY skip compression entirely.  An encoder that always
       emits plain base64 is degenerate but fully conforming.

   5.  DEFLATE level is the encoder's choice (SHOULD default to 6).
       Encoded output is therefore NOT canonical: the same input may
       yield different valid texts.  Any equality, deduplication, cache-
       keying, or MAC comparison MUST operate on the decoded bytes,
       never on ba64 text.

   6.  Padding is OPTIONAL: an encoder MAY emit method 0x02
       (Section 10).  The floor rule still binds.  A decoder MUST accept
       method 0x02 whether or not the encoder ever produces it.

10.  The Padding Policy

   Let n be the input length and P = 4*ceil(n/3) the plain-form length.
   A compressed text is 1 + 4k characters and is therefore congruent to
   1 modulo 4, while P is congruent to 0 modulo 4.  A compressed text
   can never equal the plain-form length, and the greatest length the
   floor rule permits is P - 3.

   The RECOMMENDED policy takes one parameter, a quantum Q, a positive
   multiple of 4 in characters:

   1.  Build the method-0x02 frame with pad_len = 0; call its text
       length L0.

   2.  Target T = Q*ceil((L0 - 1)/Q) + 1, clamped down to P - 3.

   3.  If T < L0 the compressed form cannot fit under the floor: emit
       the plain form.

   4.  Otherwise pad the frame to F = 3*(T - 1)/4 bytes.  Frame lengths
       F, F - 1 and F - 2 all encode to T characters, so an encoder that
       cannot hit F exactly (the pad_len varint changes size as the pad
       grows) uses whichever of the three it can reach.

   Q = 4 pads by at most three bytes.  A quantum greater than or equal
   to P yields P - 3 characters for every input that fits under the
   floor rule, so the emitted length is then a function of n alone.

   One bit remains observable at every quantum: whether the input
   compressed enough to fit under the floor rule.  An encoder that must
   not leak that bit uses the always-plain mode, whose length is a
   function of n alone.

Gaikwad                 Expires 17 February 2027                [Page 8]
Internet-Draft                    ba64                       August 2026

11.  Limits and Resource Safety

   Decoders MUST enforce max_decoded_len before any allocation
   proportional to the claimed size (step 6) and MUST cap inflation at
   decoded_len (step 8).  A cap of zero MUST still cap: some inflate
   APIs treat an output limit of 0 as unlimited, so a frame claiming
   decoded_len = 0 whose payload hides megabytes MUST fail after
   producing at most one byte.  A 100-byte frame claiming 8 GiB costs
   O(header) work to reject.

12.  Security Considerations

   Integrity is not authenticity: CRC-32 detects accidental corruption,
   not tampering.  Where malice is in scope, verify a MAC or signature
   over the decoded bytes at the application layer.

   Compression side channel (CRIME/BREACH class, [CRIME], [RFC7457]):
   where the emitted length is observable to an adversary who controls
   part of an input carried alongside secret data, compression leaks the
   secret through the length.  Implementations MUST NOT ba64-compress
   attacker-influenced data concatenated with secret data in such
   settings; the always-plain encoder mode applies there, and its
   length, 4*ceil(n/3), is a function of the input length alone.

   Where compression cannot be disabled, the padding of Section 6 and
   Section 10 quantizes the emitted length.  At quantum Q, a one-byte
   change in compressibility is unobservable unless it crosses a bucket
   boundary; at Q >= P all inputs of a given length that fit under the
   floor rule emit P - 3 characters.  Padding increases the number of
   observations an attack requires.  It does not remove the channel:
   bucket boundaries are still crossed under repeated queries, and the
   bit identified in Section 10 still leaks.  Padding is therefore a
   secondary control and not a substitute for separating attacker-
   influenced data from secret data.

   The pad is not covered by the CRC-32, which is computed over the
   decoded bytes.  An attacker able to rewrite the text can rewrite the
   pad undetected, as with any other part of an unauthenticated message.
   The decoded output is unaffected.

   Decompression bombs are handled structurally by the limit and the
   hard inflate cap.  Decoder error messages MUST NOT echo payload or
   decoded content by default: inputs may be secrets and errors end up
   in logs; the code and an offset suffice.

Gaikwad                 Expires 17 February 2027                [Page 9]
Internet-Draft                    ba64                       August 2026

   Canonical base64 plus minimal LEB128 plus the ordered checks of
   Section 7 ensure every ba64 text has exactly one interpretation
   (specific bytes or one specific error).  There is no decode-side
   malleability.

13.  Interoperability and Migration

   Compatibility is one-directional: a ba64 decoder reads every legacy
   base64 string unchanged (the plain form is base64); a plain base64
   decoder cannot read the compressed form.  Therefore deploy ba64
   decoders everywhere first (drop-in, no behavior change), enable
   compressing encoders per channel only once every reader decodes ba64,
   and note that reverting encoders to the always-plain mode restores
   universal readability instantly.

   All ba64 characters are safe verbatim in JSON strings and cookie
   values.  In URL components, "+", "/", and "=" MUST be percent-
   encoded.  Transport artifacts (MIME line-wrapping, trailing newlines)
   MUST be stripped by the caller before decoding.

14.  Non-Goals

   No URL-safe or alternative alphabets, no streaming, no compression
   negotiation, no dictionary registry, no public methods beyond the two
   defined here.  No version 2 is planned; the version byte is an escape
   hatch, not a roadmap.

15.  IANA Considerations

   This document has no IANA actions.  The method registry in this
   document is maintained by revisions of this document, not by IANA.

16.  Normative References

   [RFC1951]  Deutsch, P., "DEFLATE Compressed Data Format Specification
              version 1.3", RFC 1951, DOI 10.17487/RFC1951, May 1996,
              <https://www.rfc-editor.org/info/rfc1951>.

   [RFC1952]  Deutsch, P., "GZIP file format specification version 4.3",
              RFC 1952, DOI 10.17487/RFC1952, May 1996,
              <https://www.rfc-editor.org/info/rfc1952>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

Gaikwad                 Expires 17 February 2027               [Page 10]
Internet-Draft                    ba64                       August 2026

   [RFC4648]  Josefsson, S., "The Base16, Base32, and Base64 Data
              Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006,
              <https://www.rfc-editor.org/info/rfc4648>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

17.  Informative References

   [CRIME]    Rizzo, J. and T. Duong, "The CRIME Attack", ekoparty
              Security Conference, September 2012.

   [OSSFUZZ]  Google, "OSS-Fuzz: Continuous Fuzzing for Open Source
              Software", 2016, <https://github.com/google/oss-fuzz>.

   [RFC7457]  Sheffer, Y., Holz, R., and P. Saint-Andre, "Summarizing
              Known Attacks on Transport Layer Security (TLS) and
              Datagram TLS (DTLS)", RFC 7457, DOI 10.17487/RFC7457,
              February 2015, <https://www.rfc-editor.org/info/rfc7457>.

Appendix A.  Golden Examples

   Every conforming decoder MUST reproduce these exactly.

   ""                              -> ""     (empty plain form)
   "SGVsbG8sIHdvcmxkIQ=="          -> "Hello, world!" (13 bytes)
   "=AQECDg4XTQECAP3/SGk="         -> "Hi"   (stored block)
   "=AQEAAAAAAAEAAP//"             -> ""     (decoded_len 0)
   "=AQICDg4XTQABAgD9/0hp"         -> "Hi"   (0x02, pad_len 0)
   "=AQICDg4XTQMBAgD9/0hpAAAA"     -> "Hi"   (0x02, pad_len 3)
   "=AQICDg4XTQQBAgD9/0hp/wDerQ==" -> "Hi"   (0x02, nonzero pad)

   Error-direction anchors:

  "="            -> E_TRUNCATED          "=="        -> E_BASE64
  "SGVsbG8"      -> E_BASE64 (padding)   "QR==" -> E_BASE64 (bits)
  "SGVs bG8="    -> E_BASE64 (space)     version=2   -> E_VERSION
  varint 80 00   -> E_HEADER             claimed 1TB -> E_LIMIT_EXCEEDED
  pad_len past end of frame -> E_TRUNCATED
  "=AQEAAAAAAAEBAP7/WA==" -> E_LENGTH_MISMATCH (zero-cap trap)

Gaikwad                 Expires 17 February 2027               [Page 11]
Internet-Draft                    ba64                       August 2026

Appendix B.  Conformance Vectors

   The normative conformance corpus is a set of machine-readable JSON
   vector files (decode passthrough, valid frames of both methods, error
   cases with required codes, encoder invariants, resource-exhaustion
   frames, and a large differential set).  They are the executable form
   of this specification: an implementation conforms if and only if it
   reproduces every decode vector's bytes exactly and raises every error
   vector's exact code.  Six independent implementations (Python,
   TypeScript, Go, Rust, Java, C#) are maintained against this corpus
   and agree pairwise over the full differential set.  The corpus is
   published alongside this document.

Acknowledgments

   The format's testing discipline follows the SQLite and zlib approach:
   many more lines of tests and vectors than of codec, continuous
   fuzzing ([OSSFUZZ]), and every historical finding frozen into the
   corpus.

Author's Address

   Madhava Gaikwad
   Independent
   Email: gaikwad.madhav@gmail.com

Gaikwad                 Expires 17 February 2027               [Page 12]