JMAP                                                    B. Gondwana, Ed.
Internet-Draft                                                  Fastmail
Updates: 8620 (if approved)                              16 October 2021
Intended status: Standards Track
Expires: 19 April 2022


                     JMAP Blob management extension
                        draft-ietf-jmap-blob-04

Abstract

   The JMAP base protocol (RFC8620) provides the ability to upload and
   download arbitrary binary data via HTTP POST and GET on defined
   endpoint.  This binary data is called a "Blob".

   This extension adds additional ways to create and access Blobs, by
   making inline method calls within a standard JMAP request.

   This extension also adds a reverse lookup mechanism to discover where
   blobs are referenced within other data types.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 19 April 2022.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights



Gondwana                  Expires 19 April 2022                 [Page 1]


Internet-Draft                  JMAP Blob                   October 2021


   and restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License text
   as described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions Used In This Document . . . . . . . . . . . . . .   3
   3.  Addition to the Capabilities Object . . . . . . . . . . . . .   3
     3.1.  urn:ietf:params:jmap:blob . . . . . . . . . . . . . . . .   3
   4.  Blob Methods  . . . . . . . . . . . . . . . . . . . . . . . .   4
     4.1.  Blob/set  . . . . . . . . . . . . . . . . . . . . . . . .   4
       4.1.1.  create  . . . . . . . . . . . . . . . . . . . . . . .   4
       4.1.2.  update  . . . . . . . . . . . . . . . . . . . . . . .   6
       4.1.3.  destroy . . . . . . . . . . . . . . . . . . . . . . .   6
     4.2.  Blob/get  . . . . . . . . . . . . . . . . . . . . . . . .   7
     4.3.  Blob/lookup . . . . . . . . . . . . . . . . . . . . . . .   9
   5.  Security considerations . . . . . . . . . . . . . . . . . . .  10
   6.  IANA considerations . . . . . . . . . . . . . . . . . . . . .  11
     6.1.  JMAP Capability registration for "blob" . . . . . . . . .  11
     6.2.  JMAP Error Codes Registration for "unknownDataType" . . .  11
     6.3.  Creation of "JMAP Data Types" Registry  . . . . . . . . .  12
   7.  Changes . . . . . . . . . . . . . . . . . . . . . . . . . . .  12
   8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  14
   9.  Normative References  . . . . . . . . . . . . . . . . . . . .  14
   10. Informative References  . . . . . . . . . . . . . . . . . . .  14
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  14

1.  Introduction

   Sometimes JMAP ([RFC8620]) interactions require creating a Blob and
   then referencing it.  In the same way that IMAP Literals ([RFC7888])
   were extended to reduce roundtrips for simple data, embedding simple
   small blobs into the JMAP method stream can reduce roundtrips.

   Likewise, when fetching an object, it can be useful to also fetch the
   raw content of that object without a separate roundtrip.

   Where JMAP is being proxied through a system which applies additional
   access restrictions, it can be useful to be able to see where a blob
   is referenced in order to decide whether to allow it to be
   downloaded.








Gondwana                  Expires 19 April 2022                 [Page 2]


Internet-Draft                  JMAP Blob                   October 2021


2.  Conventions Used In This Document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Addition to the Capabilities Object

   The capabilities object is returned as part of the JMAP Session
   object; see [RFC8620], Section 2.

   This document defines an additional capability URI.

3.1.  urn:ietf:params:jmap:blob

   This represents support for additional API methods on the Blob
   datatype.

   The value of this property in the JMAP session "capabilities"
   property is an empty object.

   The value of this property in an account's "accountCapabilities"
   property is an object that MAY contain the following information on
   server capabilities and permissions for that account:

   *  "maxSizeBlobSet": UnsignedInt - if set, gives the maximum size of
      a blob that the server will allow you to create (including with
      catenate).  This SHOULD be the same as the RFC8620 value
      maxSizeUpload.

   *  "maxCatenateItems": UnsignedInt - if set, gives the maximum number
      of of CatenateSourceObjects allowed per creation in a Blob/set.
      Servers SHOULD allow at least 64 items.

   *  "supportedTypeNames": [String] - an array of data type names that
      are supported for Blob/lookup.

   Example:











Gondwana                  Expires 19 April 2022                 [Page 3]


Internet-Draft                  JMAP Blob                   October 2021


   {
     "capabilities": {
       ...,
       "urn:ietf:params:jmap:blob": {}
     },
     "accounts": {
       "A13842": {
         ...
         "accountCapabilities": {
           "urn:ietf:params:jmap:blob": {
             "maxSizeBlobSet": 50000000,
             "maxCatenateItems": 100,
             "supportedTypeNames" : [
               "Mailbox",
               "Thread",
               "Email"
             ]
           }
         }
       }
     }
   }

4.  Blob Methods

   A blob is a sequence of zero or more octets.

   The JMAP base spec [RFC8620] defines the Blob/copy method, which is
   unchanged by this specfication, and is selected by the
   urn:ietf:params:jmap:core capability.

   The following JMAP Methods are selected by the
   urn:ietf:params:jmap:blob capability.

4.1.  Blob/set

   This is a standard JMAP set method.

4.1.1.  create

   *Properties:*

   Exactly one of:

   *  data:asText: String|null

   *  data:asBase64: String|null




Gondwana                  Expires 19 April 2022                 [Page 4]


Internet-Draft                  JMAP Blob                   October 2021


   *  data:asHex: String|null

   *  catenate: [CatenateSourceObject] _list of one or more octet
      sources in order_

   Also:

   *  type: String|null _hint for media type of the data_

   Result is: * id: Id _the blobId_ * type: String|null _media type as
   given in the creation (if any); or detected from content; or null_ *
   size: UnsignedInt _as per RFC8620 - the size of the blob in Octets_

   Any other properties identical to those that would be returned in the
   JSON response of the RFC8620 upload endpoint.

   CatenateSourceObject:

   Exactly one of:

   *  data:asText: String|null

   *  data:asBase64: String|null

   *  data:asHex: String|null

   or a blobId source:

   *  blobId: Id

   *  offset: UnsignedInt|null _may be zero_

   *  length: UnsignedInt|null _must not be zero_

   If null then offset is assumed to be zero.  If null then length is
   the remaining octets in the blob.  If the range can not be fully
   satisfied (i.e. extends past the end of the data in the blob) then
   the catenate itself is invalid and results in a notCreated response
   for this creation id.

   If the data properties or catenate properties have any invalid
   references or invalid data contained in them, the server MUST NOT
   guess as to the user's intent, and MUST reject the creation and
   return a notCreated response for that creation id.

   Likewise, invalid data in the base64 or hex fields is an error and
   must result in a notCreated response for this creation id.




Gondwana                  Expires 19 April 2022                 [Page 5]


Internet-Draft                  JMAP Blob                   October 2021


   It is legal to create a blob by calling catenate with a single
   CatenateSourceObject.  Please note that a catenate source can not
   contain additional sub-catenates, only data or blob sources.

   A server SHOULD accept at least 64 catenate items.

4.1.2.  update

   It is not possible to update a Blob, so any update will result in a
   notUpdated response.

4.1.3.  destroy

   If an uploaded Blob is not referenced by any persistent object, the
   server SHOULD destroy the object.  Some systems use a content-based
   ID for blobs, so the server MAY respond destroyed and yet that blobId
   still exist with the same content.

   Example:

   Method Call:

   [ "Blob/set", {
     "accountId" : "account1",
     "create" : {
       "1": {
         "data:asBase64": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKA
                           AAAA1BMVEX/AAAZ4gk3AAAAAXRSTlN/gFy0ywAAAApJRE
                           FUeJxjYgAAAAYAAzY3fKgAAAAASUVORK5CYII=",
         "type" : "image/png"
       },
     },
   }, "R1" ]

   Response:

   [ "Blob/set", {
     "accountId" : "account1",
     "created" : {
       "1": {
         "id" : "G4c6751edf9dd6903ff54b792e432fba781271beb",
         "type" : "image/png",
         "size" : 95
       },
     },
   }, "R1" ]





Gondwana                  Expires 19 April 2022                 [Page 6]


Internet-Draft                  JMAP Blob                   October 2021


4.2.  Blob/get

   A standard JMAP get with two additional parameters:

   *  offset: UnsignedInt|null _start this many octets into the blob
      data_

   *  length: UnsignedInt|null _return at most this many octets of the
      blob data_

   *Request Properties:*

   Any of

   *  data:asText

   *  data:asBase64

   *  data:asHex

   *  data _selects data:asText if the content is valid UTF-8, or
      data:asBase64_

   *  size

   If not given, properties defaults to data and size.

   *Result Properties:*

   *  data:asText: String|null _the raw octets of the selected range if
      they are valid utf-8, otherwise null_

   *  data:asBase64: String _the base64 encoding of the selected range_

   *  data:asHex: String _the octets of the selected range encoded as
      lowercase hexadecimal (0-9, a-f)_

   *  isEncodingProblem: Boolean _(default: false)_

   *  isTruncated: Boolean _(default: false)_

   *  size: UnsignedInt _the number of octets in the entire blob,
      regardless of offset/length selectors_

   The size value is always the number of octets in the entire blob,
   regardless of offset and length.





Gondwana                  Expires 19 April 2022                 [Page 7]


Internet-Draft                  JMAP Blob                   October 2021


   The data fields contain a representation of the octets within the
   selected range that are present in the blob.  If the octets selected
   are not valid utf-8 (including truncating in the middle of a multi-
   octet sequence) and data:asText or data is selected, then the key
   isEncodingProblem is set to true and the data:asText value is null.
   In the case where data was requested, this also causes data:asBase64
   to be returned.

   If the selected range requests data outside the blob (i.e. the
   offset+length is larger than the blob) then the result is either just
   the octets from the offset to the end of the blob, or an empty string
   if the offset is past the end of the blob.  Either way, the
   isTruncated property in the result is set to true to tell the client
   that the requested range could not be fully satisfied.

   Example (a blob containing the string "The quick brown fox jumped
   over the lazy dog!")

Method Call:

[ "Blob/get", {
  "accountId" : "account1",
  "ids" : [ "G6ec94756e3e046be78fcb33953b85b944e70673e", "not-a-blob" ],
  "properties" : [ "data:asText", "data:asBase64",
                   "data:asHex", "size" ],
  "offset" : 4,
  "length" : 9
}, "R1" ]

Response:

[ "Blob/get", {
  "accountId" : "account1",
  "list" : [
    {
      "id" : "G6ec94756e3e046be78fcb33953b85b944e70673e",
      "data:asText" : "quick bro",
      "data:asBase64" : "cXVpY2sgYnJvCg==",
      "data:asHex" : "717569636b2062726f",
      "size" : 46
    }
  ],
  "notFound" : [ "not-a-blob" ]
}, "R1" ]







Gondwana                  Expires 19 April 2022                 [Page 8]


Internet-Draft                  JMAP Blob                   October 2021


4.3.  Blob/lookup

   Given a list of blobIds, this method does a reverse lookup in each of
   the provided type names to find the list of Ids within that data type
   which reference the provided blob.

   The definition of reference is somewhat loosely defined, but roughly
   means "you could discover this blobId by looking inside this object",
   for example if a Mailbox contains an Email which references the
   blobId, then it references that blobId.  Likewise for a Thread.

   *Parameters*

   *  accountId: "Id"

   The id of the account used for the call.

   *  types: [String]

   A list of type names from the "JMAP Data Types" registry.  Only names
   for which "Can Reference Blobs" is true may be specified, and the
   capability which defines each type must also be used by the overall
   JMAP request in which this method is called.

   If a type name is not known by the server, or the associated
   capability has not been requested, then the server returns an
   "unknownDataType" error.

   *  ids: [Id]

   A list of blobId values to be looked for.

   *Response*

   *  list: [BlobInfo]

   A list of BlobInfo objects.

   *BlobInfo Object*

   *  id: Id

   The Blob Identifier.

   *  types: String[Id List]

   A map from type name to list of Ids of that data type (e.g. the name
   "Email" maps to a list of emailIds)



Gondwana                  Expires 19 April 2022                 [Page 9]


Internet-Draft                  JMAP Blob                   October 2021


   If a blob is not visible to a user at all, then the server SHOULD
   return that blobId in the notFound array, however it may also return
   an empty list for each type name, as it may not be able to know if
   other data types do reference that blob.

   e.g.

   [ "Blob/lookup", {
     "types": ["Mailbox", "Thread", "Email"],
     "ids": ["Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
             "not-a-blob"]
   }, "R1" ]

   Response:

   [ "Blob/lookup", {
     "list": [
       {
         "id": "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
         "types": {
           "Mailbox": ["M54e97373", Mcbe6b662"],
           "Thread": ["T1530616e"],
           "Email": ["E16e70a73eb4", "E84b0930cf16"]
         }
       }
     ],
     "notFound": ["not-a-blob"]
   }, "R1"]

5.  Security considerations

   JSON parsers are not all consistent in handling non-UTF-8 data.  JMAP
   requires that all JSON data be UTF-8 encoded, so servers MUST either
   return data:asBase64 or isEncodingProblem: true and modify the data
   to be UTF-8 safe.

   Servers MUST apply any access controls such that if the authenticated
   user would be unable to discover the blobId by making queries, then
   this fact can't be discovered via a Blob/lookup.  For example, if an
   Email exists in a Mailbox which the authenticated user does not have
   access to see, then that emailId MUST not be returned in a lookup for
   a blob which is referenced by that email.









Gondwana                  Expires 19 April 2022                [Page 10]


Internet-Draft                  JMAP Blob                   October 2021


   If a server might sometimes return all names empty rather than
   putting a blobId in the notFound response to a Blob/get, then the
   server SHOULD always return the same type of response, regardless of
   whether a blob exists but the user can't access it, or doesn't exist
   at all.  This avoids leaking information about the existence of the
   blob.

   The server MUST NOT trust that the data given to a Blob/set is a well
   formed instance of the specified media type, and if the server
   attempts to parse the given blob, only hardened parsers designed to
   deal with arbitrary untrusted data should be used.  The server SHOULD
   NOT reject data on the grounds that it is not a valid specimen of the
   stated type.

   Blob/set catenate can be used to recreate dangerous content on the
   far side of security scanners (anti-virus or exfiltration scanners
   for example) which may be watching the upload endpoint.  Server
   implementations SHOULD provide a hook to allow security scanners to
   check the resulting blobId from a catenate in the same way that they
   do for the upload endpoint.

6.  IANA considerations

6.1.  JMAP Capability registration for "blob"

   IANA is requested to register the "blob" JMAP Capability as follows:

   Capability Name: urn:ietf:params:jmap:blob

   Specification document: this document

   Intended use: common

   Change Controller: IETF

   Security and privacy considerations: this document, Section XXX

6.2.  JMAP Error Codes Registration for "unknownDataType"

   IANA is requested to register the "unknownDataType" JMAP Error Code
   as follows:

   JMAP Error Code: unknownDataType

   Intended use: common

   Change Controller: IETF




Gondwana                  Expires 19 April 2022                [Page 11]


Internet-Draft                  JMAP Blob                   October 2021


   Reference: this document

   Description: The server does not recognise this data type, or the
   capability to enable it was not present.

6.3.  Creation of "JMAP Data Types" Registry

   IANA is requested to create a new registry "JMAP Data Types" with the
   initial content:

   +================+=========+======+=====================================+=========+
   |Type Name       |Can      |Can   |Capability                           |Reference|
   |                |Reference|use   |                                     |         |
   |                |Blobs    |for   |                                     |         |
   |                |         |State |                                     |         |
   |                |         |Change|                                     |         |
   +================+=========+======+=====================================+=========+
   |Core            |No       |No    |urn:ietf:params:jmap:core            |[RFC8620]|
   +----------------+---------+------+-------------------------------------+---------+
   |PushSubscription|No       |No    |urn:ietf:params:jmap:core            |[RFC8620]|
   +----------------+---------+------+-------------------------------------+---------+
   |Mailbox         |Yes      |Yes   |urn:ietf:params:jmap:mail            |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |Thread          |Yes      |Yes   |urn:ietf:params:jmap:mail            |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |Email           |Yes      |Yes   |urn:ietf:params:jmap:mail            |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |EmailDelivery   |No       |Yes   |urn:ietf:params:jmap:mail            |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |SearchSnippet   |No       |No    |urn:ietf:params:jmap:mail            |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |Identity        |No       |Yes   |urn:ietf:params:jmap:submission      |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |EmailSubmission |No       |Yes   |urn:ietf:params:jmap:submission      |[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |VacationResponse|No       |Yes   |urn:ietf:params:jmap:vacationresponse|[RFC8621]|
   +----------------+---------+------+-------------------------------------+---------+
   |MDN             |No       |No    |urn:ietf:params:jmap:mdn             |[RFC9007]|
   +----------------+---------+------+-------------------------------------+---------+

                                  Table 1

7.  Changes

   EDITOR: please remove this section before publication.






Gondwana                  Expires 19 April 2022                [Page 12]


Internet-Draft                  JMAP Blob                   October 2021


   The source of this document exists on github at:
   https://github.com/brong/draft-gondwana-jmap-blob/
   (https://github.com/brong/draft-gondwana-jmap-blob/)

   *draft-ieft-jmap-blob-04*:

   *  added security considerations for scanning catenate results

   *draft-ieft-jmap-blob-03*:

   *  added capabilities object

   *  renamed types to typeNames and matchedIds

   *  added details of how to handle non-UTF8 data and truncation in
      Blob/get

   *  added isTruncated and isEncodingProblem to Blob/get to tell the
      client if the request wasn't entirely satisfied.

   *draft-ieft-jmap-blob-02*:

   *  fixed incorrect RFC number in reference and HTTP PUT -> POST,
      thanks Ken.

   *  added acknowledgements section

   *  removed all 'datatype' text and changed to 'data type' or 'type
      name' as appropriate (issue #1 proposal)

   *  expanded security considerations section and moved optional Blob/
      lookup empty case into Blob/lookup section

   *draft-ieft-jmap-blob-01*:

   *  renamed 'datatypes' to 'types' to align with PushSubscription from
      RFC8620.

   *  added example for Blob/get

   *  specified offset and length precisely

   *draft-ieft-jmap-blob-00*:

   *  initial adoption as an IETF document, otherwise identical to
      draft-gondwana-jmap-blob-02

   *draft-gondwana-jmap-blob-02*



Gondwana                  Expires 19 April 2022                [Page 13]


Internet-Draft                  JMAP Blob                   October 2021


   *  renamed 'objects' to 'datatypes'

   *  specified Blob/lookup

   *  added IANA registry for datatypes

   *draft-gondwana-jmap-blob-01*

   *  added an example

   *draft-gondwana-jmap-blob-00*

   *  initial proposal

8.  Acknowledgements

   Alexey Melnikov, Ken Murchison, and the JMAP working group at the
   IETF.

9.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

10.  Informative References

   [RFC7888]  Melnikov, A., Ed., "IMAP4 Non-synchronizing Literals",
              RFC 7888, DOI 10.17487/RFC7888, May 2016,
              <https://www.rfc-editor.org/info/rfc7888>.

   [RFC8621]  Jenkins, N. and C. Newman, "The JSON Meta Application
              Protocol (JMAP) for Mail", RFC 8621, DOI 10.17487/RFC8621,
              August 2019, <https://www.rfc-editor.org/info/rfc8621>.

   [RFC8620]  Jenkins, N. and C. Newman, "The JSON Meta Application
              Protocol (JMAP)", RFC 8620, DOI 10.17487/RFC8620, July
              2019, <https://www.rfc-editor.org/info/rfc8620>.

Author's Address






Gondwana                  Expires 19 April 2022                [Page 14]


Internet-Draft                  JMAP Blob                   October 2021


   Bron Gondwana (editor)
   Fastmail
   Level 2, 114 William St
   Melbourne  VIC 3000
   Australia

   Email: brong@fastmailteam.com
   URI:   https://www.fastmail.com











































Gondwana                  Expires 19 April 2022                [Page 15]