JMAP Blob management extension

3.1. urn:ietf:params:jmap:blob

The capability urn:ietf:params:jmap:blob being present in the "accountCapabilities" property of an account represents support for additional API methods on the Blob datatype. Servers that include the capability in one or more "accountCapabilities" properties MUST also include the property in the "capabilities" property.¶

The value of this property in the JMAP session "capabilities" property MUST be an empty object.¶

The value of this property in an account's "accountCapabilities" property is an object that MUST contain the following information on server capabilities and permissions for that account:¶

• maxSizeBlobSet: UnsignedInt|null¶

  This is the maximum size of blob (in octets) that the server will allow to be created (including blobs created by concatenating multiple data sources together).¶

  Clients MUST NOT attempt to create blobs larger than this size.¶

  If this value is null, then clients are not required to limit the size of blob they try to create, though servers can always reject creation of blobs regardless of size; e.g. due to lack of disk space, or per-user rate limits.¶

• maxDataSources: UnsignedInt¶

  The maximum number of DataSourceObjects allowed per creation in a Blob/upload.¶

  Servers MUST allow at least 64 DataSourceObjects per creation.¶

• supportedTypeNames: String[]¶

  An array of data type names that are supported for Blob/lookup. If the server does not support lookups then this will be the empty list.¶

  NOTE, the supportedTypeNames list may include private types which are not in the JMAP Types Registry defined by this document. Clients MUST ignore type names they do not recognise.¶

• supportedDigestAlgorithms: String[]¶

  An array of supported digest algorithms that are supported for Blob/get. If the server does not support calculating blob digests, then this will be the empty list. Algorithms in this list MUST be present in the HTTP Digest Algorithms registry defined by [RFC3230], and are always lowercased.¶

  Clients SHOULD prefer algorithms listed earlier in this list.¶

{
  "capabilities": {
    ...,
    "urn:ietf:params:jmap:blob": {}
  },
  "accounts": {
    "A13842": {
      ...
      "accountCapabilities": {
        "urn:ietf:params:jmap:blob": {
          "maxSizeBlobSet": 50000000,
          "maxDataSources": 100,
          "supportedTypeNames" : [
            "Mailbox",
            "Thread",
            "Email"
          ],
          "supportedDigestAlgorithms" : [
            "sha",
            "sha-256"
          ]
        }
      }
    }
  }
}

4.1. Blob/upload

This is similar to a Foo/set from [RFC8620] in some ways, however blobs can not be updated or deleted, so only create is allowed in the method call, and blobs do not have state, so there is no state field present in the method response.¶

Parameters¶

• accountId: Id¶

  The id of the account in which the blobs will be created.¶

• create: Id[UploadObject]¶

  A map of creation id to UploadObjects.¶

Result¶

The result is the same as for Foo/set in RFC8620, with created and notCreated objects mapping from the creationId.¶

The created objects contain:¶

• id: Id¶

  the blobId which was created¶

• type: String|null¶

  the media type as given in the creation (if any); or detected from content by the server; or null¶

• size: UnsignedInt¶

  as per RFC8620 - the size of the created blob in octets¶

It will also contain any other properties identical to those that would be returned in the JSON response of the RFC8620 upload endpoint (which may be extended in the future - this document anticipates that implementations will extend both the upload endpoint and the Blob/upload responses in the same way)¶

Or if there is a problem with a creation, then the server will return a notCreated response with a map from the failed creationId to a SetError object.¶

For each successful upload, servers MUST add an entry to the creationIds map for the request. This allows the blob id to be used via back-reference in subsequent method calls.¶

The created blob will have the same lifetime and same expiry semantics as any other binary object created via the mechanism specified in [!@RFC8620] section 6.¶

Uploads using with this mechanism will be restricted by the maxUploadSize limit for JMAP requests specified by the server, and clients SHOULD consider using the upload mechanism defined by [!@RFC8620] for blobs larger than a megabyte.¶

UploadObject¶

• data: DataSourceObject[]¶

  an array of zero or more octet sources in order (zero to create an empty blob). The result of each of these sources is concatenated together in order to create the blob.¶

• type: String|null (default: null)¶

  hint for media type of the data¶

DataSourceObject¶

Exactly one of:¶

• data:asText: String|null (raw octets, must be UTF-8)¶

• data:asBase64: String|null (base64 representation of octets)¶

or a blobId source:¶

• blobId: Id¶

• offset: UnsignedInt|null (MAY be zero)¶

• length: UnsignedInt|null (MAY be zero)¶

If null then offset is assumed to be zero.¶

If null then length is the remaining octets in the blob.¶

If the range can not be fully satisfied (i.e. begins or extends past the end of the data in the blob) then the DataSourceObject is invalid and results in a notCreated response for this creation id.¶

If the data properties have any invalid references or invalid data contained in them, the server MUST NOT guess as to the user's intent, and MUST reject the creation and return a notCreated response for that creation id.¶

Likewise, invalid characters in the base64 of data:asBase64, or invalid UTF-8 in data:asText MUST result in a nonCreated response.¶

It is envisaged that the definition for DataSourceObject might be extended in the future, for example to fetch external content.¶

A server MUST accept at least 64 DataSourceObjects per create, as described in Section 3.1 of this document.¶

Method Call:

[
  "Blob/upload",
  {
    "accountId": "account1",
    "create": {
      "1": {
        "data" : [
          {
            "data:asBase64": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKA
                              AAAA1BMVEX/AAAZ4gk3AAAAAXRSTlN/gFy0ywAAAApJRE
                              FUeJxjYgAAAAYAAzY3fKgAAAAASUVORK5CYII=",
          }
        ],
        "type": "image/png"
      },
    },
  },
  "R1"
]

Response:

[
  "Blob/upload",
  {
    "accountId" : "account1",
    "created" : {
      "1": {
        "id" : "G4c6751edf9dd6903ff54b792e432fba781271beb",
        "type" : "image/png",
        "size" : 95
      },
    },
  },
  "R1"
]

Method Calls:

[
  [
    "Blob/upload",
    {
      "create": {
        "b4": {
          "data": [
            {
              "data:asText": "The quick brown fox jumped over the lazy dog."
            }
          ]
        }
      }
    },
    "S4"
  ],
  [
    "Blob/upload",
    {
      "create": {
        "cat": {
          "data": [
            {
              "data:asText": "How"
            },
            {
              "blobId": "#b4",
              "length": 7,
              "offset": 3
            },
            {
              "data:asText": "was t"
            },
            {
              "blobId": "#b4",
              "length": 1,
              "offset": 1
            },
            {
              "data:asBase64": "YXQ/"
            }
          ]
        }
      }
    },
    "CAT"
  ],
  [
    "Blob/get",
    {
      "properties": [
        "data:asText",
        "size"
      ],
      "ids": [
        "#cat"
      ]
    },
    "G4"
  ]
]

Responses:

[
  [
    "Blob/upload",
    {
      "oldState": null,
      "created": {
        "b4": {
          "id": "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
          "size": 45,
          "type": "application/octet-stream"
        }
      },
      "notCreated": null,
      "accountId": "account1"
    },
    "S4"
  ],
  [
    "Blob/upload",
    {
      "oldState": null,
      "created": {
        "cat": {
          "id": "Gcc60576f036321ae6e8037ffc56bdee589bd3e23",
          "size": 19,
          "type": "application/octet-stream"
        }
      },
      "notCreated": null,
      "accountId": "account1"
    },
    "CAT"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "Gcc60576f036321ae6e8037ffc56bdee589bd3e23",
          "data:asText": "How quick was that?",
          "size": 19
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G4"
  ]
]

4.2. Blob/get

A standard JMAP get, with two additional optional parameters:¶

• offset: UnsignedInt|null¶

  start this many octets into the blob data. If null or unspecified, this defaults to zero.¶

• length: UnsignedInt|null¶

  return at most this many octets of the blob data. If null or unspecified, then all remaining octets in the blob are returned. This can be considered equivalent to an infinitely large length value, except that the isTruncated warning is not given unless the start offset is past the end of the blob.¶

Request Properties:¶

Any of¶

• data:asText¶
• data:asBase64¶
• data (returns data:asText if the selected octets are valid UTF-8, or data:asBase64)¶
• digest:<algorithm> (where <algorithm> is one of the named algorithms in the supportedDigestAlgorithms capability)¶
• size¶

If not given, properties defaults to data and size.¶

Result Properties:¶

• data:asText: String|null¶

  the raw octets of the selected range if they are valid UTF-8, otherwise null¶

• data:asBase64: String¶

  the base64 encoding of the octets in the selected range¶

• digest:<algorithm> String¶

  the base64 encoding of the digest of the octets in the selected range, calculated using the named algorithm¶

• isEncodingProblem: Boolean (default: false)¶

• isTruncated: Boolean (default: false)¶

• size: UnsignedInt¶

  the number of octets in the entire blob¶

The size value MUST always be the number of octets in the underlying blob, regardless of offset and length.¶

The data fields contain a representation of the octets within the selected range that are present in the blob. If the octets selected are not valid UTF-8 (including truncating in the middle of a multi-octet sequence) and data or data:asText was requested, then the key isEncodingProblem MUST be set to true and the data:asText response value MUST be null. In the case where data was requested and the data is not valid UTF-8, then data:asBase64 MUST be returned.¶

If the selected range requests data outside the blob (i.e. the offset+length is larger than the blob) then the result is either just the octets from the offset to the end of the blob, or an empty string if the offset is past the end of the blob. Either way, the isTruncated property in the result MUST be set to true to tell the client that the requested range could not be fully satisfied. If digest was requested, any digest is calculated on the octets that would be returned for a data field.¶

Servers SHOULD store the size for blobs in a format which is efficient to read, and clients SHOULD limit their request to just the size parameter if that is all they need, as fetching blob content could be significantly more expensive and slower for the server.¶

Method Calls:

[
  [
    "Blob/get",
    {
      "accountId" : "account1",
      "ids" : [
        "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
        "not-a-blob"
      ],
      "properties" : [
        "data:asText",
        "digest:sha",
        "size"
      ]
    },
    "R1"
  ],
  [
    "Blob/get",
    {
      "accountId" : "account1",
      "ids" : [
        "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e"
      ],
      "properties" : [
        "data:asText",
        "digest:sha",
        "digest:sha-256",
        "size"
      ],
      "offset" : 4,
      "length" : 9
    },
    "R2"
  ]
]

Responses:

[
  [
    "Blob/get",
    {
      "accountId": "account1",
      "list": [
        {
          "id": "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
          "data:asText": "The quick brown fox jumped over the lazy dog.",
          "digest:sha": "wIVPufsDxBzOOALLDSIFKebu+U4=",
          "size": 45
        }
      ],
      "notFound": [
        "not-a-blob"
      ]
    },
    "R1"
  ],
  [
    "Blob/get",
    {
      "accountId": "account1",
      "list": [
        {
          "id": "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
          "data:asText": "quick bro",
          "digest:sha": "QiRAPtfyX8K6tm1iOAtZ87Xj3Ww=",
          "digest:sha-256": "gdg9INW7lwHK6OQ9u0dwDz2ZY/gubi0En0xlFpKt0OA=",
          "size": 45
        }
      ]
    },
    "R2"
  ]
]

Method calls:

[
  [
    "Blob/upload",
    {
      "create": {
        "b1": {
          "data": [
            {
              "data:asBase64": "VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZW
                                Qgb3ZlciB0aGUggYEgZG9nLg=="
            }
          ]
        },
        "b2": {
          "data": [
            {
              "data:asText": "hello world"
            }
          ],
          "type" : "text/plain"
        }
      }
    },
    "S1"
  ],
  [
    "Blob/get",
    {
      "ids": [
        "#b1",
        "#b2"
      ]
    },
    "G1"
  ],
  [
    "Blob/get",
    {
      "ids": [
        "#b1",
        "#b2"
      ],
      "properties": [
        "data:asText",
        "size"
      ]
    },
    "G2"
  ],
  [
    "Blob/get",
    {
      "ids": [
        "#b1",
        "#b2"
      ],
      "properties": [
        "data:asBase64",
        "size"
      ]
    },
    "G3"
  ],
  [
    "Blob/get",
    {
      "offset": 0,
      "length": 5,
      "ids": [
        "#b1",
        "#b2"
      ]
    },
    "G4"
  ],
  [
    "Blob/get",
    {
      "offset": 20,
      "length": 100,
      "ids": [
        "#b1",
        "#b2"
      ]
    },
    "G5"
  ]
]

Responses:

[
  [
    "Blob/upload",
    {
      "oldState": null,
      "created": {
        "b2": {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "size": 11,
          "type": "application/octet-stream"
        },
        "b1": {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "size": 43,
          "type": "text/plain"
        }
      },
      "updated": null,
      "destroyed": null,
      "notCreated": null,
      "notUpdated": null,
      "notDestroyed": null,
      "accountId": "account1"
    },
    "S1"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "isEncodingProblem": true,
          "data:asBase64": "VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZW
                            Qgb3ZlciB0aGUggYEgZG9nLg==",
          "size": 43
        },
        {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "data:asText": "hello world",
          "size": 11
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G1"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "isEncodingProblem": true,
          "size": 43
        },
        {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "data:asText": "hello world",
          "size": 11
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G2"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "data:asBase64": "VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wZW
                            Qgb3ZlciB0aGUggYEgZG9nLg==",
          "size": 43
        },
        {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "data:asBase64": "aGVsbG8gd29ybGQ=",
          "size": 11
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G3"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "data:asText": "The q",
          "size": 43
        },
        {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "data:asText": "hello",
          "size": 11
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G4"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "G72cfa4804194563685d9a4b695f7ba20e7739576",
          "isTruncated": true,
          "isEncodingProblem": true,
          "data:asBase64": "anVtcGVkIG92ZXIgdGhlIIGBIGRvZy4=",
          "size": 43
        },
        {
          "id": "G2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",
          "isTruncated": true,
          "data:asText": "",
          "size": 11
        }
      ],
      "notFound": [],
      "accountId": "account1"
    },
    "G5"
  ]
]

4.3. Blob/lookup

Given a list of blobIds, this method does a reverse lookup in each of the provided type names to find the list of Ids within that data type which reference the provided blob.¶

Since different datatypes will have different semantics of "contains", the definition of reference is somewhat loosely defined, but roughly means "you could discover this blobId by looking at this object or at other objects recursively contained within this object".¶

For example with an [RFC8621] server, if checking whether a Mailbox references a blob, then if any Emails within that Mailbox reference the blobId, then the Mailbox references that blobId. For any Thread which references an Email that references a blobId, it can be said that the Thread references the blobId.¶

But this does not mean that if an Email references a Mailbox in its mailboxIds property, then any blobId referenced by other Emails in that Mailbox are also referenced by the initial Email.¶

Parameters¶

• accountId: Id¶

  The id of the account used for the call.¶

• typeNames: String[]¶

  A list of names from the "JMAP Data Types" registry, or defined by private extensions which the client has requested. Only names for which "Can reference blobs" is true may be specified, and the capability which defines each type must also be used by the overall JMAP request in which this method is called.¶

  If a type name is not known by the server, or the associated capability has not been requested, then the server returns an "unknownDataType" error.¶

• ids: Id[]¶

  A list of blobId values to be looked for.¶

Response¶

• list: BlobInfo[]¶

  A list of BlobInfo objects.¶

BlobInfo Object¶

• id: Id¶

  The Blob Identifier.¶

• matchedIds: String[Id[]]¶

  A map from type name to list of Ids of that data type (e.g. the name "Email" maps to a list of emailIds)¶

If a blob is not visible to a user, or does not exist on the server at all, then the server MUST still return an empty array for each type as this doesn't leak any information about whether the blob is on the server but not visible to the requesting user.¶

Method call:

[
  "Blob/lookup",
  {
    "typeNames": [
      "Mailbox",
      "Thread",
      "Email"
    ],
    "ids": [
      "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
      "not-a-blob"
    ]
  },
  "R1"
]

Response:

[
  "Blob/lookup",
  {
    "list": [
      {
        "id": "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
        "matchedIds": {
          "Mailbox": [
            "M54e97373",
            "Mcbe6b662"
          ],
          "Thread": [
            "T1530616e"
          ],
          "Email": [
            "E16e70a73eb4",
            "E84b0930cf16"
          ]
        }
      }
    ],
    "notFound": [
      "not-a-blob"
    ]
  },
  "R1"
]

6.1. JMAP Capability registration for "blob"

IANA is requested to register the "blob" JMAP Capability as follows:¶

Capability Name: urn:ietf:params:jmap:blob¶

Specification document: this document¶

Intended use: common¶

Change Controller: IETF¶

Security and privacy considerations: this document, Section XXX¶

6.2. JMAP Error Codes Registration for "unknownDataType"

IANA is requested to register the "unknownDataType" JMAP Error Code as follows:¶

JMAP Error Code: unknownDataType¶

Intended use: common¶

Change Controller: IETF¶

Reference: this document¶

Description: The server does not recognise this data type, or the capability to enable it was not present.¶

6.3. Creation of "JMAP Data Types" Registry

IANA is requested to create a new registry "JMAP Data Types" with the initial content:¶

This policy for this registry is "Specification required", either an RFC or a similarly stable reference document which defines a JMAP Data Type and associated capability.¶

IANA is asked to appoint designated experts to review requests for additions to this registry, with guidance to allow any registration which provides a stable document describing the capability, and control over the URI namespace where the capability URI points.¶