Linearized Matrix API
draft-ralston-mimi-linearized-matrix-01
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Active".
|
|
|---|---|---|---|
| Authors | Travis Ralston , Matthew Hodgson | ||
| Last updated | 2023-06-15 (Latest revision 2023-06-06) | ||
| Replaces | draft-ralston-mimi-matrix-transport, draft-ralston-mimi-matrix-framework | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ralston-mimi-linearized-matrix-01
More Instant Messaging Interoperability T. Ralston
Internet-Draft M. Hodgson
Intended status: Standards Track The Matrix.org Foundation C.I.C.
Expires: 8 December 2023 6 June 2023
Linearized Matrix API
draft-ralston-mimi-linearized-matrix-01
Abstract
Matrix is an existing openly specified decentralized secure
communications protocol able to provide a framework for instant
messaging interoperability. However, the existing model can be
complex to reason about for simple interoperability usecases. With
modifications to the room model, Matrix can support those simpler
usecases more easily.
This document explores "Linearized Matrix": the modified room model
still backed by Matrix.
About This Document
This note is to be removed before publishing as an RFC.
The latest revision of this draft can be found at
https://turt2live.github.io/ietf-mimi-linearized-matrix/draft-
ralston-mimi-linearized-matrix.html. Status information for this
document may be found at https://datatracker.ietf.org/doc/draft-
ralston-mimi-linearized-matrix/.
Discussion of this document takes place on the More Instant Messaging
Interoperability Working Group mailing list (mailto:mimi@ietf.org),
which is archived at https://mailarchive.ietf.org/arch/browse/mimi/.
Subscribe at https://www.ietf.org/mailman/listinfo/mimi/.
Source for this draft and an issue tracker can be found at
https://github.com/turt2live/ietf-mimi-linearized-matrix.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Ralston & Hodgson Expires 8 December 2023 [Page 1]
Internet-Draft Linearized Matrix API June 2023
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 8 December 2023.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3
3. Architecture . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1. Server names / domain names . . . . . . . . . . . . . . . 5
3.2. Rooms . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3. Users . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4. Devices . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.5. Events . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.5.1. Linearized PDU . . . . . . . . . . . . . . . . . . . 9
3.5.2. State events . . . . . . . . . . . . . . . . . . . . 10
3.5.3. Event types . . . . . . . . . . . . . . . . . . . . . 10
4. MLS Considerations . . . . . . . . . . . . . . . . . . . . . 13
4.1. Server-side Room Model . . . . . . . . . . . . . . . . . 13
4.2. Client-side Room Model . . . . . . . . . . . . . . . . . 14
5. Event Signing & Authorization . . . . . . . . . . . . . . . . 15
5.1. Checks performed upon receipt of a PDU/event . . . . . . 15
5.2. Rejection . . . . . . . . . . . . . . . . . . . . . . . . 16
5.3. Soft failure . . . . . . . . . . . . . . . . . . . . . . 16
5.4. Authorization rules . . . . . . . . . . . . . . . . . . . 16
5.4.1. Auth events selection . . . . . . . . . . . . . . . . 16
5.4.2. Auth rules algorithm . . . . . . . . . . . . . . . . 17
5.5. Signing . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.5.1. Canonical JSON . . . . . . . . . . . . . . . . . . . 21
5.5.2. Signing arbitrary objects . . . . . . . . . . . . . . 21
5.5.3. Signing events . . . . . . . . . . . . . . . . . . . 22
Ralston & Hodgson Expires 8 December 2023 [Page 2]
Internet-Draft Linearized Matrix API June 2023
5.5.4. Checking a signature . . . . . . . . . . . . . . . . 23
5.6. Hashes . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.6.1. Content hash calculation . . . . . . . . . . . . . . 23
5.6.2. Reference hash . . . . . . . . . . . . . . . . . . . 23
5.7. Unpadded Base64 . . . . . . . . . . . . . . . . . . . . . 24
6. Hub transfers . . . . . . . . . . . . . . . . . . . . . . . . 24
7. Transport . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8. Security Considerations . . . . . . . . . . . . . . . . . . . 24
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 24
10.1. Normative References . . . . . . . . . . . . . . . . . . 24
10.2. Informative References . . . . . . . . . . . . . . . . . 25
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction
Alongside messaging, Matrix operates as an openly federated
communications protocol for VoIP, IoT, and more. The existing Matrix
network uses fully decentralized access control within rooms
(conversations) and is highly extensible in its structure. These
features are not critically important to a strict focus on messaging
interoperability, however.
This document describes "Linearized Matrix": a modified room model
based upon Matrix's existing room model. This document does _not_
explore how to interconnect Linearized Matrix with the existing
Matrix room model - interested readers may wish to review MSC3995
[MSC3995] within the Matrix Specification process.
2. Conventions and Definitions
This document uses [I-D.ralston-mimi-terminology] where possible.
This document additionally uses the following definitions:
* *Room*: Synonymous with "conversation" from I-D.ralston-mimi-
terminology.
* *Room Member*: Synonymous with "conversation member" from I-
D.ralston-mimi-terminology.
* *State Event*: Synonymous with "conversation property" from I-
D.ralston-mimi-terminology. A state event is a subclass of an
event.
Further terms are introduced in-context within this document.
Ralston & Hodgson Expires 8 December 2023 [Page 3]
Internet-Draft Linearized Matrix API June 2023
*TODO*: We should move/copy those definitions up here anyways.
3. Architecture
For a given conversation/room:
.------------. .------------.
| Client A | | Client B |
'-----------+' '-----------+'
^ | ^ |
| | Client-Server API | |
| V | V
+---+--------------+ +---+--------------+
| +----------( events )------->| |
| Provider/Server | | Provider/Server |
| A |<---------( events )--------+ B |
+-----+------------+ Server-Server API +------------------+
| ^
| | +------------------+
| +-----------------( events )--------+ |
| | Provider/Server |
+-----------------------( events )------->| C |
+--------------+---+
^ |
| |
| V
.+-----------.
| Client C |
'------------'
In this diagram, Server A is acting as a hub for the other two
servers. Servers B and C do not converse directly when sending
events to the room: those events are instead sent to the hub which
then distributes them back out to all participating servers.
Clients are shown in the diagram here for demonstrative purposes
only. No client-server API is specified as part of Linearized
Matrix, and the clients can be pre-existing or newly created for
messaging. The objects given to clients are implementation-
dependent, though for simplicity may be events.
This leads to two distinct roles:
* *Hub server*: the server responsible for holding conversation
history on behalf of other servers in the room.
Ralston & Hodgson Expires 8 December 2023 [Page 4]
Internet-Draft Linearized Matrix API June 2023
* *Participant server*: any non-hub server. This server is not
required to persist conversation history as it can fetch it from
the hub if needed.
*OPEN QUESTION*: Should we support having multiple hubs for increased
trust between participant and hub? (participant can pick the hub it
wants to use rather than being forced to use a single hub)
3.1. Server names / domain names
Throughout this document servers are referred to as having a "domain
name" or "server name". A server name MUST be compliant with RFC
1123 (Section 2.1) [RFC1123].
*TODO*: Should we incorporate Matrix's IPv6 extension, or are we able
to assume that everyone will be using non-literal hostnames?
*TODO*: Do we really need to make this case sensitive? Matrix does,
but is that correct?
3.2. Rooms
A room is a conceptual place where users send and receive events.
Events are sent to a room, and all users which have sufficient access
will receive that event.
Rooms have a single internal "Room ID" to identify them from another
room:
!<opaque>:<domain>
For example, !abc:example.org.
The opaque portion of the room ID, called the localpart, must not be
empty and must consist entirely of the characters [0-9a-zA-Z._~-].
The domain portion of a room ID does _NOT_ indicate the room is
"hosted" or served by that domain. The domain is used as a namespace
to prevent another server from maliciously taking over a room. The
server represented by that domain may no longer be participating in
the room.
The total length (including the sigil and domain) of a room ID MUST
NOT exceed 255 characters.
Room IDs are case sensitive.
Ralston & Hodgson Expires 8 December 2023 [Page 5]
Internet-Draft Linearized Matrix API June 2023
3.3. Users
As described by [I-D.ralston-mimi-terminology], a user is typically a
human which operates a client. In Linearized Matrix, all users have
a User ID to distinguish them:
@<localpart>:<domain>
The localpart portion of the user ID is expected to be human-
readable, MUST NOT be empty, and MUST consist solely of [0-9a-z._=-/]
characters. Note that user IDs _cannot_ contain uppercase letters in
the localpart.
The domain portion indicates which server allocated the ID, or would
allocate the resource if the user doesn't exist yet.
@alice:first.example.org is a different user on a different server
from @alice:second.example.org, for example.
The total length (including the sigil and domain) of a user ID MUST
NOT exceed 255 characters.
User IDs are case sensitive.
*Note*: User IDs are sometimes informally referenced as "MXIDs",
short for "Matrix User IDs".
*Author's note*: This draft assumes that an external system will
resolve phone number to user ID, somehow. Or that
@18005552222:example.org will resolve to +1 800 555 2222 on a given
server, or similar.
3.4. Devices
Each user can have zero or more devices/active clients. These
devices are intended to be members of the MLS group and thus have
their own key package material associated with them.
*TODO*: Do we need to define grammar and such for device IDs, or is
that covered by MLS already?
3.5. Events
All data exchanged over Linearized Matrix is expressed as an "event".
Each client action (such as sending a message) correlates with
exactly one event. All events have a type to distinguish them, and
use reverse domain name notation to namespace custom events (for
example, org.example.appname.eventname). Event types specified by
Linearized Matrix itself use m. as their namespace.
Ralston & Hodgson Expires 8 December 2023 [Page 6]
Internet-Draft Linearized Matrix API June 2023
When events are traversing a transport to another server they are
often referred to as a *Persistent Data Unit* or *PDU*.
An event has many other fields:
* room_id (string; required) - The room ID for where the event is
being sent.
* type (string; required) - A UTF-8 [RFC3629] string to distinguish
different data types being carried by events. All event types use
a reverse domain name notation to namespace themselves (for
example, org.example.appname.eventname). Event types specified by
Linearized Matrix itself use m as their namespace (for example,
m.room.member).
* state_key (string; optional) - A UTF-8 [RFC3629] string to further
distinguish an event type from other related events. Only
specified on State Events (discussed later). Can be empty.
* sender (string; required) - The user ID which is sending this
event.
* origin_server_ts (integer; required) - The milliseconds since the
unix epoch for when this event was created.
* hub_server (string; technically optional) - The domain name of the
hub server which is sending this event to the remainder of the
room. Note that all events created within Linearized Matrix will
have this field set.
* content (object; required) - The event content. The schema of
this is specific to the event type, and should be considered
untrusted data until verified otherwise. Malicious servers and
clients can, for example, exclude important fields, use invalid
value types, or otherwise attempt to disrupt a client - receivers
should treat the event with care while processing.
* hashes (object; required) - Keyed by hash algorithm, the _content
hash_ for the event.
* signatures (object; required) - Keyed first by domain name then by
key ID, the signatures for the event.
* auth_events (array of strings; required) - The event IDs which
prove the sender is able to send this event in the room. Which
specific events are put here are defined by the _auth events
selection_ algorithm.
Ralston & Hodgson Expires 8 December 2023 [Page 7]
Internet-Draft Linearized Matrix API June 2023
* prev_events (array of strings; required) - The event IDs which
precede the event. Note that all events generated within
Linearized Matrix will only ever have a single event ID here.
* unsigned (object; optional) - Additional metadata not covered by
the signing algorithm.
Note that an event ID is not specified on the schema. Event IDs are
calculated to ensure accuracy and consistency between servers. To
calculate an event ID, calculate the _reference hash_ of the event,
encode it using _URL-safe Unpadded Base64_, and prefix it with the
event ID sigil, $.
If both the sender and receiver are implementing the algorithms
correctly, the event ID will be the same. When different, the
receiver will have issues accepting the event (none of the
auth_events will make sense, for example). Both sender and receiver
should review their algorithm implementation to verify everything is
according to the specification in this case.
Events are treated as JSON [RFC8259] within the protocol, but can be
encoded and represented by any binary-compatible format. Additional
overhead may be introduced when converting between formats, however.
An example may be:
Ralston & Hodgson Expires 8 December 2023 [Page 8]
Internet-Draft Linearized Matrix API June 2023
{
"room_id": "!abc:example.org",
"type": "m.room.member",
"state_key": "@alice:first.example.org",
"sender": "@bob:second.example.org",
"origin_server_ts": 1681340188825,
"hub_server": "first.example.org",
"content": {
"membership": "invite"
},
"hashes": {
"sha256": "<unpadded base64>"
},
"signatures": {
"first.example.org": {
"ed25519:1": "<unpadded base64 for signature covering whole event>"
},
"second.example.org": {
"ed25519:1": "<unpadded base64 for signature covering LPDU>"
}
},
"auth_events": ["$first", "$second"],
"prev_events": ["$parent"],
"unsigned": {
"arbitrary": "fields"
}
}
3.5.1. Linearized PDU
The hub server is responsible for ensuring events are linearly added
to the room from all participants, which means participants cannot
set fields such as prev_events on their events. Additionally,
participant servers are not expected to store past conversation
history or even "current state" for the room, further making
participants unable to reliably populate auth_events and prev_events.
To avoid these problems, the participant server _does not_ populate
the following fields on events they are sending to the hub:
* auth_events - the participant cannot reliably determine what
allows it to send the event.
* prev_events - the participant cannot reliably know what event
precedes theirs.
* hashes - the hashes cover the above two fields.
Ralston & Hodgson Expires 8 December 2023 [Page 9]
Internet-Draft Linearized Matrix API June 2023
The participant server will receive an echo of the fully-formed event
from the hub once appended. To ensure authenticity, the participant
server signs this "Linearized PDU" or "LPDU" using the normal event
_signing algorithm_.
*TODO*: While a signature is great, it doesn't cover the content. We
need to fix hashes to actually support an LPDU hash alongside a full-
blown content hash.
3.5.2. State events
State events track metadata for the room, such as name, topic, and
members. State is keyed by a tuple of type and state_key, noting
that an empty string is a valid state key. State in the room with
the same key-tuple will be overwritten.
State events are otherwise processed like regular events in the room:
they're appended to the room history and can be referenced by that
room history.
"Current state" is the state at the time being considered (which is
often the implied HEAD of the room). In Linearized Matrix, a simple
approach to calculating current state is to iterate over all events
in order, overwriting the key-tuple for state events in an adjacent
map. That map becomes "current state" when the loop is finished.
3.5.3. Event types
Linearized Matrix defines the following event types:
3.5.3.1. m.room.create
The very first event in the room. It MUST NOT have any auth_events
or prev_events, and the domain of the sender MUST be the same as the
domain in the room_id. The state_key MUST be an empty string.
The content for a create event MUST have at least a room_version
field to denote what set of algorithms the room is using. This
document as a whole describes a single room version identified as
I.1.
*Implementation note*: Currently I.1 is not a real thing. Use
org.matrix.i-d.ralston-mimi-linearized-matrix.00 when testing against
other Linearized Matrix implementations. This room version may be
updated later.
*TODO*: Describe room versions more?
Ralston & Hodgson Expires 8 December 2023 [Page 10]
Internet-Draft Linearized Matrix API June 2023
3.5.3.2. m.room.join_rules
Defines whether users can join without an invite and other similar
conditions. The state_key MUST be an empty string.
The content for a join rules event MUST have at least a join_rule
field to denote the join policy for the room. Allowable values are:
* public - anyone can join without an invite.
* knock - users must receive an invite to join, and can request an
invite (knock) too.
* invite - users must receive an invite to join.
*TODO*: Describe restricted (and knock_restricted) rooms?
3.5.3.3. m.room.member
Defines the membership for a user in the room. If the user does not
have a membership event then they are presumed to be in the leave
state.
The state_key MUST be a non-empty string denotating the user ID the
membership is affecting.
The content for a membership event MUST have at least a membership
field to denote the membership state for the user. Allowable values
are:
* leave - not participating in the room. If the state_key and
sender do not match, this was a kick rather than voluntary leave.
* join - participating in the room.
* knock - requesting an invite to the room.
* invite - invited to participate in the room.
* ban - implies kicked/not participating. Cannot be invited or join
the room without being unbanned first (moderator sends a kick,
essentially).
The _auth rules_ define how these membership states interact and what
legal transitions are possible. For example, preventing users from
unbanning themselves falls under the auth rules.
Ralston & Hodgson Expires 8 December 2023 [Page 11]
Internet-Draft Linearized Matrix API June 2023
3.5.3.4. m.room.power_levels
Defines what given users can and can't do, as well as which event
types they are able to send. The enforcement of these power levels
is determined by the _auth rules_.
The state_key MUST be an empty string.
The content for a power levels event SHOULD have at least the
following:
* ban (integer) - the level required to ban a user. Defaults to 50
if unspecified.
* kick (integer) - the level required to kick a user. Defaults to
50 if unspecified.
* invite (integer) - the level required to invite a user. Defaults
to 0 if unspecified.
* redact (integer) - the level required to redact an event sent by
another user. Defaults to 50 if unspecified.
* events (map) - keyed by event type string, the level required to
send that event type to the room. Defaults to an empty map if
unspecified.
* events_default (integer) - the level required to send events in
the room. Overridden by the events map. Defaults to 0 if
unspecified.
* state_default (integer) - the level required to send state events
in the room. Overridden by the events map. Defaults to 50 if
unspecified.
* users (map) - keyed by user ID, the level of that user. Defaults
to an empty map if unspecified.
* users_default (integer) - the level for users. Overridden by the
users map. Defaults to 0 if unspecified.
*TODO*: Include notifications for at-room here too?
Note that if no power levels event is specified in the room then the
room creator (sender of the m.room.create state event) has a default
power level of 100.
Ralston & Hodgson Expires 8 December 2023 [Page 12]
Internet-Draft Linearized Matrix API June 2023
3.5.3.5. TODO: Other events
*TODO*: m.room.name, m.room.topic, m.room.avatar, m.room.encryption,
m.room.history_visibility
*TODO*: Drop m.room.encryption and pack it into the create event
instead?
4. MLS Considerations
MIMI has a chartered requirement to use MLS for encryption, and MLS
requires that all group members (devices) know of all other devices.
If we consider each Matrix room to have an MLS group, we encounter
scenarios where the room and group membership might diverge or
otherwise not be equivalent.
In a traditional Matrix room, membership is not managed at a per-
device level but rather a per-user level. Devices are authenticated
to use the room by being attached to a user. This model doesn't work
in MLS, though.
A couple of options present themselves:
1. Keep managing the room state at the server level, as is
traditional for Matrix, and define a set of rules/methods for
engaging devices/users in the room with the MLS group. Servers
have an ability to instruct devices on how/when to add/remove MLS
group members, but not an ability to handle the MLS Proposals and
Commits directly.
2. Coordinate a room's state at the device level, leaving servers to
figure out how to push events between servers (and by extension,
other devices). Servers would not have knowledge or ability to
reject proposals based on authorization beyond transport-level
authenticity concerns.
At this stage of drafting in the document, it is not clear which
would be preferred. Both are explored.
4.1. Server-side Room Model
In this model, servers handle the room state on behalf of devices.
This gives the server an ability to apply access control at a user
level, and instruct other devices on when/how to add or remove
devices from the underlying MLS group. The server does not have an
ability to participate in the MLS group directly.
Ralston & Hodgson Expires 8 December 2023 [Page 13]
Internet-Draft Linearized Matrix API June 2023
This is how traditional Matrix rooms work by handling state changes
(user membership, etc) in cleartext for everyone to see. A user's
devices would be tracked and added/removed from the MLS group as
needed.
The exact rules for how a user's devices become engaged with the MLS
group is not yet defined.
An advantage over this model compared to client-side is the server is
able to reduce the client's traffic by rejecting events earlier and
deal with conflicts that may arise, keeping the conversation as
linear as possible for the client.
A clear disadvantage is that without cross-signing or other
cryptographic mechanism, the server would be able to add malicious
devices to its users and therefore the MLS group. A precise
mitigation strategy is not yet defined by this document, but would
involve building verifiable trust in a device before it is allowed to
participate in the MLS group.
The existing model used by Linearized Matrix is covered by "Event
Signing & Authorization" later in this document.
*TODO*: We might also need DMLS to handle some of the server-side
conflicts?
4.2. Client-side Room Model
Here, the room's state is completely managed within the MLS group.
This provides a key advantage where servers become message-passing
nodes (in essence), but increases implementation complexity on the
clients/devices.
Much of this model is based around the server-side model discussed
above: event authorization rules, redactions, etc still behave the
same, but on the client-side instead. The server would likely be
responsible for ensuring incoming events are properly signed, but
otherwise leave it up to clients to accept or reject them into their
internal linked list.
A potential consequence of this model is clients needing to implement
a conflict resolution algorithm despite having linear room history.
This is due to clients receiving MLS messages out of guaranteed
order.
*TODO*: This could be DMLS, state res, or both.
Ralston & Hodgson Expires 8 December 2023 [Page 14]
Internet-Draft Linearized Matrix API June 2023
5. Event Signing & Authorization
There are a few aspects of an event which verify its authenticity and
therefore whether it can be accepted into the room. All of these
checks work with the fully-formed PDU for an event.
First, the event has a _content hash_ which covers the unredacted
content of the event. The purpose of this hash is to ensure that the
original contents are not modified, therefore if the hash check fails
then the event is redacted before continuing processing. This
removes any potentially malicious or malformed details from the
event.
Second, the event has a _reference hash_ which covers the redacted
event. This hash serves as the event's ID and thus any difference in
calculation will result in an entirely different event ID.
Third, the event must be signed by the domain implied by the sender.
In Linearized Matrix, this will usually be the LPDU signature
discussed earlier in this document. This signature covers the
content hash of the event.
*TODO*: Except the LPDU signature doesn't cover the participant's
content hash, because the participant doesn't have a content hash at
the moment.
Finally, the event must be signed by the hub_server domain if
present. This is to ensure that the event has actually been
processed by the hub and isn't falsely being advertised as sent by a
hub.
*TODO*: Does the hub's signature actually guard anything?
5.1. Checks performed upon receipt of a PDU/event
When a hub receives an LPDU from a participant it adds the missing
fields to create a fully formed PDU then sends that PDU back out to
all participants, including the original sender.
When a server (hub or participant) receives a PDU, it:
1. Verifies the event is in a valid shape. This will mean ensuring
the required schema is met and of the correct type (there is a
string type, etc). Note that the event may have additional
fields in various places, such as at the top level or within
content: the receiver should ensure these additional fields do
not cause the event to be invalid. If the event fails this
validation, it is dropped.
Ralston & Hodgson Expires 8 December 2023 [Page 15]
Internet-Draft Linearized Matrix API June 2023
2. Ensures the required signatures are present and valid. If the
event fails this, it is dropped.
3. Ensures the event has a valid content hash. If the event's hash
doesn't match, it is redacted before processing further. The
server will ultimately persist the redacted copy.
4. Ensures the event passes the authorization rules for the state
identified by the event's auth_events. If it fails, it is
rejected.
5. Ensures the event passes the authorization rules for the state of
the room immediately before where the event would be inserted.
If it fails, it is rejected.
6. Ensures the event passes the authorization rules for the current
state of the room (which may very well be the same as the step
above). If it fails, it is soft-failed.
5.2. Rejection
Events which are rejected are not relayed to any local clients and
are not appended to the room in any way. Within Linearized Matrix,
events which reference rejected events are rejected themselves.
5.3. Soft failure
When an event is "soft-failed" it should not be relayed to any local
clients nor be used in auth_events. The event is otherwise handled
as per usual.
5.4. Authorization rules
These are the rules which govern whether an event is accepted into
the room, depending on the state events surrounding that event. A
given event is checked against multiple different sets of state.
5.4.1. Auth events selection
The auth_events on an event MUST consist of the following state
events, with the exception of an m.room.create event which has no
auth_events.
1. The m.room.create state event.
2. The current m.room.power_levels state event, if any.
3. The sender's current m.room.member state event, if any.
Ralston & Hodgson Expires 8 December 2023 [Page 16]
Internet-Draft Linearized Matrix API June 2023
4. If the type is m.room.member:
1. The target's (state_key) current m.room.member state event,
if any.
2. If content.membership is join or invite, the current
m.room.join_rules state event, if any.
*TODO*: Talk about restricted room joins here?
5.4.2. Auth rules algorithm
With consideration for default/calculated power levels, the ordered
rules which affect authorization of a given event are:
*TODO*: should we reference m.federate?
1. Events must be signed by the server denoted by the sender field.
Note that this may be an LPDU if the hub_server is specified and
not the same server.
2. If hub_server is present, events must be signed by that server.
3. If type is m.room.create:
1. If it has any prev_events, reject.
2. If the domain of the room_id is not the same domain as the
sender, reject.
3. If content.room_version is not I.1, reject. *TODO*:
Incorporate room versions properly.
4. Otherwise, allow.
4. Considering the event's auth_events:
1. If there are duplicate entries for a given type and
state_key pair, reject.
2. If there are entries whose type and state_key do not match
those specified by the auth events selection algorithm,
reject.
3. If there are entries where the referenced event was rejected
during receipt, reject.
Ralston & Hodgson Expires 8 December 2023 [Page 17]
Internet-Draft Linearized Matrix API June 2023
4. If there is no m.room.create event among the entries,
reject.
5. If type is m.room.member:
1. If there is no state_key property, or no membership in
content, reject.
2. If membership is join:
1. If the previous event is an m.room.create event and the
state_key is the creator, allow.
2. If sender does not match state_key, reject.
3. If the sender is banned, reject.
4. If the join_rule for m.room.join_rules is invite or
knock, then allow if the current membership state is
invite or join.
5. If the join_rule for m.room.join_rules is public, allow.
6. Otherwise, reject.
3. If membership is invite:
1. If the sender's current membership state is not join,
reject.
2. If the target user's (state_key) membership is join or
ban, reject.
3. If the sender's power level is greater than or equal to
the power level needed to send invites, allow.
4. Otherwise, reject.
4. If membership is leave:
1. If the sender matches the state_key, allow if and only
if that user's current membership state is knock, join,
or invite.
2. If the sender's current membership state is not join,
reject.
Ralston & Hodgson Expires 8 December 2023 [Page 18]
Internet-Draft Linearized Matrix API June 2023
3. If the target user's (state_key) current membership
state is ban, and the sender's power level is less than
the power level needed to ban other users, reject.
4. If the sender's power level is greater than or equal to
the power level needed to kick users, and the target
user's (state_key) power level is less than the
sender's, allow.
5. Otherwise, reject.
5. If membership is ban:
1. If the sender's current membership state is not join,
reject.
2. If the sender's power level is greater than or equal to
the power level needed to ban users, and the target
user's (state_key) power level is less than the sender's
power level, allow.
3. Otherwise, reject.
6. If membership is knock:
1. If the join_rule for m.room.join_rules is anything other
than knock, reject.
2. If the sender does not match the state_key, reject.
3. If the sender's current membership state is not ban or
join, allow.
4. Otherwise, reject.
7. Otherwise, the membership is unknown. Reject.
6. If the sender's current membership state is not join, reject.
7. If the event type's required power level is greater than the
sender's power level, reject.
8. If the event has a state_key which starts with an @ and does not
match the sender, reject.
9. If type is m.room.power_levels:
Ralston & Hodgson Expires 8 December 2023 [Page 19]
Internet-Draft Linearized Matrix API June 2023
1. If any of the fields users_default, events_default,
state_default, ban, redact, kick, or invite in content are
present and not an integer, reject.
2. If events in content is present and not an object with
values that are integers, reject.
3. If the users in content is present and not an object with
valid user IDs as keys and integers as values, reject.
4. If there is no previous m.room.power_levels event in the
room, allow.
5. For the fields users_default, events_default,
state_default, ban, redact, kick, and invite, check if they
were added, changed, or removed. For each found
alteration:
1. If the current value is higher than the sender's
current power level, reject.
2. If the new value is higher than the sender's current
power level, reject.
6. For each entry being changed in or removed from events:
1. If the current value is higher than the sender's
current power level, reject.
7. For each entry being added to or changed in events:
1. If the new value is greater than the sender's current
power level, reject.
8. For each entry being changed in or removed from users,
other than the sender's own entry:
1. If the current value is higher than the sender's
current power level, reject.
9. For each entry being added to or changed in users:
1. If the new value is greater than the sender's current
power level, reject.
10. Otherwise, allow.
10. Otherwise, allow.
Ralston & Hodgson Expires 8 December 2023 [Page 20]
Internet-Draft Linearized Matrix API June 2023
There are some consequences to these rules:
* Unless you are already a member of the room, the only permitted
operations (aside from the initial create/join) are being able to
join public rooms, accept invites to rooms, and reject invites to
rooms.
* To unban another user, the sender must have a power level greater
than or equal to both the kick and ban power levels, _and_ greater
than the target user's power level.
*TODO*: If we want to enforce a single hub in a room, we'd do so here
with auth rules.
5.5. Signing
All servers, including hubs and participants, publish an ed25519
[RFC8032] signing key to be used by other servers when verifying
signatures.
*TODO*: Verify RFC reference. We might be using a slightly different
ed25519 key today? See https://hdevalence.ca/blog/2020-10-04-its-
25519am
5.5.1. Canonical JSON
When signing a JSON object, such as an event, it is important that
the bytes be ordered in the same way for everyone. Otherwise, the
signatures will never match.
To canonicalize a JSON object, use [RFC8785].
*TODO*: Matrix currently doesn't use RFC8785, but it should (or
similar).
5.5.2. Signing arbitrary objects
Though events receive a lot of signing, it is often necessary for a
server to sign arbitary, non-event, payloads as well. For example,
in Matrix's existing HTTPS+JSON transport, requests are signed to
ensure they came from the source they claim to be.
To sign an object, the JSON is canonically encoded without the
signatures or unsigned fields. The bytes of the canonically encoded
JSON are then signed using the ed25519 signing key for the server.
The resulting signature is then encoded using unpadded base64.
Ralston & Hodgson Expires 8 December 2023 [Page 21]
Internet-Draft Linearized Matrix API June 2023
5.5.3. Signing events
Signing events is very similar to signing an arbitary object, however
with a note that an event is first redacted before signing. This
ensures that later if the event were to be redacted in the room that
the signature check still passes.
Note that the content hash covers the event's contents in case of
redaction.
5.5.3.1. Redacting an event
All fields at the top level except the following are stripped from
the event:
* type
* room_id
* sender
* state_key
* content
* origin_server_ts
* hashes
* signatures
* prev_events
* auth_events
* hub_server
Additionally, some event types retain specific fields under the
event's content. All other fields are stripped.
* m.room.create retains all fields in content.
* m.room.member retains membership.
* m.room.join_rules retains join_rule.
* m.room.power_levels retains ban, events, events_default, kick,
redact, state_default, users, users_default, and invite.
Ralston & Hodgson Expires 8 December 2023 [Page 22]
Internet-Draft Linearized Matrix API June 2023
* m.room.history_visibility retains history_visibility.
5.5.4. Checking a signature
If the signatures field is missing, doesn't contain the entity that
is expected to have done the signing (a server name), doesn't have a
known key ID, or is otherwise structurally invalid then the signature
check fails.
If decoding the base64 fails, the check fails.
If removing the signatures and unsigned properties, canonicalizing
the JSON, and verifying the signature fails, the check fails.
Otherwise, the check passes.
5.6. Hashes
An event is covered by two hashes: a content hash and a reference
hash. The content hash covers the unredacted event to ensure it was
not modified in transit. The reference hash covers the essential
fields of the event, including content hashes, and serves as the
event's ID.
5.6.1. Content hash calculation
1. Remove any existing unsigned, signatures, and hashes fields.
2. Encode the object using canonical JSON.
3. Hash the resulting bytes with SHA-256 [RFC6234].
4. Encode the hash using unpadded base64.
5.6.2. Reference hash
1. Redact the event.
2. Remove signatures and unsigned fields.
3. Encode the object using canonical JSON.
4. Hash the resulting bytes with SHA-256 [RFC6234].
5. Encode the hash using URL-safe unpadded base64.
Ralston & Hodgson Expires 8 December 2023 [Page 23]
Internet-Draft Linearized Matrix API June 2023
5.7. Unpadded Base64
Throughout this document, "unpadded base64" is used to represent
binary values as strings. Base64 is as specified by [RFC4648], and
_unpadded_ base64 simply removes any = padding from the resulting
string.
Implementations SHOULD accept input with or without padding on base64
values.
Section 5 of [RFC4648] describes _URL-safe_ base64. The same changes
are adopted here. Namely, the 62nd and 63rd characters are replaced
with - and _ respectively. The unpadded behaviour is as described
above.
6. Hub transfers
*TODO*: This section, if we want a single canonical hub in the room.
Some expected problems in this area are: who signs the transfer
event? who _sends_ the transfer event? how does a transfer start?
*TODO*: Is this section better placed in the MSC for now?
7. Transport
*TODO*: This section, though this is likely (should be?) to be a
dedicated I-D.
Topics: * Server discovery * Publishing of signing keys * Sending
events between servers * Media handling * etc
Matrix currently uses an HTTPS+JSON transport for this.
8. Security Considerations
*TODO*: Expand upon this section.
9. IANA Considerations
The m.* namespace likely needs formal registration in some capacity.
10. References
10.1. Normative References
Ralston & Hodgson Expires 8 December 2023 [Page 24]
Internet-Draft Linearized Matrix API June 2023
[I-D.ralston-mimi-terminology]
Ralston, T., "MIMI Terminology", Work in Progress,
Internet-Draft, draft-ralston-mimi-terminology-00, 11
April 2023, <https://datatracker.ietf.org/doc/html/draft-
ralston-mimi-terminology-00>.
[RFC1123] Braden, R., Ed., "Requirements for Internet Hosts -
Application and Support", STD 3, RFC 1123,
DOI 10.17487/RFC1123, October 1989,
<https://www.rfc-editor.org/rfc/rfc1123>.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November
2003, <https://www.rfc-editor.org/rfc/rfc3629>.
[RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data
Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006,
<https://www.rfc-editor.org/rfc/rfc4648>.
[RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms
(SHA and SHA-based HMAC and HKDF)", RFC 6234,
DOI 10.17487/RFC6234, May 2011,
<https://www.rfc-editor.org/rfc/rfc6234>.
[RFC8032] Josefsson, S. and I. Liusvaara, "Edwards-Curve Digital
Signature Algorithm (EdDSA)", RFC 8032,
DOI 10.17487/RFC8032, January 2017,
<https://www.rfc-editor.org/rfc/rfc8032>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/rfc/rfc8259>.
[RFC8785] Rundgren, A., Jordan, B., and S. Erdtman, "JSON
Canonicalization Scheme (JCS)", RFC 8785,
DOI 10.17487/RFC8785, June 2020,
<https://www.rfc-editor.org/rfc/rfc8785>.
10.2. Informative References
[DMLS] Chathi, H., "Decentralised MLS", 29 May 2023,
<https://gitlab.matrix.org/matrix-org/mls-ts/-
/blob/48efb972075233c3a0f3e3ca01c4d4f888342205/
decentralised.org>.
Ralston & Hodgson Expires 8 December 2023 [Page 25]
Internet-Draft Linearized Matrix API June 2023
[MSC3995] Ralston, T., "MSC3995: [WIP] Linearized Matrix", 12 April
2023, <https://github.com/matrix-org/matrix-spec-
proposals/pull/3995>.
Acknowledgments
Thank you to the Matrix Spec Core Team (SCT), and in particular
Richard van der Hoff, for exploring how Matrix rooms could be
represented as a linear structure, leading to this document.
Authors' Addresses
Travis Ralston
The Matrix.org Foundation C.I.C.
Email: travisr@matrix.org
Matthew Hodgson
The Matrix.org Foundation C.I.C.
Email: matthew@matrix.org
Ralston & Hodgson Expires 8 December 2023 [Page 26]