Network Working Group S. Nandakumar
Internet-Draft C. Jennings
Intended status: Informational Cisco
Expires: 27 April 2023 24 October 2022
Cached and Async meSsage Transport (CAST)
draft-nandakumar-mimi-transport-00
Abstract
This specification defines message transport, called CAST, based on
publish/subscribe semantics for interoperable inter-server messaging
that is rooted in modern, cloud friendly and scalable architectural
underpinnings.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 27 April 2023.
Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Motivation
2. Modern Messaging Federation Transport
2.1. Load Balancing and Reliability
3. Security Considerations
Appendix A. Acknowledgements
Authors' Addresses
1. Motivation
When designing federation systems for message exchange, one common
way to implement a messaging transport would follow architectural
patterns like "Send and Forget Architecture".
In this type of architecture, the requirement of the message
transport is to attempt to send the message from the source domain to
the target domain with no additional protections against losses in
transit or causes that render messages not being delivered.
As depicted in the diagram below, the happy path flow results in the
message being from alice@abc.com to be delivered to bob@xyz.com.
┌────────────┐ (2)Post:hi bob ┌────────────┐
│ │ │ │
┌──▶│ abc.com │──┬────────────▶ │ xyz.com │───┐
│ │ │ │ │ │ │
│ ───────────┘ │ └────────────┘ │
(1)Post:hi│bob ▲ │ (4│Post:hi bob
│ │ │ │
│ └────────┘ │
│ ▼
.─────────. (3)Delete:hi bob .─────────.
╱ ╲ ╱ ╲
(alice@abc.com) ( bob@xyz.com )
`. ,' `. ,'
`───────' `───────'
Figure 1: Send and Forget Pattern
One of the shortcomings of this architecture is the federation
message transport doesn't provide protections or options to recover
from reasons that might result in messages being undelivered. As
shown in the below depiction, when one/more resources at the target
domain becomes unavailable (due to server(s) failure, server down due
to maintenance), the message from alice@abc.com misses its delivery
to bob@xyz.com. Also to note, mechanisms to retry may be
unsuccessful under these circumstances. In such cases, the onerous
task is on the message sender to realize and retry/resend the
messages. This is either done out of band with human intervention
typically.
▮ ▮
▮▮ ▮▮
┌────────────┐ (2)Post:hi bob ┌───▮▮─▮▮────┐
│ │ │ ▮▮ │
┌──▶│ abc.com │──┬────────────▶ │ xy▮▮▮▮m │
│ │ │ │ │ ▮ ▮▮▮ │
│ ───────────┘ │ └──▮──────▮▮─┘
(1)Post:hi│bob ▲ │ ▮▮ ▮▮
│ │ │
│ └────────┘
│
.─────────. (3)Delete:hi bob .─────────.
╱ ╲ ╱ ╲
(alice@abc.com) ( bob@xyz.com )
`. ,' `. ,'
`───────' `───────'
Figure 2: Send and Forget Pattern, Failure Case
SMTP and XMPP based federation transports are typical examples where
"Send and Forget" architecture is employed.
This specification proposes a message interop transport suited for
modern messaging and cloud friendly architectures which intends to
address shortcoming with existing transports.
2. Modern Messaging Federation Transport
Any considerations for building interop messaging protocol for modern
messaging workloads needs to meet certain functional requirements as
listed below:
* Leverage modern cloud native architectures
* Scale to large number of messages and consumers
* Be less onerous on the message senders to ensure the delivery
* Easy to build high reliability cloud design
* Easy to build horizontal scalability cloud design
* Better aligned with internal architecture of some existing
solutions
* Easy to build gateways to existing APIs
CAST is publish/subscribe based interoperable messaging transport for
inter-server message delivery. Such a transport can be used for
delivering messages between servers within the same domain of
operation or for cross domain message delivery.
An example set of CAST messages exchanged for delivering messages for
the flow depicted are given below
(1)Add Subscription
messages/xyz.com/* -> xyz.com
▮ ▮ ▮
▮ ▮
▮ ▮ (0)Sub:messages/xyz.com/*
┌──────────▾─┐ ▮ ┌────────────┐
│ │◀──▮───────────────────────│ │
┌────▶│ abc.com │ │ xyz.com │─────┐
│ │ │───▮──────────────────────▶│ │ │
│ └───────────▲┘ ▮ ─────────┘ │
│ ▮ ▮ (4)Publish │
│ ▮ ▮ Name:... │
│ ▮▮▮ │
│ │
│ (3)Store:
│ Publish message (5)Publish
│ Name:messages/xyz.com/..
Msg:{body:"hi bob"....}
Name:messages/xyz.com/1
Msg:{body:"hi bob"..} │
▼
│ .─────────.
.─────────. ╱ ╲
╱ ╲ ( bob@xyz.com )
(alice@abc.com) `. ,'
`. ,' `───────'
`───────'
Figure 3: CAST Message Flow
In the example, the target domain, xyz.com, is interested in having
messages from the domain abc.com to be delivered. Such a setup may
be possible due to business relationships between the domains or any
entity within the target domain expresses such an interest.
1. The CAST endpoint in the domain xyz.com sends "Subscribe" message
to CAST endpoint in the "abc.com" indicating its interest to
receive any message that matches the name "messages/xyz.com/*",
i.e any message targetted to the domain "xyz.com"
2. On receiving the "Subscribe" message, the CAST entity at the
domain "abc.com" creates an active subscription entry for the
name "messages/xyz.com/*" against the domain "xyz.com". Thus
created subscriptions remain active until they expire or are
canceled. Subscriptions can be renewed periodically to keep them
active as well.
3. When Alice from abc.com publishes message to Bob at xyz.com, by
sending "Publish" message, the CAST entity (within abc.com)
performs the following steps on receiving the message: - Store
the message (with name messages/xyz.com/1) for at least for 24
hours. - Look up for any active subscriptions that matches the
name - Forward the message to the CAST endpoint that matches the
name from the previous step - In this example, since there exists
an active subscription for the pattern "messages/xyz.com/*",
Alice's message will be delivered to the CAST entity at "xyz.com"
based on the lookup result
4. On receiving the CAST Publish message, the CAST entity at the
domain "xyz.com" will have sufficient information in the message
to forward it to the right target within its domain, in this
case, to Bob
Messages within CAST are cached for at least 24 hours by default,
regardless of the status of message delivery. This allows, for
example, "xyz.com" to ask for the message again if the transaction to
store it is unsuccessful.
In scenarios where subscriber CAST entity is unavailable at the time
of the message delivery, the CAST entity resyncs its state by
reissuing a "Subscribe" message, when it's back in operation and thus
retrieve any cached messages as well as stream new messages that get
published in the future.
2.1. Load Balancing and Reliability
To horizontally scale to a huge volume of messages, the servers in
the abc.com domain can use any of the traditional techniques for load
balancing requests across a cluster of servers including DNS, IP
hashing, and others. Allowing the receiving domain, xyz.com, to load
balance the incoming message across many servers can be slightly more
complicated. The technique proposed here is to allow the
subscription to filter a subset of message and make sure there is a
sperate subscription for each incoming set. The filter criteria is
is done by taking the hash of the UUID for the message and computing
a modulo for it and checking it that matches a constant provide for
each subscription. For example if the xyz.com wanted to split the
data across 3 connection, it would specify a module of 3 and then
each connection would specify 0,1,2 respectively as the constant.
The fact that the abc.com domain retains the message for some period
of time and they can be requested a second time if needed allows the
messages processing to be pipelined and batched with no
acknowledgement which can greatly increase the speed of processing
some sort of transaction where the sender can discard the message
once the receivers acknowledge having it. The UUID in the messages
allow the receiver to easily deal with receiving the same message
more than once.
3. Security Considerations
The assumption is all messages are end to end encrypted and neither
domain can read the contents of any message between alice and bob.
The assumption is that a major mitigation of SPAM will be that alice
sends a connection request to bob and bob accepts that before any
messages with user generated content can be sent between alice and
bob.
Within the CAST architecture, the interacting domains are trusted to
deliver each other messages for their users and are bound by business
agreements that further constrain the rules related to use of
messages exchanged, dealing with spam and any other policies that
govern the successful federation. This is meant for major services
to connect to other major services and not designed to deal with the
issue of a domain with no business relationship to another domain
connected to it.
A given domain like abc.com does not reveal to xyz.com all the users
it has but if alice in abc sends a message to bob in xyz, it does
reveal to abc the existence of bob, and to xyz the existence of
alice.
Appendix A. Acknowledgements
TODO
Authors' Addresses
Suhas Nandakumar
Cisco
Email: snandaku@cisco.com
Cullen Jennings
Cisco
Email: fluffy@iii.ca