IBM's Shared Memory Communications over RDMA (SMC-R) Protocol
RFC 7609

Document Type RFC - Informational (August 2015; No errata)
Last updated 2015-08-14
Stream ISE
Formats plain text pdf html
IETF conflict review conflict-review-fox-tcpm-shared-memory-rdma
Stream ISE state Published RFC
Document shepherd Nevil Brownlee
Shepherd write-up Show (last changed 2015-04-06)
IESG IESG state RFC 7609 (Informational)
Telechat date
Responsible AD Martin Stiemerling
Send notices to mjfox@us.ibm.com, kassimis@us.ibm.com, sjerry@us.ibm.com, draft-fox-tcpm-shared-memory-rdma@ietf.org, "Nevil Brownlee" <rfc-ise@rfc-editor.org>
IANA IANA review state Version Changed - Review Needed
IANA action state No IC
Independent Submission                                            M. Fox
Request for Comments: 7609                                   C. Kassimis
Category: Informational                                       J. Stevens
ISSN: 2070-1721                                                      IBM
                                                             August 2015

     IBM's Shared Memory Communications over RDMA (SMC-R) Protocol

Abstract

   This document describes IBM's Shared Memory Communications over RDMA
   (SMC-R) protocol.  This protocol provides Remote Direct Memory Access
   (RDMA) communications to TCP endpoints in a manner that is
   transparent to socket applications.  It further provides for dynamic
   discovery of partner RDMA capabilities and dynamic setup of RDMA
   connections, as well as transparent high availability and load
   balancing when redundant RDMA network paths are available.  It
   maintains many of the traditional TCP/IP qualities of service such as
   filtering that enterprise users demand, as well as TCP socket
   semantics such as urgent data.

Status of This Memo

   This document is not an Internet Standards Track specification; it is
   published for informational purposes.

   This is a contribution to the RFC Series, independently of any other
   RFC stream.  The RFC Editor has chosen to publish this document at
   its discretion and makes no statement about its value for
   implementation or deployment.  Documents approved for publication by
   the RFC Editor are not a candidate for any level of Internet
   Standard; see Section 2 of RFC 5741.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc7609.

Fox, et al.                   Informational                     [Page 1]
RFC 7609      IBM's Shared Memory Communications over RDMA   August 2015

Copyright Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.

Table of Contents

   1. Introduction ....................................................5
      1.1. Protocol Overview ..........................................6
           1.1.1. Hardware Requirements ...............................8
      1.2. Definition of Common Terms .................................8
      1.3. Conventions Used in This Document .........................11
   2. Link Architecture ..............................................11
      2.1. Remote Memory Buffers (RMBs) ..............................12
      2.2. SMC-R Link Groups .........................................18
           2.2.1. Link Group Types ...................................18
           2.2.2. Maximum Number of Links in Link Group ..............21
           2.2.3. Forming and Managing Link Groups ...................23
           2.2.4. SMC-R Link Identifiers .............................24
      2.3. SMC-R Resilience and Load Balancing .......................24
   3. SMC-R Rendezvous Architecture ..................................26
      3.1. TCP Options ...............................................26
      3.2. Connection Layer Control (CLC) Messages ...................27
      3.3. LLC Messages ..............................................27
      3.4. CDC Messages ..............................................29
      3.5. Rendezvous Flows ..........................................29
           3.5.1. First Contact ......................................29
                  3.5.1.1. Pre-negotiation of TCP Options ............29
                  3.5.1.2. Client Proposal ...........................30
                  3.5.1.3. Server Acceptance .........................32
                  3.5.1.4. Client Confirmation .......................32
                  3.5.1.5. Link (QP) Confirmation ....................32
                  3.5.1.6. Second SMC-R Link Setup ...................35
                           3.5.1.6.1. Client Processing of ADD LINK
                                      LLC Message from Server ........35
                           3.5.1.6.2. Server Processing of ADD LINK
                                      Reply LLC Message from Client ..36
                           3.5.1.6.3. Exchange of RKeys on
                                      Second SMC-R Link ..............38
                           3.5.1.6.4. Aborting SMC-R and
                                      Falling Back to IP .............38
Show full document text