The Architecture of Direct Data Placement (DDP) and Remote Direct Memory Access (RDMA) on Internet Protocols
RFC 4296

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions
• 07

Network Working Group                                          S. Bailey
Request for Comments: 4296                                     Sandburst
Category: Informational                                        T. Talpey
                                                                  NetApp
                                                           December 2005

            The Architecture of Direct Data Placement (DDP)
      and Remote Direct Memory Access (RDMA) on Internet Protocols

Status of This Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2005).

Abstract

   This document defines an abstract architecture for Direct Data
   Placement (DDP) and Remote Direct Memory Access (RDMA) protocols to
   run on Internet Protocol-suite transports.  This architecture does
   not necessarily reflect the proper way to implement such protocols,
   but is, rather, a descriptive tool for defining and understanding the
   protocols.  DDP allows the efficient placement of data into buffers
   designated by Upper Layer Protocols (e.g., RDMA).  RDMA provides the
   semantics to enable Remote Direct Memory Access between peers in a
   way consistent with application requirements.

Bailey & Talpey              Informational                      [Page 1]
RFC 4296               DDP and RDMA Architecture           December 2005

Table of Contents

   1. Introduction ....................................................2
      1.1. Terminology ................................................2
      1.2. DDP and RDMA Protocols .....................................3
   2. Architecture ....................................................4
      2.1. Direct Data Placement (DDP) Protocol Architecture ..........4
           2.1.1. Transport Operations ................................6
           2.1.2. DDP Operations ......................................7
           2.1.3. Transport Characteristics in DDP ...................10
      2.2. Remote Direct Memory Access (RDMA) Protocol Architecture ..12
           2.2.1. RDMA Operations ....................................14
           2.2.2. Transport Characteristics in RDMA ..................16
   3. Security Considerations ........................................17
      3.1. Security Services .........................................18
      3.2. Error Considerations ......................................19
   4. Acknowledgements ...............................................19
   5. Informative References .........................................20

1.  Introduction

   This document defines an abstract architecture for Direct Data
   Placement (DDP) and Remote Direct Memory Access (RDMA) protocols to
   run on Internet Protocol-suite transports.  This architecture does
   not necessarily reflect the proper way to implement such protocols,
   but is, rather, a descriptive tool for defining and understanding the
   protocols.  This document uses C language notation as a shorthand to
   describe the architectural elements of DDP and RDMA protocols.  The
   choice of C notation is not intended to describe concrete protocols
   or programming interfaces.

   The first part of the document describes the architecture of DDP
   protocols, including what assumptions are made about the transports
   on which DDP is built.  The second part describes the architecture of
   RDMA protocols layered on top of DDP.

1.1.  Terminology

   Before introducing the protocols, certain definitions will be useful
   to guide discussion:

   o    Placement - writing to a data buffer.

   o    Operation - a protocol message, or sequence of messages, which
        provide an architectural semantic, such as reading or writing of
        a data buffer.

Bailey & Talpey              Informational                      [Page 2]
RFC 4296               DDP and RDMA Architecture           December 2005

   o    Delivery - informing any Upper Layer or application that a
        particular message is available for use.  Therefore, delivery
        may be viewed as the "control" signal associated with a unit of
        data.  Note that the order of delivery is defined more strictly
        than it is for placement.

   o    Completion - informing any Upper Layer or application that a
        particular operation has finished.  A completion, for instance,
        may require the delivery of several messages, or it may also
        reflect that some local processing has finished.

   o    Data Sink - the peer on which any placement occurs.

   o    Data Source - the peer from which the placed data originates.

   o    Steering Tag - a "handle" used to identify the buffer that is
        the target of placement.  A "tagged" message is one that
        references such a handle.

   o    RDMA Write - an Operation that places data from a local data