Internet Draft                                 Jim Pinkerton
Document: draft-pinkerton-rddp-security-00.txt   Microsoft Corporation
Expires: December, 2003                        Ellen Deleganes
                                                 Intel Corporation
                                               Bernard Aboba
                                                 Microsoft Corporation


                                               June 2003




                            DDP/RDMAP Security

1  Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

2  Abstract

   This document analyzes security issues around implementation and use
   of the Direct Data Placement Protocol(DDP) and Remote Direct Memory
   Access Protocol (RDMAP). It first defines an architectural model for
   an RDMA Network Interface Card (RNIC), which can implement DDP or
   RDMAP and DDP. The model includes a definition of resources that can
   be attacked. This document then introduces various Trust Models
   between a local peer and a remote peer and the tools that can be
   used to create countermeasures against attacks. Finally, the
   document reviews various attacks and the countermeasures to be used
   against them, grouping the attacks into spoofing, tampering,

J. Pinkerton, et al.    Expires December 2003                       1
Internet-Draft           RDDP/RDMAP Security              June 2003

   information disclosure, denial of service, and elevation of
   privilege.














































J. Pinkerton, et al.   Expires - December 2003               [Page 2]


Internet-Draft           RDDP/RDMAP Security              June 2003

   Table of Contents

   1    Status of this Memo.........................................1
   2    Abstract....................................................1
   2.1  Issues......................................................4
   3    Introduction................................................5
   4    Architectural Model.........................................7
   4.1  Components..................................................8
   4.2  Resources...................................................9
   4.2.1  Connection Context Memory..................................9
   4.2.2  Data Buffers...............................................9
   4.2.3  Page Translation Tables...................................10
   4.2.4  STag Namespace............................................10
   4.2.5  Completion Queues.........................................10
   4.2.6  RDMA Read Request Queue...................................10
   4.3  System Properties..........................................10
   5    Trust Models...............................................11
   6    Attacker Capabilities......................................13
   7    Attacks and Countermeasures................................14
   7.1  Tools for Countermeasures..................................14
   7.1.1  Protection Domain (PD)....................................14
   7.1.2  Limiting STag Scope.......................................14
   7.1.3  Access Rights.............................................15
   7.1.4  Limiting the Scope of the Completion Queue................15
   7.1.5  Limiting the Scope of an Error............................16
   7.2  Spoofing...................................................16
   7.2.1  Using an STag on a different connection...................16
   7.3  Tampering..................................................17
   7.3.1  RDMA Write or RDMA Read Response to Memory Outside of the
   Buffer 18
   7.3.2  Modifying a Buffer After Indicating the Contents Are Ready18
   7.4  Information Disclosure.....................................19
   7.4.1  Probing memory outside of the buffer bounds...............19
   7.4.2  Using RDMA Read to Access Stale Data......................19
   7.4.3  Accessing a buffer after the transfer is over.............20
   7.4.4  Accessing data within a valid STag that was unintended....20
   7.4.5  Using RDMA Read on a buffer meant only for RDMA Write.....20
   7.4.6  Using Multiple Stags to access the same buffer............21
   7.4.7  Remote node loading firmware onto the RNIC................21
   7.5  Denial of Service (DOS)....................................21
   7.5.1  RNIC Resource Consumption.................................21
   7.5.2  Resource Consumption By Active Applications...............22
   7.5.2.1   Receive Data Buffers..................................23
   7.5.2.2   Completion Queue (CQ) Resource Consumption............24
   7.5.2.3   RDMA Read Request Queue...............................26
   7.5.3  Resource Consumption by Idle Applications.................27
   7.5.4  Exercise of non-optimal code paths........................27

J. Pinkerton, et al.   Expires - December 2003               [Page 3]


Internet-Draft           RDDP/RDMAP Security              June 2003

   7.6  Elevation of Privilege.....................................28
   7.6.1  Loading Firmware into the RNIC............................28
   8    IPsec and RDDP.............................................29
   8.1  Introduction to IPsec......................................29
   8.2  Recommendations for IPsec Encapsulation of RDDP............30
   9    Summary Table of Attacks and Trust Models..................31
   10   References.................................................33
   10.1   Normative References......................................33
   10.2   Informative References....................................34
   11   AuthorÆs Addresses.........................................35
   12   Acknowledgments............................................36
   13   Full Copyright Statement...................................37


   Table of Figures

   Figure 1 û RDMA Security Model....................................7
   Figure 2 û Summary Attacks and Trust Model Table.................32



2.1  Issues

   This section is temporary and will go away when all issues have been
   resolved.

   Note: this is far from a complete list of issues; as more are
   raised, they will be added to this list until some sort of consensus
   is reached.  They are in the order found in the specification.

   Issue: IPsec recommendations for RDMAP/DDP.......................30

















J. Pinkerton, et al.   Expires - December 2003               [Page 4]


Internet-Draft           RDDP/RDMAP Security              June 2003

3  Introduction

   RDMA enables new levels of flexibility when communicating between
   two parties compared to current conventional networking practice
   (e.g. a stream-based model or datagram model). This flexibility
   brings new security issues that must be carefully understood when
   designing application protocols utilizing RDMA and when implementing
   RDMA-aware NICs (RNICs). Note that for the purposes of this security
   analysis, an RNIC may implement RDMAP and DDP, or just DDP.

   This specification is work in progress û the intent is to begin the
   discussion with a well thought out framework to analyze the issues.
   An area of particular concern is that there may be more attacks
   possible than have been enumerated here.

   The specification first develops an architectural model that is
   relevant for the security analysis û it details components,
   resources, and system properties that may be attacked. The
   specification then defines Trust Models. Trust is defined as:

        Trust - When one party depends upon the other party to not
        subvert the goals of the protocols, e.g., it will not attempt
        to perform the following attacks: spoofing, repudiation,
        information disclosure, denial of service, or elevation of
        privilege.

   An Untrusted peer is a party that may (or may not) attempt to
   perform one or more of the above attacks. A partially trusted peer
   (either the Local Peer or Remote Peer) may be trusted to not attempt
   to perform some subset of the above attacks, but not trusted to
   perform a different subset.

   For the Untrusted peer, a brief list of capabilities is enumerated.
   The rest of the specification is focused on analyzing attacks.
   First, the tools for mitigating attacks are listed, and then a
   series of attacks on components, resources, or system properties is
   enumerated. For each attack, possible countermeasures are reviewed.

   Applications within a host are divided into two categories û
   Privileged and Non-Privileged. Both application types can send and
   receive data and request resources. The key differences between the
   two are:

        The Privileged Application is partially trusted. It is assumed
        that the Privileged Application will not intentionally attack
        the system (e.g., it is a kernel application), although it may
        be greedy for resources.

J. Pinkerton, et al.   Expires - December 2003               [Page 5]


Internet-Draft           RDDP/RDMAP Security              June 2003

        A Non-Privileged ApplicationÆs capabilities are a logical sub-
        set of the Privileged ApplicationÆs. It is assumed by the local
        host infrastructure that a Non-Privileged Application is
        Untrusted. All Non-Privileged Application interactions with the
        RNIC Engine that could affect other applications need to be
        done through a Trusted intermediary that can verify the Non-
        Privileged Application requests.









































J. Pinkerton, et al.   Expires - December 2003               [Page 6]


Internet-Draft           RDDP/RDMAP Security              June 2003

4  Architectural Model

   This section describes an RDMA architectural reference model that is
   used as security issues are examined. It introduces the components
   of the model, the resources that can be attacked, and the system
   properties, which should be preserved when under attack.

   Figure 1 shows the components comprising the architecture and the
   interfaces where potential security attacks could be launched.
   External attacks can be injected into the system from an application
   that sits above the RI or from the Internet.


             +-------------+     Request Proxy Interface
             |  Privileged |<--------------------------------+
             |  Resource   |                                 |
    Admin<-->|  Manager    |     App Control Interface       |
             |             |<------+-------------------+     |
             +-------------+       |                   |     |
                   ^               v                   v     v
                   |         +-------------+   +-----------------+
                   |         | Privileged  |   |  Non-Privileged |
                   |         | Application |   |  Application    |
                   |         +-------------+   +-----------------+
                   |               ^                   ^
                   |Privileged     |Privileged         |Non-Privileged
                   |Control        |Data               |Data
                   |Interface      |Interface          |Interface
   RNIC            |               |                   |
   Interface(RI)   v               v                   v
   =================================================================

         +-----------------------------------------+
         |                                         |
         |               RNIC Engine               | <------ Firmware
         |                                         |
         +-----------------------------------------+
                             ^
                             |
                             v
                          Internet

                      Figure 1 û RDMA Security Model





J. Pinkerton, et al.   Expires - December 2003               [Page 7]


Internet-Draft           RDDP/RDMAP Security              June 2003

4.1  Components

   The components shown in Figure 1 û RDMA Security Model are:

       *   RNIC Engine û the component that implements the RDMA
           protocol and/or DDP protocol.

       *   Privileged Resource Manager û the component responsible for
           managing and allocating resources associated with the RNIC
           Engine. The Resource Manager does not send or receive data.

       *   Privileged Application û See Section 3 Introduction for a
           definition of Privileged Application. The local host
           infrastructure can enable the Privileged Application to map
           a data buffer directly from the RNIC Engine to the host
           through the RNIC Interface, but it does not allow the
           Privileged Application to directly consume RNIC Engine
           resources.

       *   Non-Privileged Application û See Section 3 Introduction for
           a definition of Non-Privileged Application. All Non-
           Privileged Application interactions with the RNIC Engine
           that could affect other applications MUST be done using the
           Privileged Resource Manager as a proxy.

   A design goal of the DDP and RDMAP protocols is to allow, under
   constrained conditions, Non-Privileged applications to send and
   receive data directly to/from the RDMA Engine without Privileged
   Resource Manager intervention - while ensuring that the host remains
   secure. Thus, one of the primary goals of this paper is to analyze
   this usage model for the enforcement that is required in the RNIC
   Engine to ensure the system remains secure.

   The host interfaces that could be exercised include:

       *   Control Interface - A Privileged Resource Manager uses the
           RI to allocate and manage RNIC Engine resources, control the
           state within the RNIC Engine, and monitor various events
           from the RNIC Engine. It also uses this interface to act as
           a proxy for some operations that a Non-Privileged
           Application may require (after performing appropriate
           countermeasures).

       *   Non-Privileged Data Transfer Interface û A Non-Privileged
           Application uses this interface to initiate and to check the
           status of data transfer operations.


J. Pinkerton, et al.   Expires - December 2003               [Page 8]


Internet-Draft           RDDP/RDMAP Security              June 2003

       *   Privileged Data Transfer Interface û A superset of the
           functionality provided by the Non-Privileged Data Transfer
           Interface. The application is allowed to directly manipulate
           RNIC Engine mapping resources to map an STag to an
           application data buffer.

       *   Request Proxy Interface û a Non-Privileged Application uses
           this interface to control RNIC Engine resources that could
           affect other applications û such as manipulating the RNIC
           Engine's mapping of an STag to an application data buffer.
           The Privileged Resource Manager implements countermeasures
           to ensure that if the Non-Privileged Application launches an
           attack it can prevent the attack from affecting other
           applications.

       *   Figure 1 also shows the ability to load new firmware in the
           RNIC Engine. Not all RNICs will support this, but it is
           shown for completeness and is also reviewed under potential
           attacks.

   If Internet control messages, such as ICMP, ARP, RIPv4, etc. are
   processed by the RNIC Engine, the threat analyses for those
   protocols is also applicable, but outside the scope of this paper.

4.2  Resources

   This section describes the primary resources in the RNIC Engine that
   could be affected if under attack. For RDMAP, all of the defined
   resources apply. For DDP, all of the resources except the RDMA Read
   Queue apply.

4.2.1  Connection Context Memory

   The state information for each connection is maintained in memory,
   which could be located in a number of places - on the NIC, inside
   RAM attached to the NIC, in host memory, or in any combination of
   the three, depending on the implementation.

4.2.2  Data Buffers

   There are two different ways to expose a data buffer; a buffer can
   be exposed for receiving RDMAP Send Type Messages (a.k.a. DDP
   Untagged Messages) on DDP Queue zero or the buffer can be exposed
   for remote access through STags (a.k.a. DDP Tagged Messages). This
   distinction is important because the attacks and the countermeasures
   used to protect against the attack are different depending on the
   method for exposing the buffer to the Internet.

J. Pinkerton, et al.   Expires - December 2003               [Page 9]


Internet-Draft           RDDP/RDMAP Security              June 2003

4.2.3  Page Translation Tables

   Page Translation Tables are the structures used by the RNIC to be
   able to access application memory for data transfer operations. Even
   though these structures are called "Page" Translation Tables, they
   may not reference a page at all û conceptually they are used to map
   an application address space representation of a buffer to the
   physical addresses that are used by the RNIC Engine to move data. If
   on a specific system, a mapping is not used, then a subset of the
   attacks examined may be appropriate.

4.2.4  STag Namespace

   The DDP specification defines a 32-bit namespace for the STag.
   Implementations may vary in terms of the actual number of STags that
   are supported. In any case, this is a bounded resource that can come
   under attack.

4.2.5  Completion Queues

   Completion Queues are used in this specification to conceptually
   represent how the RNIC Engine notifies the Application of the
   completion of the transmission of data, or the completion of the
   reception of data through the Data Transfer Interface. Because there
   could be many transmissions or receptions in flight at any one time,
   completions are modeled as a queue rather than a single event. An
   implementation may also use the Completion Queue to notify the
   application of other activities, for example, the completion of a
   mapping of an STag to a specific application buffer.

4.2.6  RDMA Read Request Queue

   The RDMA Read Request Queue is the memory holding state information
   for one or more RDMA Read Request Messages that have arrived, but
   for which the RDMA Read Response Messages have not yet been
   completely sent.

4.3  System Properties

   System properties that can be attacked included system integrity,
   system stability (liveness, large fluctuations in performance), and
   confidentiality.






J. Pinkerton, et al.   Expires - December 2003              [Page 10]


Internet-Draft           RDDP/RDMAP Security              June 2003

5  Trust Models

   The Trust Models described in this section have three primary
   distinguishing characteristics.

       *   Local Resource Sharing (yes/no) - When local resources are
           shared, they are shared across a grouping of RDMAP/DDP
           Streams. If local resources are not shared, the resources
           are dedicated on a per Stream basis. Resources are defined
           in Section 4.2 - Resources on page 9. The advantage of not
           sharing resources between Streams is that it reduces the
           number of types of attacks that are possible. The
           disadvantage is that applications might run out of
           resources.

       *   Local Trust (yes/no) û Local Trust is determined based on
           whether the local grouping of RDMAP/DDP Streams (which
           typically equates to one application or group of
           applications) mutually trust each other.

       *   Remote Trust (yes/no) û The Remote Trust level is determined
           based on whether the Local Peer of a specific RDMAP/DDP
           Stream trusts the Remote Peer of the Stream (using the same
           definition of trust as stated in the definition of Trust in
           Section 3 Introduction).

   It is assumed in this paper that the component that implements the
   mechanism to control sharing of RNIC Engine resources, is the
   Privileged Resource Manager. The RNIC Engine exposes its resources
   through the RI to the Privileged Resource Manager. All Privileged
   and Non-Privileged applications request resources from the Resource
   Manager. The Resource Manager implements resource management
   policies to ensure fair access to resources. The Resource Manager
   should be designed to take into account security attacks detailed in
   this specification.

   The sharing of resources across connections should be under the
   control of the application, both in terms of the Trust Model the
   application wishes to operate under, as well as the level of
   resource sharing the application wishes to give Local Peer
   processes.

   Not all of the combinations of the trust characteristics are
   expected to be used by applications. This paper specifically
   analyzes five application Trust Models that are expected to be in
   common use. The Trust Models are as follows:


J. Pinkerton, et al.   Expires - December 2003              [Page 11]


Internet-Draft           RDDP/RDMAP Security              June 2003

   1.  NS-NT - Non-Shared Local Resources, no Local Trust, no Remote
       Trust û typically a server application that wants to run in a
       mode that has the least number of potential attacks.

   2.  NS-RT - Non-Shared Local Resources, no Local Trust, Remote Trust
       û typically a peer-to-peer application, which has, by some
       method outside of the scope of this specification, authenticated
       the Remote Peer. <TBD: mention authentication>

   3.  S-NT - Shared Local Resources, no Local Trust, no Remote Trust û
       typically a server application that runs in an untrusted
       environment where the amount of resources required is either too
       large or too dynamic to dedicate for each RDMAP/DDP Stream.

   4.  S-LT - Shared Local Resources, Local Trust, no Remote Trust û
       typically an application, which provides a session layer and
       uses multiple Streams, to provide additional throughput or fail-
       over capabilities. All of the Streams within the local
       application trust each other, but do not trust the remote peer.

   5.  S-T - Shared Local Resources, Local Trust, Remote Trust û
       typically a distributed application, such as a distributed
       database application or a High Performance Computer (HPC)
       application, which is intended to run on a cluster. Due to
       extreme resource and performance requirements, the application
       typically authenticates with all of its peers and then runs in a
       highly trusted environment. The application peers are all in a
       single application fault domain and depend on one another to be
       well-behaved when accessing data structures. If a trusted Remote
       Peer has an implementation defect that results in poor behavior,
       the application could be corrupted. <TBD: mention
       authentication>

   Models NS-NT and S-NT above are typical for Internet networking û
   neither other Local Peers nor the Remote Peer is trusted. Sometimes
   optimizations can be done that enable sharing of Page Translation
   Tables across multiple Local Peers, thus Model S-LT can be
   advantageous. Model S-T is typically used when resource scaling
   across a large parallel application makes it infeasible to use any
   other model. Resource scaling issues can either be due to
   performance around scaling or because there simply are not enough
   resources. Model NS-RT is probably the least likely model to be
   used, but is presented for completeness.





J. Pinkerton, et al.   Expires - December 2003              [Page 12]


Internet-Draft           RDDP/RDMAP Security              June 2003

6  Attacker Capabilities

   An attackerÆs capabilities delimit the types of attacks that
   attacker is able to launch. RDMAP and DDP require that the initial
   LLP Stream (and connection) be set up prior to transferring
   RDMAP/DDP Messages. For the attacker to actively generate an
   RDMAP/DDP protocol attack, it must have the capability to both send
   and receive messages. Attackers with send only capabilities should
   be addressed by the LLP, not by RDMAP/DDP.







































J. Pinkerton, et al.   Expires - December 2003              [Page 13]


Internet-Draft           RDDP/RDMAP Security              June 2003

7  Attacks and Countermeasures

   This section describes the attacks that are possible against the
   RDMA system defined in Figure 1 û RDMA Security Model and the RNIC
   Engine resources defined in Section 4.2. The analysis includes a
   detailed description of each attack, the Trust Models the attack
   applies to (see Section 5 for a description of the Trust Models),
   and a description of the countermeasures appropriate to the Trust
   Model(s) that can be taken to thwart the attack.

   Note that, connection setup and teardown is presumed to be done in
   stream mode (i.e. no RDMA encapsulation of the payload), so there
   are no new attacks related to connection setup/teardown beyond what
   is already present in the LLP (e.g. TCP or SCTP). Consequently, any
   existing analysis of Spoofing, Tampering, Repudiation, Information
   Disclosure, Denial of Service, or Elevation of Privilege continues
   to apply. Thus, the analysis in this section focuses on attacks that
   are present regardless of the LLP Stream type.

7.1  Tools for Countermeasures

   The tools described in this section are the primary mechanisms that
   can be used to provide countermeasures to potential attacks.

7.1.1  Protection Domain (PD)

   Protection Domains are associated with two of the resources of
   concern, connection context memory and STags associated with data
   buffers. Protection Domains are used mainly to ensure that an STag
   can only be used to access the associated data buffer through
   connections in the same Protection Domain as that STag.

   For the Trust Models that are defined to have non-shared resources
   (Trust Models NS-NT and NS-RT), it is recommended that each Stream
   be associated with its own, unique Protection Domain. For those
   Trust Models where resources are shared (Trust Models S-NT, S-LT and
   S-T), it is recommended that Protection Domain be limited to the
   number of Streams that share the same Trust Model.

7.1.2  Limiting STag Scope

   The key to protecting a local data buffer is to limit the scope of
   its STag to the level appropriate for the Trust Model. The scope of
   the STag can be measured in multiple ways.

       *   Number of Connections and/or Streams on which the STag is
           valid. One way to limit the scope of the STag is to limit

J. Pinkerton, et al.   Expires - December 2003              [Page 14]


Internet-Draft           RDDP/RDMAP Security              June 2003

           the connections and/or Streams that are allowed to use the
           STag. As noted in the previous section, use of Protection
           Domains appropriately can limit the scope of the STag. It is
           also possible to create an STag that is valid only on a
           single connection, even in the case where several
           connections are associated with the Protection Domain of the
           STag.

       *   Limit the time an STag is valid. By Invalidating an
           Advertised STag (e.g., revoking remote access to the buffers
           described by an STag when done with the transfer), an entire
           class of attacks can be eliminated.

       *   Limit the buffer the STag can reference. Limiting the scope
           of an STag access to *just* the intended buffers to be
           exposed is critical to prevent certain forms of attacks.

7.1.3  Access Rights

   Access Rights associated with a specific Advertised STag or
   RDMAP/DDP Stream provide another mechanism for applications to limit
   the attack capabilities of the Remote Peer. The Local Peer can
   control whether a data buffer is exposed for local only, or local
   and remote access, and assign specific access privileges (read,
   write, read and write).

   For DDP, when an STag is advertised, the Remote Peer is presumably
   given write access rights to the data (otherwise there was not much
   point to the advertisement). For RDMAP, when an application
   advertises an STag, it can enable write-only, read-only, or both
   write and read access rights.

   Similarly, some applications may wish to provide a single buffer
   with different access rights on a per-connection or per-Stream
   basis. For example, some connections may have read-only access, some
   may have remote read and write access, while on other connections
   only the Local Peer is allowed access.

7.1.4  Limiting the Scope of the Completion Queue

   Completions associated with sending and receiving data, or setting
   up buffers for sending and receiving data, could be accumulated in a
   shared Completion Queue for a group of RDMAP/DDP Streams, or a
   specific RDMAP/DDP Stream could have a dedicated Completion Queue.
   Limiting Completion Queue association to one, or a small number of
   RDMAP/DDP Streams can prevent several forms of Denial of Service
   attacks.

J. Pinkerton, et al.   Expires - December 2003              [Page 15]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.1.5  Limiting the Scope of an Error

   To prevent a variety of attacks, it is important that an RDMAP/DDP
   implementation be robust in the face of errors. If an error on a
   specific Stream can cause other unrelated Streams to fail, then a
   broad class of attacks are enabled against the implementation.

7.2  Spoofing

   Because the RDMAP Stream is only offloaded if it is in the
   ESTABLISHED state, certain types of traditional forms of wire
   attacks do not apply -- an end-to-end handshake must have occurred
   to establish the RDMAP Stream. So, the only form of spoofing that
   applies is one when a remote node can both send and receive packets.

   If a man-in-the-middle has the ability to inject packets, which will
   be accepted by the LLP (e.g., TCP sequence number is correct), one
   style of attack is for the man-in-the-middle to send Tagged Messages
   (either RDMAP or DDP). If it can discover a buffer that has been
   exposed for STag enabled access, then the man-in-the-middle can use
   an RDMA Read operation to read the contents of the associated data
   buffer, perform an RDMA Write Operation to modify the contents of
   the associated data buffer, or invalidate the STag to disable
   further access to the buffer. The only countermeasure for this form
   of attack is to either secure the RDMAP/DDP Stream or attempt to
   provide physical security to prevent man-in-the-middle type access.

   The best protection against this form of attack is end-to-end
   authentication, such as IPsec (see Section 8 IPsec and RDDP on page
   29), to prevent spoofing or tampering. If authentication is not
   used, then a man-in-the-middle attack can occur, enabling spoofing,
   tampering, and repudiation.

   Another approach is to restrict access to only the local
   subnet/link, and provide some mechanism to limit access, such as
   physical security or 802.1.x. This model is an extremely limited
   deployment scenario, and will not be further examined here.

7.2.1  Using an STag on a different connection

   One style of attack from the Remote Peer is for it to attempt to use
   STag values that it is not authorized to use. Note that, if the
   Remote Peer sends an invalid STag to the Local Peer, per the DDP and
   RDMAP specifications, the connection must be torn down. Thus, the
   threat exists if an STag has been enabled for Remote Access on one
   connection and a Remote Peer is able to use it on an unrelated
   connection. If the attack is successful, the attacker could

J. Pinkerton, et al.   Expires - December 2003              [Page 16]


Internet-Draft           RDDP/RDMAP Security              June 2003

   potentially be able to perform either RDMA Read Operations to read
   the contents of the associated data buffer, perform RDMA Write
   Operations to modify the contents of the associated data buffer, or
   to Invalidate the STag to disable further access to the buffer.

   An attempt by a Remote Peer to access a buffer with an STag on a
   different connection in the same Protection Domain may or may not be
   an attack depending on the Trust Model employed by the application.
   For Trust Model S-T, where resources are shared between connections,
   and both Local and Remote Peers are Trusted, using an STag on
   multiple connections within the same Protection Domain is allowed,
   and could be desired behavior. For the other four Trust Models where
   the Remote Peer is not Trusted, and/or resources are not intended to
   be shared, attempting to use an STag on a different connection could
   be considered to be an attack.

   In the case where the Trust Model is defined with no shared
   resources between connections (Trust Models NS-NT and NS-RT), this
   attack can be defeated by assigning each connection to a different
   Protection Domain. Before allowing remote access to the buffer, the
   Protection Domain of the connection where the access attempt was
   made is matched against the Protection Domain of the STag. If the
   Protection Domains do not match, access to the buffer is denied, an
   error is generated, and the RDMAP Stream associated with the
   attacking connection should be terminated. Thus, for Trust Models
   NS-NT and NS-RT, it is RECOMMENDED that each connection be in a
   separate Protection Domain.

   For Trust Models S-NT and S-LT, where resources are shared, but the
   Remote Peers are Untrusted, it may not be practical to separate each
   connection into its own Protection Domain. In this case, the
   application can still limit the scope of any of the STags it is
   enabling for remote access to a single connection. If the STag scope
   has been limited to a single connection, any attempt to use that
   STag on a different connection will result in an error, and the RDMA
   Stream associated with that connection should be terminated. Thus,
   for Trust Models S-NT and S-LT, it is RECOMMENDED that the scope of
   an STag be limited to a single connection.

7.3  Tampering

   A Remote Peer can attempt to tamper with the contents of data
   buffers on a Local Peer that have been enabled for remote write
   access. The types of tampering attacks that are possible are
   outlined in the sections that follow.



J. Pinkerton, et al.   Expires - December 2003              [Page 17]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.3.1  RDMA Write or RDMA Read Response to Memory Outside of the Buffer

   This attack is an attempt by the Remote Peer to perform an RDMA
   Write or RDMA Read Response to memory outside of the valid length
   range of the data buffer enabled for remote write access. This
   attack applies primarily to Trust Models with Untrusted Remote Peers
   (NS-NT, S-NT and S-LT), and can occur even when no resources are
   shared across connections.

   The countermeasure for this type of attack must be in the RNIC
   implementation, using the STag. When the Local Peer specifies to the
   RI, the base and the number of bytes in the buffer that it wishes to
   make accessible, the RI must ensure that the base and bounds check
   are applied to any access to the buffer referenced by the STag
   before the STag is enabled for access. When an RDMA data transfer
   operation (which includes an STag) arrives on a connection, a base
   and bounds byte granularity access check must be performed to ensure
   the operation accesses only memory locations within the buffer
   described by that STag.

   Thus, it is RECOMMENDED that an RI implementation ensure that a
   Remote Peer, regardless of Trust Model, will not be able to access
   memory outside of the buffer specified when the STag was enabled for
   remote access.

7.3.2  Modifying a Buffer After Indicating the Contents Are Ready

   This attack occurs if a Remote Peer attempts to modify the contents
   by performing an RDMA Write or an RDMA Read Response after it had
   indicated to the Local Peer that the data buffer contents were ready
   for use.

   This attack applies primarily to the Trust Models where the Remote
   Peers are not Trusted (Trust Models NS-NT, S-NT and S-LT), and can
   occur even when no resources are shared across connections. Note
   that, an error on the part of a Trusted Remote Peer could also
   result in this problem.

   The Local Peer can protect itself from this type of attack by
   revoking remote access when the original data transfer has completed
   and before it validates the contents of the buffer. The Local Peer
   can either do this by explicitly revoking remote access rights for
   the STag when the Remote Peer indicates the operation has completed,
   or by checking to make sure the Remote Peer Invalidated the STag
   through the RDMAP Invalidate capability, and if it did not, the
   Local Peer then explicitly revokes the STag remote access rights.


J. Pinkerton, et al.   Expires - December 2003              [Page 18]


Internet-Draft           RDDP/RDMAP Security              June 2003

   It is RECOMMENDED that the Local Peer follow the above procedure for
   Trust Models NS-NT, S-NT, and S-LT to protect the buffer. The Local
   Peer MAY also wish to use this procedure for Trust Models NS-RT and
   S-T to protect itself from unintended tampering due to an error in
   the Remote Peer.

7.4  Information Disclosure

   The main potential source for information disclosure is through a
   local buffer that has been enabled for remote access. If the buffer
   can be probed by a Remote Peer on another connection, then there is
   potential for information disclosure.

   Information disclosure attacks mainly apply to the Trust Models that
   include Untrusted Remote Peers (Trust Models NS-NT, S-NT, and S-LT
   as defined in Section 5). Trusted Remote Peers are assumed not to
   purposely attempt such attacks - any attempt is assumed to be due to
   an error or other unexpected failure in the Remote Peer.

   The potential attacks that could result in unintended information
   disclosure and countermeasures are as follows:

7.4.1  Probing memory outside of the buffer bounds

   This is essentially the same attack as described in Section 7.3.1,
   except an RDMA Read Request is used to mount the attack. The same
   countermeasure applies.

7.4.2  Using RDMA Read to Access Stale Data

   If a buffer is being used for a combination of reads and writes
   (either remote or local), and is exposed to the Remote Peer with at
   least remote read access rights, the Remote Peer may be able to
   examine the contents of the buffer before they are initialized with
   the correct data. In this situation, whatever contents were present
   in the buffer before the buffer is initialized can be viewed by the
   Remote Peer, if the Remote Peer performs an RDMA Read. This threat
   applies to Trust Models NS-NT, S-NT, and S-LT.

   Because of this, it is RECOMMENDED that the Local Peer ensure that
   no stale data is contained in the buffer when remote read access
   rights are initially granted (this can be done by zeroing the
   contents of the memory, for example).





J. Pinkerton, et al.   Expires - December 2003              [Page 19]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.4.3  Accessing a buffer after the transfer is over

   If the Remote Peer has remote read access to a buffer, and by some
   mechanism tells the Local Peer that the transfer has been completed,
   but the Local Peer does not disable remote access to the buffer
   before modifying the data, it is possible for the Remote Peer to
   retrieve the new data.

   This is similar to the attack defined in Section 7.3.2 Modifying a
   Buffer After Indicating the Contents Are Ready on page 18. The same
   countermeasures apply. In addition, it is RECOMMENDED that the Local
   Peer should grant remote read access rights only for the amount of
   time needed to retrieve the data.

7.4.4  Accessing data within a valid STag that was unintended

   <TBD: 7.4.4 and 7.4.5 are the same û group together>

   If the Local Peer enables remote access to a buffer using an STag
   that references the entire buffer, but intends only a portion of the
   buffer to be accessed, it is possible for the Remote Peer to access
   the other parts of the buffer anyway. This threat applies to Trust
   Models NS-NT, S-NT, and S-LT.

   To prevent this attack, it is RECOMMENDED that the Local Peer set
   the base and bounds of the buffer when the STag is initialized to
   expose only the data to be retrieved.

7.4.5  Using RDMA Read on a buffer meant only for RDMA Write

   One form of disclosure can occur if the access rights on the buffer
   were set for remote read, when only remote write access was
   intended. This attack applies primarily to Trust Models with
   Untrusted Remote Peers (NS-NT, S-NT and S-LT). If the buffer
   contained application data, or data from a transfer on an unrelated
   connection, the Remote Peer could retrieve the data through an RDMA
   Read operation.

   The most obvious countermeasure for this attack is to not grant
   remote read access if the buffer is intended to be write-only. The
   Remote Peer would not be able to retrieve data associated with the
   buffer. An attempt to do so would result in an error and the RDMAP
   Stream associated with the connection would be terminated.

   Thus, it is RECOMMENDED that if an application only intends a buffer
   to be exposed for remote write access, it set the access rights to
   the buffer to only enable remote write access.

J. Pinkerton, et al.   Expires - December 2003              [Page 20]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.4.6  Using Multiple Stags to access the same buffer

   Multiple STags accessing the same buffer at the same time can result
   in unintentional information disclosure. When one STag has remote
   read access enabled and a different STag has remote write access
   enabled to the same buffer, it is possible for one connection to
   view the contents that have been written by another Remote Peer.
   This is not a problem when all of the STags accessing the buffer
   have only remote read enabled. For Trust Models NS-NT, S-NT, S-LT it
   is RECOMMENDED that multiple Remote Peers not be granted access to
   the same buffer through different STags at the same time. A buffer
   should be exposed to only one Untrusted Remote Peer at a time to
   ensure that no information disclosure occurs between peers.

7.4.7  Remote node loading firmware onto the RNIC

   If the Remote Peer can cause firmware to be loaded onto the RNIC,
   there is an opportunity for information disclosure. See Elevation of
   Privilege in Section 7.6 for this analysis.

7.5  Denial of Service (DOS)

   A DOS attack is one of the primary security risks of RDMAP. This is
   because RNIC resources are valuable and scarce, and many application
   environments require communication with Untrusted Remote Peers. If
   the remote application can be authenticated or encrypted, clearly,
   the DOS profile can be reduced. For the purposes of this analysis,
   it is assumed that the RNIC must be able to operate in Untrusted
   environments, which are open to DOS style attacks.

   Denial of service attacks against RI resources are not the typical
   unknown party spraying packets at a random host (such as a TCP SYN
   attack). Because the connection must be fully established, the
   attacker must be able to both send and receive messages over that
   connection, or be able to guess a valid packet on an existing RDMAP
   Stream.

   This section outlines the potential attacks and the countermeasures
   available for dealing with each attack. For each attack, the Trust
   Model that it applies to is described.

7.5.1  RNIC Resource Consumption

   This section covers attacks that fall into the general category of a
   Local Peer attempting to unfairly allocate scarce RNIC resources.
   The Local Peer may be attempting to allocate resources on its own
   behalf, or on behalf of a Remote Peer. Resources that fall into this

J. Pinkerton, et al.   Expires - December 2003              [Page 21]


Internet-Draft           RDDP/RDMAP Security              June 2003

   category include: Protection Domains, Connection Context Memory,
   Translation and Protection Tables, and STag namespace. These can be
   attacks by currently active Local Peers or ones that allocated
   resources earlier, but are now idle.

   These attacks generally apply to any Trust Model that includes
   Untrusted Local Peers (Trust Models NS-NT, NS-RT and S-NT). This
   type of attack can occur even when resources are not shared across
   connections.

   It is RECOMMENDED that the allocation of all scarce resources be
   placed under the control of a Resource Manager. This allows the
   Resource Manager to:

       *   prevent a Local Peer from allocating more than its fair
           share of resources, and

       *   detect if a Remote Peer is attempting to launch a DOS attack
           by attempting to create an excessive number of connections
           and take corrective action (such as refusing the request or
           applying network layer filters against the Remote Peer).

   Note that, placing scarce resource management under the control of a
   Resource Manager also prevents other Trusted Local Peers from
   attempting to allocate more than their fair share of resources.

   This analysis assumes that the Resource Manager is responsible for
   handing out Protection Domains, and RNIC implementations will
   provide enough Protection Domains to allow the Resource Manager to
   be able to assign a unique Protection Domain for each unrelated,
   Untrusted Local Peer (for a bounded, reasonable number of Local
   Peers). This analysis further assumes that the Resource Manager
   implements policies to ensure that Untrusted Local Peers are not
   able to consume all of the Protection Domains through a DOS attack.
   Note that Protection Domain consumption cannot result from a DOS
   attack launched by a Remote Peer, unless a Local Peer is acting on
   the Remote PeerÆs behalf.

7.5.2  Resource Consumption By Active Applications

   This section describes DOS attacks from Local and Remote Peers that
   are actively exchanging messages. Attacks on each RDMA NIC resource
   are examined, including the Trust Models that apply, and the
   specific countermeasures. Note that, attacks on Connection Context
   Memory, Page Translation Tables, and STag namespace are covered in
   Section 7.5.1 RNIC Resource Consumption, so are not included here.


J. Pinkerton, et al.   Expires - December 2003              [Page 22]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.5.2.1  Receive Data Buffers

   The Remote Peer can attempt to consume more than its fair share of
   receive data buffers (Untagged DDP buffers or for RDMAP buffers
   consumed with Send Type Messages). If resources are not shared
   across multiple connections (Trust Models NS-NT, NS-RT), then this
   attack is not possible because the Remote Peer will not be able to
   consume more buffers than were allocated to the Stream. The worst
   case scenario is that the Remote Peer can consume more receive
   buffers than the Local Peer allowed, resulting in no buffers to be
   available, which would cause the Remote PeerÆs connection to the
   Local Peer to be torn down.

   If local receive data buffers are shared among multiple Streams and
   the Remote Peer is not Trusted (Trust Models S-NT, S-LT), then the
   Remote Peer can attempt to consume more than its fair share of the
   receive buffers, causing a different Stream to be short of receive
   buffers, thus possibly causing the other Stream to be torn down.

   One method the Local Peer could use is to recognize that a Remote
   Peer is attempting to use more than its fair share of resources and
   terminate the Stream. However, if the Local Peer is sufficiently
   slow, it may be possible for the Remote Peer to still mount a denial
   of service attack. An RNIC Engine implementation that enables a more
   robust countermeasure is one that provides high and low-water
   notifications to enable the Local Peer to detect and prevent DOS
   attacks against shared data buffers. If a low-water notification is
   enabled, and the Local Peer is able to bound the amount of time that
   it takes to replenish receive buffers, and the Local Peer maintains
   statistics to determine which Remote Peer is consuming buffers, then
   the Local Peer can size the amount of local receive buffers posted
   on the receive queue such that the low-water notification can arrive
   before resources are depleted and corrective action can be taken
   (e.g., terminate the Stream of the attacking Remote Peer). Enabling
   the high-water notification can help the Local Peer detect a Remote
   Peer that is launching an attack by sending a large number of out-
   of-order packets. The notification is generated when more than the
   specified number of buffers are in process simultaneously on a
   Stream (i.e., packets have started to arrive for the buffer, but
   have not yet been delivered to the ULP).

   A different countermeasure is for the RNIC Engine to provide the
   capability to limit the Remote PeerÆs ability to consume receive
   buffers on a per Stream basis. Unfortunately this requires a large
   amount of state to be tracked on a per RNIC basis.



J. Pinkerton, et al.   Expires - December 2003              [Page 23]


Internet-Draft           RDDP/RDMAP Security              June 2003

   Thus, if an RNIC Engine provides the ability to share receive
   buffers across multiple Streams, it is RECOMMENDED that it enable
   the Local Peer to detect if the Remote Peer is attempting to consume
   more than its fair share of resources so that the application can
   apply countermeasures to detect and prevent the attack.

7.5.2.2  Completion Queue (CQ) Resource Consumption

   DOS attacks against the Completion Queue can be caused by either the
   Local Peer or the Remote Peer if either attempts to cause more
   completions than its fair share, thus potentially starving another
   unrelated Stream such that no Completion Queue entries are
   available.

   A Completion Queue entry can potentially be consumed by a completion
   from the send queue or a receive completion. In the former, the
   attacker is the Local Peer (Trust Models S-NT). In the later, the
   attacker is the Remote Peer (S-NT, S-LT).

   The potential attacks and the countermeasures for each are described
   in the subsections that follow.

7.5.2.2.1   Local Peer Attacking a Shared CQ

   A form of attack can occur for Trust Models NS-NT, NS-RT, and S-NT,
   where the Local Peers are Untrusted, and Local Peers can consume
   resources on the CQ. Sharing a CQ across connections that belong to
   different Protection Domains is NOT RECOMMENDED in cases where any
   of the Local Peers are Untrusted. A Local Peer that is slow to free
   resources on the CQ by not reaping the completion status quick
   enough could stall all other Local Peers attempting to use that CQ.

   One of two countermeasures can be used to avoid this kind of attack.
   The first is to use a different CQ per Untrusted Local Peer. The
   other is to use a Trusted Local Peer to act as a third party to free
   resources on the CQ and place the status in intermediate storage
   until the Untrusted Local Peer reaps the status information.

7.5.2.2.2   Remote Peer Attacking a Shared CQ

   The Remote Peer can attack a CQ by consuming more than its fair
   share of CQ entries by using one of two methods. The first method
   can only be used if the ULP protocol allows the Remote Peer to
   reserve a specified number of CQ entries, possibly leaving
   insufficient entries for other connections that are sharing the CQ.
   The other method is if the Remote Peer can attack the CQ by


J. Pinkerton, et al.   Expires - December 2003              [Page 24]


Internet-Draft           RDDP/RDMAP Security              June 2003

   overwhelming the CQ with completions, which can affect completion
   processing on other Streams sharing that connection.

   The first method of attack can be avoided if the ULP does not allow
   a Remote Peer to reserve CQ entries. This is RECOMMENDED
   particularly for Trust Models S-NT and S-LT, with shared resources
   and Untrusted Remote Peers. If a Local Peer allows this type of
   resource allocation, and it has any Untrusted Remote Peers, then the
   Local Peer it is RECOMMENDED that the CQ be isolated to connections
   within a single Protection Domain.

   One way that a Remote Peer can attempt to overwhelm its CQ with
   completions is by sending minimum length RDMAP/DDP Messages to cause
   as many completions per second as possible. Assuming that the Local
   Peer does not run out of receive buffers (if they do, then this is a
   different attack, documented in Section 7.5.2.1 Receive Data Buffers
   on page 23), then it might be possible for the Remote Peer to
   consume more than its fair share of Completion Queue entries.
   Depending upon the CQ implementation, this could either cause the CQ
   to overflow (if it is not large enough to handle all of the
   completions generated) or for another Stream to not be able to
   generate CQ entries (if the RNIC had flow control on generation of
   CQ entries into the CQ). In either case, the CQ will stop
   functioning correctly and any connections expecting completions on
   the CQ will stop functioning.

   This attack can occur regardless of whether all of the connections
   associated with the CQ are in the same Protection Domain or are in
   different Protection Domains. Because this attack assumes a shared
   local resource and an Untrusted Remote Peer, Trust Models S-NT, S-LT
   apply.

   The Local Peer can protect itself from this type of attack using
   either of the following methods:

       *   Resize the CQ to the appropriate level(note that resizing
           the CQ can fail, so the CQ resize should be done before
           sizing the buffers on the connection), OR

       *   Grant fewer resources than the Remote Peer requested (not
           supplying the number of Receive Data Buffers requested).

   The proper sizing of the CQ is dependent on the Trust Model. For the
   Trust Model described in Section 5, with Trusted Local Peers and
   Untrusted Remote Peers (Trust Model S-LT), a correctly sized CQ
   means that the CQ is large enough to hold completion status for all
   of the outstanding Receive Data Buffers, or:

J. Pinkerton, et al.   Expires - December 2003              [Page 25]


Internet-Draft           RDDP/RDMAP Security              June 2003

   CQ_MIN_SIZE = SUM(MaxPostedOnEachRQ)
                + SUM(MaxPostedOnEachS-RQ)
                + SUM(MaxPostedOnEachSQ)

   If the Trust Model assumes neither the Local Peer nor the Remote
   Peer is trusted (Trust Model S-NT or S-LT), then the CQ must be
   sized to accommodate the maximum number of operations or Receive
   Data Buffers that it is possible to post at any one time, thus the
   equation becomes:

            CQ_MIN_SIZE = SUM(SizeOfEachRQ)
                          + SUM(SizeOfEachS-RQ)
                          + SUM(SizeOfEachSQ)

   Where MaxPosted*OnEach*Q and SizeOfEach*Q varies on a per connection
   or per Shared Receive Queue basis.

7.5.2.3  RDMA Read Request Queue

   Two types of attacks are possible against resources associated with
   RDMA Read Request Queues. One style of attack can only occur when
   the RDMA Read Request Queue resources are pooled across multiple
   connections. This attack occurs when an Untrusted Local Peer
   attempts to unfairly allocate RDMA Read Request Queue resources for
   its connections.

   It is RECOMMENDED that access to interfaces that allocate RDMA Read
   Request Queue entries be restricted to a Trusted Local Peer, such as
   a Resource Manager. The Resource Manager should prevent a Local Peer
   from allocating more than its fair share of resources.

   Another form of attack is the Remote Peer sending more RDMA Read
   Requests than the depth of the RDMA Read Request Queue at the Local
   Peer.

   This attack can be prevented by properly configuring the connection
   when the connection is established. The Remote PeerÆs end of the
   connection should be configured by a trusted agent such that the
   RNIC will not transmit RDMA Read Requests that exceed the depth of
   the RDMA Read Request Queue at the Local Peer. If the connection is
   correctly configured, and if the Remote Peer submits more requests
   than the Local PeerÆs RDMA Read Request Queue can handle, the
   requests will be queued at the Remote PeerÆs connection until
   previous requests complete. If the Remote PeerÆs connection is not
   configured correctly, the RDMAP Stream for that connection is
   terminated when more RDMA Read Requests arrive at the Local Peer


J. Pinkerton, et al.   Expires - December 2003              [Page 26]


Internet-Draft           RDDP/RDMAP Security              June 2003

   than the Local Peer can handle. Thus, the Remote Peer is able to
   only affect its own connection.

7.5.3  Resource Consumption by Idle Applications

   The simplest form of a DOS attack given a fixed amount of resources
   is for the Remote Peer to create a RDMAP Stream to a Local Peer,
   request dedicated resources then do no actual work. This allows the
   Remote Peer to be very light weight (i.e. only negotiate resources,
   but do no data transfer) and consumes a disproportionate amount of
   resources in the server.

   A general countermeasure for this style of attack is to monitor
   active RDMAP Streams and if resources are getting low, reap the
   resources from RDMAP Streams that are not transferring data and
   possibly terminate the connection. This needs to be under
   administrative control, and demonstrates the need for a MIB for
   RDMAP so this condition can be detected and acted upon.

   Refer to Section 7.5.1 for the analysis and countermeasures for this
   style of attack on the following RNIC resources: Connection Context
   Memory, Page Translation Tables and STag namespace.

   Note that some RNIC resources are not at risk of this type of attack
   from a Remote Peer because an attack requires the Remote Peer to
   send messages in order to consume the resource. Receive Data
   Buffers, Completion Queue, and RDMA Read Request Queue resources are
   examples. These resources are, however, at risk form a Local Peer
   that attempts to allocate resources, then goes idle. The general
   countermeasure described in this section can be used to free
   resources allocated by an idle Local Peer.

7.5.4  Exercise of non-optimal code paths

   Another form of DOS attack is to attempt to exercise data paths that
   can consume a disproportionate amount of resources. An example might
   be if error cases are handled on a ôslow pathö (consuming either
   host or RNIC computational resources), and an attacker generates
   excessive numbers of errors in an attempt to consume these
   resources.

   It is RECOMMENDED that an implementation provide the ability to
   detect the above condition and allow an administrator to act,
   including potentially administratively tearing down the RDMAP Stream
   associated with the connection exercising data paths consuming a
   disproportionate amount of resources.


J. Pinkerton, et al.   Expires - December 2003              [Page 27]


Internet-Draft           RDDP/RDMAP Security              June 2003

7.6  Elevation of Privilege

   The RDMAP/DDP Security Architecture explicitly differentiates
   between three levels of privilege û Non-Privileged, Privileged, and
   the Privileged Resource Manager. If a Non-Privileged Application is
   able to elevate its privilege level to a Privileged Application,
   then mapping a physical address list to an STag can provide local
   and remote access to any physical address location on the node. If a
   Privileged Mode Application is able to promote itself to be a
   Resource Manager, then it is possible for it to perform denial of
   service type attacks where substantial amounts of local resources
   could be consumed.

   There is only one mechanism discovered to date, other than
   implementation defects, which would potentially allow an elevation
   of privilege.

7.6.1  Loading Firmware into the RNIC

   If the RI implementation, by some insecure mechanism (or
   implementation defect), can enable a Remote Peer or un-trusted Local
   Peer to load firmware into the RNIC Engine, it is possible to use
   the RNIC to attack the host. Thus, it is RECOMMENDED that an
   implementation not enable firmware to be loaded on the RNIC Engine
   directly from a Remote Peer, unless the update is done via a secure
   protocol, such as IPsec (See Section 8 IPsec and RDDP on page 29).
   It is RECOMMENDED that an implementation not allow a Non-Privileged
   Local Peer to update firmware in the RNIC Engine.




















J. Pinkerton, et al.   Expires - December 2003              [Page 28]


Internet-Draft           RDDP/RDMAP Security              June 2003

8  IPsec and RDDP

8.1  Introduction to IPsec

   IPsec is a protocol suite which is used to secure communication at
   the network layer between two peers.  The IPsec protocol suite is
   specified within the IP Security Architecture [RFC2401], IKE
   [RFC2409], IPsec Authentication Header (AH) [RFC2402] and IPsec
   Encapsulating Security Payload (ESP) [RFC2406] documents.  IKE is
   the key management protocol while AH and ESP are used to protect IP
   traffic.

   An IPsec SA is a one-way security association, uniquely identified
   by the 3-tuple: Security Parameter Index (SPI), protocol (ESP) and
   destination IP.  The parameters for an IPsec security association
   are typically established by a key management protocol. These
   include the encapsulation mode, encapsulation type, session keys and
   SPI values.

   IKE is a two phase negotiation protocol based on the modular
   exchange of messages defined by ISAKMP [RFC2408],and the IP Security
   Domain of Interpretation (DOI) [RFC2407]. IKE has two phases, and
   accomplishes the following functions:



   1.  Protected cipher suite and options negotiation - using keyed
       MACs and encryption and anti-replay mechanisms

   2.  Master key generation - such as via MODP Diffie-Hellman
       calculations

   3.  Authentication of end-points

   4.  IPsec SA management (selector negotiation, options negotiation,
       create, delete, and rekeying)

   Items 1 through 3 are accomplished in IKE Phase 1, while item 4 is
   handled in IKE Phase 2.

   An IKE Phase 2 negotiation is performed to establish both an inbound
   and an outbound IPsec SA.  The traffic to be protected by an IPsec
   SA is determined by a selector which has been proposed by the IKE
   initiator and accepted by the IKE Responder.  In IPsec transport
   mode, the IPsec SA selector can be a "filter" or traffic classifier,
   defined as the 5-tuple: <Source IP address, Destination IP address,
   transport protocol (UDP/SCTP/TCP), Source port, Destination port>.

J. Pinkerton, et al.   Expires - December 2003              [Page 29]


Internet-Draft           RDDP/RDMAP Security              June 2003

   The successful establishment of a IKE Phase-2 SA results in the
   creation of two uni-directional IPsec SAs fully qualified by the
   tuple <Protocol (ESP/AH), destination address, SPI>.

   The session keys for each IPsec SA are derived from a master key,
   typically via a MODP Diffie-Hellman computation.  Rekeying of an
   existing IPsec SA pair is accomplished by creating two new IPsec
   SAs, making them active, and then optionally deleting the older
   IPsec SA pair.  Typically the new outbound SA is used immediately,
   and the old inbound SA is left active to receive packets for some
   locally defined time, perhaps 30 seconds or 1 minute.

8.2  Recommendations for IPsec Encapsulation of RDDP

   Issue: IPsec recommendations for RDMAP/DDP


   This is work that is still to be done.






























J. Pinkerton, et al.   Expires - December 2003              [Page 30]


Internet-Draft           RDDP/RDMAP Security              June 2003

9  Summary Table of Attacks and Trust Models

   <editor: This section is under construction, and will be completed
   in a future version of this document>

   Rows are the attack (grouped into categories)

   Columns are the:

       *   Sec û Section the attack is discussed

       *   Attack Name û short name for the attack

       *   Threat - threat type (DOS, etc)

       *   Columns labeled 1-5 are the Trust Model number (see section
           5 Trust Models on page 11). Each entry has a value of +, -,
           and R (research).






























J. Pinkerton, et al.   Expires - December 2003              [Page 31]


Internet-Draft           RDDP/RDMAP Security              June 2003

+-------+--------------------------+-------+---+---+---+---+---+
|  Sec  | Attack Name              |Threat | 1 | 2 | 3 | 4 | 5 |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.2.1 | STag use on different    | Spoof |   |   |   |   |   |
|       |   connection in same PD  |       |   |   |   |   |   |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.3.1 | Memory write outside of  | Tamper|   |   |   |   |   |
|       |   buffer range           |       |   |   |   |   |   |
| 7.3.2 | Modify Buffer after      | Tamper|   |   |   |   |   |
|       |   contents ready         |       |   |   |   |   |   |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.4.1 | Probe memory outside of  | ID    |   |   |   |   |   |
|       |   buffer bounds          |       |   |   |   |   |   |
| 7.4.2 | Access stale data        | ID    |   |   |   |   |   |
| 7.4.3 | Access buffer after      | ID    |   |   |   |   |   |
|       |   transfer over          |       |   |   |   |   |   |
| 7.4.4 | Unintended data access   | ID    |   |   |   |   |   |
|       |   using valid STag       |       |   |   |   |   |   |
| 7.4.5 | Using RDMA Read on a     | ID    |   |   |   |   |   |
|       |   buffer meant only for  |       |   |   |   |   |   |
|       |   RDMA Write             |       |   |   |   |   |   |
| 7.4.6 | Using multiple STags to  | ID    |   |   |   |   |   |
|       |   access the same buffer |       |   |   |   |   |   |
| 7.4.7 | Remote node loading      | ID    |   |   |   |   |   |
|       |   firmware onto RNIC     |       |   |   |   |   |   |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.5.1 | RNIC resource consumption| DOS   |   |   |   |   |   |
| 7.5.2 | Resource consumption by  | DOS   |   |   |   |   |   |
|       |   active processes       |       |   |   |   |   |   |
| 7.5.3 | Resource consumption by  | DOS   |   |   |   |   |   |
|       |   idle processes         |       |   |   |   |   |   |
| 7.5.4 | Non-optimal code paths   | DOS   |   |   |   |   |   |
+-------+--------------------------+-------+---+---+---+---+---+
| 7.6.1 | Loading firmware onto    | Elev  |   |   |   |   |   |
|       |   RNIC                   |       |   |   |   |   |   |
+-------+--------------------------+-------+---+---+---+---+---+

                  Figure 2 û Summary Attacks and Trust Model Table










J. Pinkerton, et al.   Expires - December 2003              [Page 32]


Internet-Draft           RDDP/RDMAP Security              June 2003

10 References

10.1 Normative References

   [RFC2828] Shirley, R., "Internet Security Glossary", FYI 36, RFC
       2828, May 2000.

   [DDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct Data
       Placement over Reliable Transports", Internet-Draft draft-ietf-
       rddp-ddp-01.txt, February 2003.

   [RDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA
       Protocol Specification", Internet-Draft draft-ietf-rddp-rdmap-
       01.txt, February 2003.

    [SEC-CONS] Rescorla, E., B. Korver, IAB, "Guidelines for Writing
       RFC Text on Security Considerations", Internet-Draft draft-ab-
       sec-cons-03.txt, January 2003.

    [RFC2401] Atkinson, R. and Kent, S., "Security Architecture for the
       Internet Protocol", RFC 2401, November 1998

    [RFC2402] Kent, S., Atkinson, R., "IP Authentication Header", RFC
       2402, November 1998

    [RFC2404] Madson, C., Glenn, R., "The Use of HMAC-SHA-1-96 within
       ESP and AH", RFC 2404, November 1998

    [RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security
       Payload (ESP)", RFC 2406, November 1998

    [RFC2407] Piper, D., "The Internet IP Security Domain of
       Interpretation of ISAKMP", RFC 2407, November 1998

    [RFC2408] Maughan, D., Schertler, M., Schneider, M., Turner, J.,
       "Internet Security Association and Key Management Protocol
       (ISAKMP), RFC 2408, November 1998

    [RFC2409] Harkins, D., Carrel, D., "The Internet Key Exchange
       (IKE)", RFC 2409, November 1998








J. Pinkerton, et al.   Expires - December 2003              [Page 33]


Internet-Draft           RDDP/RDMAP Security              June 2003

10.2 Informative References

    [IPv6-Trust] Nikander, P., J.Kempf, E. Nordmark, "IPv6 Neighbor
       Discovery trust modelsTrust Models and threats", Internet-Draft
       draft-ietf-send-psreq-01.txt, January 2003.











































J. Pinkerton, et al.   Expires - December 2003              [Page 34]


Internet-Draft           RDDP/RDMAP Security              June 2003

11 AuthorÆs Addresses

   James Pinkerton
   Microsoft Corporation
   One Microsoft Way
   Redmond, WA. 98052 USA
   Phone: +1 (425) 705-5442
   Email: jpink@windows.microsoft.com

   Ellen Deleganes
   Intel Corporation
   MS JF5-355
   2111 NE 25th Ave.
   Hillsboro, OR 97124 USA
   Phone: +1 (503) 712-4173
   Email: ellen.m.deleganes@intel.com

   Bernard Aboba
   Microsoft Corporation
   One Microsoft Way
   Redmond, WA. 98052 USA
   Phone: +1
   Email:

























J. Pinkerton, et al.   Expires - December 2003              [Page 35]


Internet-Draft           RDDP/RDMAP Security              June 2003

12 Acknowledgments

   Allyn Romanow

   Catherine Meadows

   Pat Thaler

   James Livingston

   John Carrier





































J. Pinkerton, et al.   Expires - December 2003              [Page 36]


Internet-Draft           RDDP/RDMAP Security              June 2003

13 Full Copyright Statement

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

   Funding for the RFC Editor function is currently provided by the
   Internet Society.


















J. Pinkerton, et al.   Expires - December 2003              [Page 37]