Improving the Performance and Reliability of RPC Replies on RPC-over-RDMA Transports
draft-cel-nfsv4-rpcrdma-reliable-reply-05

Document Type Active Internet-Draft (individual)
Last updated 2019-05-20
Stream (None)
Intended RFC status (None)
Formats plain text xml pdf html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
Network File System Version 4                                   C. Lever
Internet-Draft                                                    Oracle
Intended status: Experimental                               May 20, 2019
Expires: November 21, 2019

 Improving the Performance and Reliability of RPC Replies on RPC-over-
                            RDMA Transports
               draft-cel-nfsv4-rpcrdma-reliable-reply-05

Abstract

   RPC transports such as RPC-over-RDMA version 1 require reply buffers
   to be in place before an RPC Call is sent.  However, RPC consumers
   sometimes have difficulty estimating the expected maximum size of a
   particular RPC reply.  This introduces the risk that an RPC Reply
   message can overrun reply resources provided by the requester,
   preventing delivery of the message, through no fault of the
   requester.  This document describes a mechanism that eliminates the
   need for pre-allocation of reply resources for unpredictably large
   replies.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on November 21, 2019.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents

Lever                   Expires November 21, 2019               [Page 1]
Internet-Draft        RPC-Over-RDMA Reliable Reply              May 2019

   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   4
   3.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   4
     3.1.  Reply Chunk Overrun . . . . . . . . . . . . . . . . . . .   4
     3.2.  Reply Size Calculation  . . . . . . . . . . . . . . . . .   5
     3.3.  Requester Registration Costs  . . . . . . . . . . . . . .   5
     3.4.  Denial of Service . . . . . . . . . . . . . . . . . . . .   6
     3.5.  Estimating Transport Header Size  . . . . . . . . . . . .   6
   4.  Responder-Provided Read Chunks  . . . . . . . . . . . . . . .   7
     4.1.  Specification . . . . . . . . . . . . . . . . . . . . . .   7
       4.1.1.  Responder Duties  . . . . . . . . . . . . . . . . . .   8
       4.1.2.  Requester Duties  . . . . . . . . . . . . . . . . . .   8
       4.1.3.  Pull Completion Notification  . . . . . . . . . . . .   8
       4.1.4.  Remote Invalidation . . . . . . . . . . . . . . . . .   9
   5.  Analysis  . . . . . . . . . . . . . . . . . . . . . . . . . .   9
     5.1.  Benefits  . . . . . . . . . . . . . . . . . . . . . . . .   9
       5.1.1.  Less Frequent Use of Explicit RDMA  . . . . . . . . .   9
       5.1.2.  Support for Arbitrarily Large Replies . . . . . . . .  10
       5.1.3.  Protection of Connection After RPC Cancellation . . .  10
       5.1.4.  Asynchronous Chunk Invalidation . . . . . . . . . . .  10
     5.2.  Costs . . . . . . . . . . . . . . . . . . . . . . . . . .  10
       5.2.1.  Responder Memory Exposure . . . . . . . . . . . . . .  10
       5.2.2.  Round Trip Penalty  . . . . . . . . . . . . . . . . .  10
       5.2.3.  Credit Accounting Complexity  . . . . . . . . . . . .  11
     5.3.  Selecting a Reply Mechanism . . . . . . . . . . . . . . .  11
       5.3.1.  Requester . . . . . . . . . . . . . . . . . . . . . .  11
       5.3.2.  Responder . . . . . . . . . . . . . . . . . . . . . .  12
     5.4.  Implementation Complexity . . . . . . . . . . . . . . . .  12
       5.4.1.  RPC Call Path . . . . . . . . . . . . . . . . . . . .  13
       5.4.2.  RPC Reply Path  . . . . . . . . . . . . . . . . . . .  13
       5.4.3.  Managing RDMA_DONE messages . . . . . . . . . . . . .  13
     5.5.  Alternatives  . . . . . . . . . . . . . . . . . . . . . .  14
   6.  Interoperation Considerations . . . . . . . . . . . . . . . .  14
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  14
Show full document text