Remote Direct Memory Access (RDMA) over IP Problem Statement
RFC 4297
Document | Type | RFC - Informational (December 2005; No errata) | |
---|---|---|---|
Authors | Sandburst Corporation , Jeffrey Mogul , Allyn Romanow , Tom Talpey | ||
Last updated | 2018-12-20 | ||
Replaces | draft-romanow-rdma-over-ip-problem-statement | ||
Stream | IETF | ||
Formats | plain text html pdf htmlized bibtex | ||
Stream | WG state | (None) | |
Document shepherd | No shepherd assigned | ||
IESG | IESG state | RFC 4297 (Informational) | |
Action Holders |
(None)
|
||
Consensus Boilerplate | Unknown | ||
Telechat date | |||
Responsible AD | Jon Peterson | ||
Send notices to | (None) |
Network Working Group A. Romanow Request for Comments: 4297 Cisco Category: Informational J. Mogul HP T. Talpey NetApp S. Bailey Sandburst December 2005 Remote Direct Memory Access (RDMA) over IP Problem Statement Status of This Memo This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Copyright Notice Copyright (C) The Internet Society (2005). Abstract Overhead due to the movement of user data in the end-system network I/O processing path at high speeds is significant, and has limited the use of Internet protocols in interconnection networks, and the Internet itself -- especially where high bandwidth, low latency, and/or low overhead are required by the hosted application. This document examines this overhead, and addresses an architectural, IP-based "copy avoidance" solution for its elimination, by enabling Remote Direct Memory Access (RDMA). Romanow, et al. Informational [Page 1] RFC 4297 RDMA over IP Problem Statement December 2005 Table of Contents 1. Introduction ....................................................2 2. The High Cost of Data Movement Operations in Network I/O ........4 2.1. Copy avoidance improves processing overhead. ...............5 3. Memory bandwidth is the root cause of the problem. ..............6 4. High copy overhead is problematic for many key Internet applications. ...................................................8 5. Copy Avoidance Techniques ......................................10 5.1. A Conceptual Framework: DDP and RDMA ......................11 6. Conclusions ....................................................12 7. Security Considerations ........................................12 8. Terminology ....................................................14 9. Acknowledgements ...............................................14 10. Informative References ........................................15 1. Introduction This document considers the problem of high host processing overhead associated with the movement of user data to and from the network interface under high speed conditions. This problem is often referred to as the "I/O bottleneck" [CT90]. More specifically, the source of high overhead that is of interest here is data movement operations, i.e., copying. The throughput of a system may therefore be limited by the overhead of this copying. This issue is not to be confused with TCP offload, which is not addressed here. High speed refers to conditions where the network link speed is high, relative to the bandwidths of the host CPU and memory. With today's computer systems, one Gigabit per second (Gbits/s) and over is considered high speed. High costs associated with copying are an issue primarily for large scale systems. Although smaller systems such as rack-mounted PCs and small workstations would benefit from a reduction in copying overhead, the benefit to smaller machines will be primarily in the next few years as they scale the amount of bandwidth they handle. Today, it is large system machines with high bandwidth feeds, usually multiprocessors and clusters, that are adversely affected by copying overhead. Examples of such machines include all varieties of servers: database servers, storage servers, application servers for transaction processing, for e-commerce, and web serving, content distribution, video distribution, backups, data mining and decision support, and scientific computing. Note that such servers almost exclusively service many concurrent sessions (transport connections), which, in aggregate, are responsible for > 1 Gbits/s of communication. Nonetheless, the cost Romanow, et al. Informational [Page 2] RFC 4297 RDMA over IP Problem Statement December 2005 of copying overhead for a particular load is the same whether from few or many sessions. The I/O bottleneck, and the role of data movement operations, have been widely studied in research and industry over the last approximately 14 years, and we draw freely on these results. Historically, the I/O bottleneck has received attention whenever new networking technology has substantially increased line rates: 100Show full document text