INTERNET-DRAFT P. Culley draft-culley-mpa-issueresponses-00.txt Hewlett-Packard Company Expires: March 2004 Comment responses for Marker PDU Aligned Framing for TCP Specification Culley Expires: March 2004 [Page 1]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Status of this Memo This document is an Internet-Draft and is subject to all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Culley Expires: March 2004 [Page 2]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Abstract This draft is a response to comments about draft-ietf-rddp-mpa-00 received on the RDDP reflector, the response is long enough to warrant a draft. Comments on connection startup sequence, rejecting a connection, and allowing "Active/Active" connections. Table of Contents Status of this Memo.................................................2 Abstract............................................................3 1 Introduction...............................................4 2 Issue on allowed startup sequence..........................5 2.1 Dual Stack Designs with the current draft..................6 2.2 Dual Stack with no FPDU sending restriction...............11 2.3 Why should we consider changing the draft?................14 3 Active/Active Connections.................................15 3.1 Why should we consider changing the draft?................17 4 Adding a "rejected connection" bit........................18 5 References................................................20 5.1 Normative References......................................20 5.2 Informative References....................................20 6 Author's Addresses........................................20 Table of Figures Figure 1 Typical MPA/TCP immediate startup..........................8 Figure 2 MPA/TCP immediate startup with Early FPDU sent by responder11 Figure 3 MPA/TCP immediate startup "Active/Active".................16 Culley Expires: March 2004 [Page 3]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 1 Introduction The reader is presumed to be familiar with [MPA], [DDP], and may find it useful to have looked at the connection management sections of [verbs]. Some portions of the comments also imply knowledge of [RDMA]. This draft is simply a discussion of the design choices that led to the current MPA draft and is intended to assist the workgroup in making informed decisions on the appropriate direction for future MPA drafts. It is unlikely that this draft will be updated. Comments and suggested directions are welcome and should be directed to the RDDP mailing list (rddp@ietf.org). Culley Expires: March 2004 [Page 4]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 2 Issue on allowed startup sequence An issue was brought regarding a specific MUST in the MPA specification (included below): 4. MPA "Responder" mode implementations MUST receive and validate at least one FPDU before sending any FPDUs or markers. Note: this requirement is present to allow the Initiator time to get its receiver into full operation before an FPDU arrives, avoiding potentially difficult requirements on the receiver. Caitlin wrote: "As previously covered, I believe this rule is overly broad. The only requirement is that neither side send their MPA Start Frame before they are ready to process DDP Segments." Caitlin's requirement can be interpreted at least two ways, either: A) It is ok to send FPDUs as soon as the start frame is sent (but potentially before the incoming start frame is received), OR, B) It is ok to send as long as the start frame has been sent and received. Using the first interpretation, Caitlin's rule has the following problems: 1) The sender must know the receiver's correct marker/CRC settings before sending FPDU's or risk having the connection torn down. Caitlin also feels that this may be a reasonable response, she wrote: "If a mismatch in capabilities will result in terminating the connection, then there is nothing wrong with sending FPDUs before this is confirmed. If there is a mismatch, the FPDUs will be flushed anyway. Obviously traffic should not be placed on the network that has any substantial chance of being thrown away. But if the sender has reason to believe that it already knows its peers mode (perhaps based on Culley Expires: March 2004 [Page 5]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 directory information or prior connections) then it should not be required to stall waiting for dynamic confirmation." 2) The inclusion of the "private data" in the startup frame was based on the design requirement for the ULP to send and get a reply to some private data before binding the TCP/MPA connection to DDP (and choosing a PD). The rule above allows binding to DDP and sending an FPDU before the reply arrives, also increasing the opportunity for incorrect ULP configuration. As above, if the ULP already knows the peers configuration, then this may not be an issue (and the private data was not actually necessary for this ULP). 3) The rule requires that the receiver design be prepared to receive an FPDU BEFORE it has actually been bound to DDP. This can happen with out of order TCP segment arrival, or due to internal processing delay. It is also possible that the start frame might even be in the same TCP segment. This, while not any kind of violation of TCP, makes for much more Difficult "Dual Stack" implementations. The effects on dual stacks and the transition between them will be discussed further below. Using the second interpretation of Caitlin's rule (it is ok to send FPDUs as long as start frame has been sent and received), the problem is only of implementation, again making a "Dual Stack" design more difficult. In order to evaluate this issue, we present a discussion of what we expect to be a common design implementation, a "dual stack". 2.1 Dual Stack Designs with the current draft Warning, the following is highly implementation specific, and as such must not be regarded as any kind of requirement on implementations. MPA/DDP implementations are commonly expected to be implemented as part of a "Dual stack" architecture. One "stack" is the traditional TCP stack, usually with a sockets interface API and the various OS dependent layers and a "standard" NIC. The second stack is the MPA/DDP "stack" with its own API, and potentially separate code or hardware to deal with the MPA/DDP data. Of course, implementations may vary, so the following comments are of an advisory nature only. Culley Expires: March 2004 [Page 6]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 The use of the two "stacks" offers advantages: TCP connection setup is usually done with the TCP stack. This allows use of the usual naming and addressing mechanisms. It also means that any mechanisms used to "harden" the connection setup against security threats are also used when starting MPA/DDP. Some applications may have been originally designed for TCP, but are "enhanced" to utilize MPA/DDP after a negotiation reveals the capability to do so. The negotiation process takes place in TCP's streaming mode, using the usual TCP APIs. Even new applications, designed for RDMA or DDP, still need to exchange some private data (and the start frames) prior to binding DDP. Using the traditional TCP streaming mode for this exchange allows this to be done using well understood methods and simplifies the full MPA/DDP stack (since full streaming mode operations and API's need not also be supported by that stack). The main disadvantage of using two stacks is the conversion of an active TCP connection between them. This process must be done with care to prevent loss of data. Following is the proposed connection setup showing interactions at each level of the stack. Note that, while the example shows an MPA startup just after the TCP connects, it is also expected that some applications will pass some amount of streaming data before performing the MPA startup. Culley Expires: March 2004 [Page 7]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 +----------------+ +----------------+ | MPA INITIATOR | | MPA RESPONDER | +----------------+ +----------------+ ULP DDP MPA TCP TCP MPA DDP ULP | | | | | | | | | | | | |<-------------|A A|------------->| SYN | | | | | | | |-------->| | | | | | | | SYN+ACK | | | | | | | |<--------| | | | | | | | ACK | | | | | | | |-------->| | | | B|<-------------| |------------->|B | | | | | | | | | | | | | | | | | | | | | |<--------|C C|-------->|--->| MPA REQ | |1 | | | | 1|2 |-------->| | | | | | | | |--->|-------->|D | | | | | 2|3 | | | | | | | | | | | | | | MPA REP |<---|<---|<---|E | | | |<--------| 5|4 | | D|<--------|<---| | | | | | | 4|3 | | | | | | | | | | | | | E|--->| | | | | | | | | | | | | | | F|--->|--->|===>| | | | | | | 5|6 |========>| | | | | | | | |===>|--->|--->|G | | | | | 6| | | | | | | | | | | | | | | |<===|<---|<---|H | | | |<========| 8|7 | | G|<---|<---|<===| | | | | | | |7 | | | | | | | | | | | | | Figure 1 Typical MPA/TCP immediate startup Culley Expires: March 2004 [Page 8]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Initiator ULP A connect ULP B TCP connection established ULP C MPA initiate negotiation, send Request frame ULP D receive MPA Reply + private data ULP E connect DDP (post untagged buffers) ULP F send DDP message ULP G receive DDP message Responder ULP A listen ULP B accept (connection established) ULP C MPA listen ULP D receive MPA Request Frame + private data ULP E Accept: Send Response frame + private data, connect DDP (post untagged buffers) ULP G receive DDP message ULP H send DDP message Initiator Using the Legacy stacks MPA 1 ULP requests MPA to 'initiate' mode frame exchange MPA 2 MPA sends "MPA Request Frame" + private data MPA 3 MPA received "MPA Reply Frame" from responder MPA 4 MPA sends private data to ULP Using the DDP/MPA stack MPA 5 ULP/DDP requests MPA to send DDP message MPA 6 MPA sends FPDU to responder MPA 7 MAP receives FPDU from responder Responder Using the Legacy stacks MPA 1 ULP requests MPA to 'respond' to mode frame exchange MPA 2 MPA receives "MPA Request Frame" from initiator MPA 3 MPA sends private data to ULP MPA 4 MPA receives private data from ULP Using the DDP/MPA stack MPA 5 MPA sends "MPA Reply Frame" to initiator MPA 6 MPA receives FPDU from initiator MPA 7 ULP/DDP requests MPA to send DDP message MPA 8 MPA sends FPDU to initiator The switching between the legacy stack, and the TCP/MPA/DDP stack occurs at point (ULP E) for both the initiator and responder. For the initiator, no additional incoming streaming data is expected or buffered, so that no buffers need be passed between stacks. If any data did appear, it would be a protocol violation and could be dropped. Culley Expires: March 2004 [Page 9]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Note that if the last Ack was not sent at the time of the transition, or if packet loss occurs, the initiator's RDDP stack is responsible for sending or resending that Ack. Any repeated incoming data can be dropped. For the Responder, the transition point occurs at the time the last streaming mode message (Start frame) is sent. To allow an implementation to avoid a potential race between getting into FPDU mode and the arrival of the first FPDU, the transition (as described in [verbs]) suggests that the last streaming mode message be sent by the RDDP stack. This allows the stack to do any of several strategies to avoid the race. At the time of the transition, again no data is expected inbound or buffered, so that no buffers need be passed between stacks. As with the initiator, the responder may have to (re)acknowledge prior data. It may also have to retransmit it's last streaming message while being prepared to receive an FPDU. As can be seen, one advantage of implementing this way is that there is no need to pass "undelivered" data between the stacks. Culley Expires: March 2004 [Page 10]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 2.2 Dual Stack with no FPDU sending restriction Let us look at using a dual stack design, but allowing FPDUs to be sent any time after sending and receiving the start frames (the second interpretation of Caitlin's proposed rule). +----------------+ +----------------+ | MPA INITIATOR | | MPA RESPONDER | +----------------+ +----------------+ ULP DDP MPA TCP TCP MPA DDP ULP | | | | | | | | | | | | |<-------------|A A|------------->| SYN | | | | | | | |-------->| | | | | | | | SYN+ACK | | | | | | | |<--------| | | | | | | | ACK | | | | | | | |-------->| | | | B|<-------------| |------------->|B | | | | | | | | | | | | | | | | | | | | | |<--------|C C|-------->|--->| MPA REQ | |1 | | | | 1|2 |-------->| | | | | | | | |--->|-------->|D | | | | | 2|3 | | | | | | | | | | | | | | MPA REP |<---|<---|<---|E | | | |<--------| 5|4 | | D|<--------|<---| | | | | | | 4|3 | | | | | | | | | |<===|<---|<---|H | | | |<========| 6|5 | | | | | 5| | | | | E|--->| | | | | | | | | | | | | | | F|<---|<---|<===| | | | | | | |6 | | | | | | | | | | | | | G|--->|--->|===>| | | | | | | 7|8 |========>| | | | | | | | |===>|--->|--->|G | | | | | 6| | | Figure 2 MPA/TCP immediate startup with Early FPDU sent by responder Culley Expires: March 2004 [Page 11]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Initiator ULP A connect ULP B TCP connection established ULP C MPA initiate negotiation, send Request frame ULP D receive MPA Reply + private data ULP E connect DDP (post untagged buffers) ULP F receive DDP message ULP G send DDP message Responder ULP A listen ULP B accept (connection established) ULP C MPA listen ULP D receive MPA Request Frame + private data ULP E Accept: Send Response frame + private data, connect DDP (post untagged buffers) ULP H send DDP message ULP G receive DDP message Initiator Using the Legacy stacks MPA 1 ULP requests MPA to 'initiate' mode frame exchange MPA 2 MPA sends "MPA Request Frame" + private data MPA 3 MPA received "MPA Reply Frame" from responder MPA 4 MPA sends private data to ULP TCP 5 TCP receives the FPDU but has nowhere to deliver it Using the DDP/MPA stack MPA 6 MPA receives DDP message from TCP as DDP is bound and passes it up to DDP (and ultimately the app) MPA 7 ULP/DDP requests MPA to send DDP message MPA 8 MPA sends FPDU to responder Responder Using the Legacy stacks MPA 1 ULP enables MPA to Listen for MPA Request Frame MPA 2 MPA receives "MPA Request Frame" from initiator MPA 3 MPA sends private data to ULP MPA 4 MPA receives private data from ULP Using the DDP/MPA stack MPA 5 MPA sends "MPA Reply Frame" to initiator MPA 6 ULP/DDP requests MPA to send DDP message MPA 7 MPA sends FPDU to initiator MPA 8 MPA receives FPDU from initiator Culley Expires: March 2004 [Page 12]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Note that between the initiator ULP 'D' and 'E', the incoming messages can arrive from TCP in lots of ways. The example shows them arriving separately with no DDP buffer present to place the FPDU, because DDP is not yet enabled. However, several other situations could occur, such as; * The two messages can arrive in the same TCP segment. * The two messages can arrive out of order. * The second message could be incomplete. * A message or message fragment can arrive during the stack conversion. All of these situations require additional design care in dealing with the conversion between stacks. None of these situations are present if the rules in the current draft are adopted. Note: It is still possible for an attacker or improperly implemented sender to violate the draft rules and send data when it is not expected (during the stack conversion). In this case, a receiver implementation SHOULD ignore any unexpected data and terminate the connection. If any such data arrives such that it is lost but not easily detected the receiver MAY continue to maintain the connection. This could happen if the data arrived in the legacy stack, but was not transferred to the MPA/DDP stack at binding time, and normal FPDUs or no data at all arrived for the MPA/DDP stack. The above note will be added to the Security section of MPA if the original draft mechanisms are adopted. If we were to further consider using the first interpretation of Caitlin's rule (can send FPDUs as soon as we send a start frame) we end up with the problems identified at the beginning of the draft, as well as the dual stack transition problems just described. Culley Expires: March 2004 [Page 13]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 2.3 Why should we consider changing the draft? Caitlin's suggestions seem to make some sense at first glance, in that they can potentially reduce the time for a DDP/MPA connection to get into operation. Let's look at what kind of applications can actually take advantage of the reduced setup time. We believe that most ULPs that connect DDP immediately after TCP will be using a client/server design. In this design, the Request Frame would start from the client and would carry things like the client credentials, Send and RDMA Read Buffer credits and other ULP required data as the "private data". The server would respond with its own credits and authorization or denial (and closing of the connection) in the Private Data of the Response Frame. The next transaction would be the client beginning the real working request in DDP mode. This fits well with the current draft proposal. ULPs that might not be as good a fit would be those that wanted the server to transmit first after the client started the MPA connection; perhaps the initial client request was included in the Private Data of the Request Frame. A ULP that might have been designed this way will have to insert another message on the wire incurring another round trip of delay before getting the first response. Of course this means that the encoding of the first request is different than other requests as it is sent in the private data, instead of a DDP message, potentially leading to greater ULP protocol complexity. Another ULP that would not fit as well would be where the server started the MPA connection first; in this case the client might want to get to work as soon as it had sent the Response Frame. We don't think this will be common since the server would not have an opportunity to authorize the client until after it had committed more significant server side resources to the session (goes against current DOS avoidance practice). The ability to reduce the startup time in the latter two cases does require dual stack designs to have the ability to pass undelivered data between stacks, some of which is potentially out of order or incomplete. The Stack switch would also have to handle potential asynchronous IP packet or TCP segment arrival. Since this adds considerable complexity and provides opportunity for errors in implementations, the author group felt that the gains were not worth the costs. Culley Expires: March 2004 [Page 14]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 3 Active/Active Connections A second issue brought up for discussion was the possibility for a so called "Active-Active" MPA connection startup. "All that is required is to define a third value for "instant start", which each side sends *and* expects to receive." This suggestion has some of the same problems as the prior issue and also has the problem that the startup frames and "private data" is no longer a request/response type format, but is instead blindly sent by both sides prior to the MPA/DDP binding. Without more protocol work to deal with potential rejected configurations, there is no way to respond to a bad connection configuration except to tear it down. An attempt to add "rejected only" responses would complicate the protocol and the MPA/DDP engine design, in that a receiver would not be sure if the next data was an FPDU or a response frame. For the following example, the TCP session is already connected, and the MPA connection type is Caitlin's proposed "Active/Active". Notice that there is no opportunity for a "private data" response. Again we are assuming the second interpretation of Caitlin's startup sequence rule (It is ok to send as long as the start frame has been sent and received). Culley Expires: March 2004 [Page 15]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 +----------------+ +----------------+ | MPA INITIATOR | | MPA RESPONDER | +----------------+ +----------------+ ULP DDP MPA TCP TCP MPA DDP ULP | | | | | | | | C|-------->|--->| MPA REQ MPA REQ |<---|<--------|C | | 1|2 |----- -----------| 2|1 | | | | | | \ / | | | | | | | | X | | | | | | | | / \ | | | | | | | |<---- \ | | | | D|<--------|<---| \ | | | | | | 4|3 | | | | | | | | | | | | | | | E|--->| | | | | | | | | | | | | | | | | F|--->|--->|===>| \ | | | | | | 5|6 |===========+======>|(Z) | | | | | | | \ | | | | ----->| | | | | | | | |--->|-------->|D | | | | | 3|4 | | | | | | | | | | | | | | | | |<---|E | | | | |(*) | | | | | | | |===>|--->|--->|F | | | | | 5| | | | | | | | | | | | | | | |<===|<---|<---|G | | | |<==================| 7|6 | | G|<---|<---|<===| | | | | | | |7 | | | | | | | | | | | | | Note *: The TCP stack must hold the received out of order FPDU and transfer it to the DDP stack only after the DDP stack is enabled at point (E) on the responder. Figure 3 MPA/TCP immediate startup "Active/Active" Initiator ULP C MPA initiate negotiation, send Request frame ULP D receive MPA Request + private data ULP E connect DDP (post untagged buffers) ULP F send DDP message ULP G receive DDP message Culley Expires: March 2004 [Page 16]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Responder ULP C MPA initiate negotiation, send Request frame ULP D receive MPA Request Frame + private data ULP E connect DDP (post untagged buffers) ULP F receive DDP message ULP G send DDP message Initiator Using the Legacy stacks MPA 1 ULP requests MPA to 'initiate' mode frame exchange MPA 2 MPA sends "MPA Request Frame" + private data MPA 3 MPA received "MPA Request Frame" from responder MPA 4 MPA sends private data to ULP Using the DDP/MPA stack MPA 5 ULP/DDP requests MPA to send DDP message MPA 6 MPA sends FPDU to responder MPA 7 MAP receives FPDU from responder, sends to DDP Responder Using the Legacy stacks MPA 1 ULP requests MPA to 'initiate' mode frame exchange MPA 2 MPA sends "MPA Request Frame" + private data MPA 3 MPA received "MPA Request Frame" from initiator MPA 4 MPA sends private data to ULP Using the DDP/MPA stack MPA 5 MPA receives FPDU from initiator, sends to DDP MPA 6 ULP/DDP requests MPA to send DDP message MPA 7 MPA sends FPDU to initiator At the stack transition (point E at the responder), the responder's TCP stack is holding FPDU data (in this example). So it is necessary to pass that data (in whatever fragmented condition) to the DDP/MPA stack during the transition. 3.1 Why should we consider changing the draft? Even for TCP, active/active startup is seldom used by ULPs. While it is always possible to imagine an application that can use this type of startup, it is also usually possible to design around that need. The author's believe that adding the protocol and implementation support for "active/active" startup, while intellectually stimulating, is not worth the time, cost, and effort for the potential payback. Culley Expires: March 2004 [Page 17]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 4 Adding a "rejected connection" bit Arkady Kanevsky submitted a comment to the reflector, quoted below: The MPA draft now defines very nicely an Initiator/Responder sequence. But it looks deficient for a broad set of ULPs. Most of the client-server ULP uses two type of Responses: "accept" and "reject". The MPA draft does not provide a way to differentiate between these 2 responses. Here is one example of an application use. One of the common way the private data is used is to negotiate RDMA Read credits for a connection to be used by RDDP. An Initiator passes in private data a requested RDMA credits. Responder either accepts a connection or rejects a request specifying how many RDMA Read credits it can support. Currently the Responder has 2 legal choices. One it can terminate a connection. This does not convey any information to the Initiator. The second choice is to generate Responder Frame even though RDMA Read credits requested can not be satisfied. This will establish connection but it can not be used as intended by the Initiator. Initiator can either tear down the established stream mode connection or use connection with Responder RDMA credits supported. This is de-facto ULP protocol since Responder Frame will include the RDMA Read credits it can support. On top of that both sides have to agree on the action they will take. While ULP can use a protocol over private data to differentiate between accept and reject, given a commonality of this semantic for ULPs we can use one bit from the Reserved area of MPA frame for the Responder frame to differentiate between accept and reject responses. Proposal: for Responder MPA frame the 1st Reserved bit to be used as accept/reject bit. 0 for accept, 1 for reject. The bit will be called Type (T) in figure 7 {of the MPA draft}. Culley Expires: March 2004 [Page 18]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 Arkady is mostly correct in his analysis, but he may have missed one point; that is that the initiator has the opportunity and right to examine the private data in the MPA Response Frame before binding DDP to MPA. It may either complete the connection setup by binding DDP (for an "accepted" connection), or tear the connection down (for a "rejected" connection). It is also legal for the responder to tear the connection down immediately after sending a "rejected" MPA Response Frame. In either case, no DDP connection is actually established until both sides have seen the private data in the MPA Response Frame. In both cases the initiator can get a "reason" for a rejected connection. The choice is up to the ULP and can be encoded into the response private data just as easily as in a specific bit. Note: it is also legal for the responder to tear down the TCP connection before sending a response frame, although this offers no opportunity to send a "rejected reason". Again the choice is up to the ULP. Putting the accept/reject bit in a standard place would be useful if MPA or another standard layer needed to examine that bit for its own uses. As currently defined, however, MPA must send private data to the ULP for examination, and depends on the ULP for the next step (continue or teardown). So MPA could not do anything with that bit other than pass it on to the ULP. The author believes that defining an "accept/reject" bit is not necessary, that the response private data allows all the room for such information necessary (including more detailed reasons if the ULP needs them). Adding such a bit would also require adding another input and output command parameter to the MPA ULP interface which may be insufficient for many applications anyway. Culley Expires: March 2004 [Page 19]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 5 References 5.1 Normative References [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. [RFC793] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", RFC 793, September 1981. 5.2 Informative References [DDP] H. Shah et al., "Direct Data Placement over Reliable Transports", draft-ietf-rddp-ddp-01.txt (Work in progress), November 2003 [RDMA] R. Recio et al., "RDMA Protocol Specification", draft-ietf-rddp-rdmap-01.txt, (Work in progress), November 2003 [MPA] Culley, P., "Marker PDU Aligned Framing for TCP Specification" draft-ietf-rddp-mpa-00.txt, October 2003. [Verbs] J. Hilland et al., "RDMA Protocol Verbs Specification" draft-hilland-rddp-verbs-v1.0.pdf, April 2003. (see http://www.rdmaconsortium.org/) 6 Author's Addresses Paul R. Culley Hewlett-Packard Company 20555 SH 249 Houston, Tx. USA 77070-2698 Phone: 281-514-5543 Email: paul.culley@hp.com Culley Expires: March 2004 [Page 20]
INTERNET-DRAFT MPA Framing for TCP 23 January 2004 7 Full Copyright Statement Copyright (C) The Internet Society (2000). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Culley Expires: March 2004 [Page 21]