Douglas Otis
INTERNET-DRAFT SANlight
(Expires May 21, 2001) David Rand
AboveNet Communications
FC over SCTP/IP (FC/SCTP/IP)
<draft-otis-fc-sctp-ip-02.txt>
Status of this Memo
This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC 2026 [1]
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as Reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/lid-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Copyright (C) The Internet Society (2000). All Rights Reserved.
Abstract
This specification incorporates an encapsulation scheme for Fibre-
Channel (FC) using SCTP. The essential advantages for using SCTP
compared to TCP as it relates to FC are as follows:
- Headers contained within one frame.
- Objects aligned at 32-bit boundaries.
- Out of sequence frame processing.
- Standard authentication.
- Independent streams under common control.
- Session restart.
- Improved error detection.
- Prevention of blind spoofing and denial of service attacks.
- Standard Heartbeat and multi-homing. (optional)
The principle focus of this document is to provide an implementation
of SCTP as an encapsulation with network related management required
for FC traffic. As FC traffic is predominately related to storage,
functions in this area affect storage as well as a possible means of
direct IP access. Stream filtering to block non-storage related
traffic is an example of a storage specific function covered.
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 2]
Normative References:
RFC 826, Ethernet Address Resolution Protocol(ARP)
RFC 792, Internet Control Message Protocol (ICMP)
RFC 2132, DHCP Options and BOOTP Vendor Extensions (DHCP-BOOTP)
RFC 2589, Lightweight Directory Access Protocol (LDAPv3)
RFC 1157, Simple Network Management Protocol (SNMP)
RFC 1907, Management Information Base for SNMP (SNMPv2-MIB)
RFC 2960, Stream Control Transmission Protocol (SCTP)
RFC "wip", LDAP structures for SCSI Transport (SCSI-LDAP)
ANSI X3.xxx-199x, Fibre Channel - Switch Fabric (FC-SW)
ISO/IEC 14165-113 Fibre Channel 3rd Gen. Physical Interface (FC-PH-3)
ISO/IEC 14776-222 SCSI-3 Fibre Channel Protocol - 2 (FCP-2)
Definitions
Fibre-Channel (FC) Frames are encapsulated as Data Payload chunks
within the SCTP protocol. See FC Frame Format for a definition of the
FC delimiter encoding. The Payload Protocol Identifier is determined
by IANA. All Payload Data types will be sent Unordered. The Host
Name Address option shall not be supported to protect from DNS
attacks.
Stream Management
Stream 0 is reserved as a control stream that exchanges configuration
structures to assign FC port and attributes to streams within Data
Payload Chunks. These structures are as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Request Tag |Revision Major |Revision Minor |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Management Request | Management Response |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Fig. 1 Management Request/Response Header
The Request Tag is assigned by the client to track a Management
Response. The Revision Major indicates compatibility level, whereas,
the Revision Minor indicates additional features. The Management
Request header and arguments are sent by the client and echoed in
Response. The Management Response field is first set to zero by the
client.
Management Request:
- 0 = Report supported Revisions
- 1 = Assign Stream
- 2 = Release Stream
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 3]
Management Response:
- 0 = Good
- 1 = Invalid Request
- 2 = Unavailable Resource
- 3 = Busy
- 4 = Refused
- 5 = Unsupported Revision
- 6 = Unsupported Encapsulation
Report Supported Revisions:
Nothing other than the header is sent on the Revision request. The
response is a list of highest to lowest with the last revision being
0.0, which is mandatory.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Revision Major |Revision Minor |(Revision Major|Revision Minor)|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (additional revisions) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Fig. 2 Revision Response
Assign Stream:
This request is to obtain a stream connected to a B_Port. The B_Port
is defined in FC-SW-2. The B_Port supports the R_RDY flow control
with Buffer-to-Buffer Credit specified in its parameters. This
request also provides an advisory priority desired. It can also
assert a time limit for dropping stale frames. The response will
return the request with the Management Response indicating acceptance
and if Good is returned, the B_Port/Stream field will be modified to
indicate the Stream assigned otherwise it will be set to zero. The
server, to reflect any adjustments, may modify Priority and Time
Limit. Should the Time Limit be zero, the Timestamp field is not
valid and no stale frames are dropped. Future versions supporting
different encapsulations may redefine the B_Port/Stream field and add
additional fields to the Assign Stream Request.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| B_Port/Stream | Priority | Encapsulation |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time Limit (milli-seconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Fig. 3 Assign Stream
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Stream | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Fig. 4 Release Stream
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 4]
B_Port/Stream field is used to exchange the FC medium desired by the
client and the Stream this request will be granted. The client may
request any number of FC medium connections at B_Ports.
The Priority field indicates the relative stream priority desired by
the client for the particular FC medium connection. The lower value
receives less priority by the SCTP sender.
The Encapsulation field presently is defined as containing the value
zero. In the future, this field may be used to indicate alternative
encapsulation techniques or structures.
Authentication Management
Authentication management is facilitated with SCSI-LDAP. Agents
implementing FC/SCTP/IP are intended to use LDAP to obtain the
required information for establishing communications. These
structures extend into stream management attributes in some cases.
Advisory
Devices implementing FC/SCTP/IP will also implement ARP, ICMP and
optionally DHCP-BOOTP, and SNMP.
Only FC Class 3, Class 2, and Class F are supported. The B_Port will
only support Buffer-to-Buffer flow control. As such, SOFi3, SOFn3,
EOFn, EOFt delimiters are predominately expected. The Exchange Error
Policy is _Abort, discard multiple sequences._ To send SCSI commands,
the R_CTL field is set with 0x6 to indicate an unsolicited command,
0x1 to indicate solicited data. R_CTL 0x5 indicates Data Descriptor,
and 0x7 indicates Response. The TYPE field with 0x8 indicates SCSI-
FCP. The S_ID identify the initiator and D_ID the target. The
initiator with the Originator Exchange ID, OX_ID, uniquely identifies
every exchange between the initiator and target with a value other
than 0xFFFF. OX_ID is analogous to the Queue Tag but the number of
open sequences is limited by the SEQ_ID, which is assigned by the
sequence initiator to a unique number to identify the sequence.
SEQ_CNT uniquely identifies FC frames within a sequence and the first
frame always starts at 0. A target may assign a RX_ID to an exchange
as an alias for S_ID and OX_ID but not used by the initiator.
The Frame Control field, F_CTL, contains state information:
- Bit 8 = From Responder
- Bit 9 = Recipient/Initiator Sequence Context
- Bit 10 = First Sequence
- Bit 11 = Last Sequence
- Bit 12 = Last Frame of Sequence
- Bit 13-14 = don't care
- Bit 15 = Transfer/Hold Sequence Initiative
- Bit 16-17 = don't care
- Bit 18-21 = reserved
- Bit 22-27 = don't care
- Bit 28 = Parameter equals Relative Offset (set by target)
- Bit 29 = reserved
- Bit 30-31 = Number of fill bytes.
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 5]
The DF_CTL specifies the presents of optional header and should be set
to 0. The 4 byte CRC field does not include the SOF and EOF fields
and uses the polynomial:
x^32+x^26+x^23+x^22+x^16+x^12+x^11+x^10+x^8+x^7+x^5+x^4+x^2+x+1
FC Frame Format Encapsulation:
The exact size of each frame varies depending on the size of the
variable fields. The size of the variable field ranges from 0 to
2112-bytes as shown in the FC Frame Format in Fig. 5 resulting in the
minimum size FC Frame of 36 bytes and the maximum size FC frame of
2148 bytes. The FC Frame will be prefixed with 8 bytes of additional
information to assist the SCTP transport as a Data Payload chunk as
follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp (milli-seconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OSS |N|C| Tcredit | SOF | EOF |
>+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<
| R_CTL | D_ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved | S_ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TYPE | F_CTL |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SEQ_ID | DF_CTL | SEQ_CNT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OX_ID | RX_ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Parameter |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Optional Header) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Payload) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Fig. 5 Encapsulated FC Frame
The Timestamp should be used to ensure overlapped recovery does not
create a cascade of traffic. IP networks introduce latency delays
that warrant special considerations with respect to Fibre Channel
E_D_TOV (Node Life) and R_A_TOV (Fabric Life) values. The total path
delay between a source and a destination FC B_Port should be less than
a maximum value of 3 seconds, to conform to normal Fibre Channel time
out values. The timestamp applied at the initial reception however
only measures the differences due to queuing and packet retries at the
IP level, which will comprise the greatest component of this possible
delay. The SCTP sender just prior to issue measures the time limit to
comply with timeouts. The sender must monitor and control the send
queue depth to ensure minimal latency following the issuing of the FC
frame and monitor the network RTT.
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 6]
Ordered Set Sequences, OSS, are detected between FC frames and are
indicated within this field. Should SOF and EOF have the value of
zero, the remainder of the frame is not valid and may be truncated.
Such a truncated field will not affect credit/debit accounting.
The N bit indicates the CRC field is not valid and must be constructed
before passing to FC.
The C bit indicates the server asserts login credits. The client, as
acknowledgement, returns this bit.
Tcredit indicates either return of a used credit or possibly
additional credits for active streams where additional buffers are
available. Should the server set the C bit, this field indicates the
current debit while de-allocating any additional credits that have
been granted. The client does not set Tcredit. The credit is
established as Buffer-to-Buffer credits during login. The SCTP server
agent may add additional credits via Tcredit.
The remaining portion of this chunk contains the FC Frame. The Frame
Header is 24-bytes long and has several fields associated with the
identification and control of the payload.
+------+--------+-----------+----//-------+------+------+
| SOF | Frame | Optional | Frame | CRC | EOF |
| (4B) | Header | Header | Payload | (4B) | (4B) |
| | (24B) |<----------------------->| | |
| | | Data Field = (0-2112B) | | |
+------+--------+-----------+----//-------+------+------+
Fig. 6 FC Frame Format
SOF and EOF Delimiters:
On a FC link, SOF and EOF are called Ordered Sets and are sent as
special out-of-band words constructed from the 10-bit comma character
(K28.5) followed by 3 additional 10-bit data characters. On a non-
Fibre Channel link the Start of Frame (SOF) and End of Frame (EOF)
delimiters are both byte-encoded and 4-bytes long.
On a FC link the SOF delimiter serves to identify the beginning of a
frame and prepares the receiver for frame reception. The SOF contains
information about the frame's Class of Service, position within a
sequence, and in some cases, connection status.
The EOF delimiter identifies the end of the frame and the final frame
of a sequence. In addition, it serves to force the running disparity
to negative. The EOF is used to end the connection in connection-
oriented classes of service.
It is therefore important to preserve the information conveyed by the
delimiters across the IP-based network, so that the receiving FC to IP
device can correctly construct the FC frame in its original SOF and
EOF format before forwarding it to its ultimate FC destination on the
FC link if acting as a bridge.
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 7]
Start of Frame (SOF) and End of Frame (EOF) byte encoding are defined
in Annex A. The SOF and EOF codes are encoded as a single byte to
represent each FC Ordered Set.
Frame Header:
The Frame Header is 24-bytes long and has several fields that are
associated with the identification and control of the payload.
Current FC Standards allow up to 3 Optional Header fields:
- Network Header (16-bytes)
- Association Header (32-bytes)
- Device Header (up to 64-bytes)
Frame Payload:
The FC Frame Payload is transparent to the FC to IP device. An FC
application level payload is called an Information Unit at the FC-4
Level. This is mapped into the Frame Payload of the FC Frame. A
large Information Unit is segmented using a structure consisting of FC
Sequences. Typically, a Sequence consists of more than one FC frame.
FC to IP does not maintain any state information regarding the
relationship of frames within a FC Sequence.
CRC:
The CRC is 4-bytes long and uses the same 32-bit polynomial used in
FDDI and Ethernet and is specified in ANSI X3.139 Fiber Distributed
Data Interface. When FC frames are encapsulated into IP packets, the
CRC is untouched.
APPENDIX A: Fibre Channel EOF and SOF Encoding
A.1 Ordered Sets
On a FC link, Ordered Sets (OS) are sent as special out-of-band words
constructed of the 10-bit comma character (K28.5) followed by 3
additional 10-bit data characters. The Ordered Sets defined by FC
include the Frame Delimiter, Start of Frame (SOF) and End of Frame
(EOF), and other Primitive Signals.
When FC frames are encapsulated in an IP packet, the Byte-encoded
frame format is used. The Byte-encoded frame format uses 32-bit OS
Code Words to represent valid FC frame delimiter. This format uses a
single-byte OS Code to represent each FC Ordered Set.
For frame delimiters, the encoding is mapped using the encoding
defined in a proposal for a Fibre-Channel Backbone (FC-BB) located at:
ftp://ftp.t11.org/t11/pub/fc/bb/00-068v3.pdf
FC frame delimiters shall be encoded as shown in Table A.2.
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 8]
A.2 Encoded FC Frame Delimiters K28.5, Daa.b, Dcc.d, Dcc.d
+-----------------+----------------------------------------------+
| OS-Code (hex) | Delimiter Name |
+-----------------+----------------------------------------------+
| 0x3F | SOFc1 Connect Class 1 |
+-----------------+----------------------------------------------+
| 0x2F | SOFi1 Initiate Class 1 |
+-----------------+----------------------------------------------+
| 0x37 | SOFn1 Normal Class 1 |
+-----------------+----------------------------------------------+
| 0x2D | SOFi2 Initiate Class 2 |
+-----------------+----------------------------------------------+
| 0x35 | SOFn2 Normal Class 2 |
+-----------------+----------------------------------------------+
| 0x2E * | SOFi3 Initiate Class 3 (Initialize Loop)|
+-----------------+----------------------------------------------+
| 0x36 * | SOFn3 Normal Class 3 |
+-----------------+----------------------------------------------+
| 0x39 | SOFc4 Activate Class 4 |
+-----------------+----------------------------------------------+
| 0x29 | SOFi4 Initiate Class 4 |
+-----------------+----------------------------------------------+
| 0x31 | SOFn4 Normal Class 4 |
+-----------------+----------------------------------------------+
| 0x28 | SOFf Fabric |
+-----------------+----------------------------------------------+
| 0x42 * | EOFt Terminate |
+-----------------+----------------------------------------------+
| 0x46 | EOFdt Disconnect-Terminate |
+-----------------+----------------------------------------------+
| 0x50 | EOFa Abort |
+-----------------+----------------------------------------------+
| 0x41 * | EOFn Normal |
+-----------------+----------------------------------------------+
| 0x4E | EOFdti Disconnect Terminate-Invalid |
+-----------------+----------------------------------------------+
| 0x49 | EOFni Normal Invalid |
+-----------------+----------------------------------------------+
* Indicates used for Class 3 SCSI-FCP
A.3 Encoded Primitive Sequences K28.5, Daa.b, Dcc.d, Dee.f
+-----------------+---------------------------------+
| OSS (hex) | Delimiter Name |
+-----------------+---------------------------------+
| 0x0 | No Sequence |
+-----------------+---------------------------------+
| 0x1 | NOS Not Operational |
+-----------------+---------------------------------+
| 0x2 | LRR Link Reset Response |
+-----------------+---------------------------------+
| 0x3 | LR Link Reset Offline |
+-----------------+---------------------------------+
| 0x4 | OLS Offline |
+-----------------+---------------------------------+
Internet Draft, 21-Nov-00 FC/SCTP/IP [Page 9]
Author's Address:
Douglas Otis
SANlight Inc.
160 Saratoga Ave, #46
Santa Clara, CA 95051
Tel: (408) 260-1400 x2001
dotis@sanlight.net
Full Copyright Statement
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published and
distributed, in whole or in part, without restriction of any kind,
provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of developing
Internet standards in which case the procedures for copyrights defined
in the Internet Standards process must be followed, or as required to
translate it into languages other than English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN
WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
------