IP Storage Working Group Charles Monia INTERNET DRAFT Rod Mullendore Expires July 2001 Josh Tseng <draft-monia-ips-ifcp-01.txt> Nishan Systems Franco Travostino Nortel Networks David Robinson Sun Microsystems Wayland Jeong Troika Networks Rory Bolt Quantum/ATL Paul Rutherford ADIC Mark Edwards Eurologic January 2001 iFCP - A Protocol for Internet Fibre Channel Storage Networking Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Comments Comments should be sent to the ips mailing list (ips@ece.cmu.edu) or to the author(s). Monia, Mullendore, Tseng 1 iFCP January 2001 1. Abstract This document specifies iFCP, a gateway-to-gateway protocol for the implementation of a Fibre Channel fabric in which TCP/IP switching and routing elements replace Fibre Channel components. The protocol enables the attachment of existing Fibre Channel storage products to an IP network by supporting the subset of fabric services required by such devices. The encapsulation mechanism described in this document can also be leveraged to support other applications that transport Fibre Channel frames over IP. 2. About This Document 2.1 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. All frame formats are in big endian network byte order. 2.2 Purpose of this document This is a standards-track document, which specifies a protocol for the implementation of Fibre Channel transport services on a TCP/IP network. Some portions of this document contain material from standards controlled by NCITS T10 and T11. This material is included here for informational purposes only. The authoritative information is given in the appropriate NCITS standards document. The authoritative portions of this document specify the protocol for mapping standards-compliant fibre Channel storage and adapter implementations to TCP/IP. This mapping includes sections of this document which describe the "iFCP Layer" (see section 4.1). The Fibre Channel frame encapsulation described in section 5 of this document is of general applicability, and can be used for other applications that transport Fibre Channel frames over TCP. One example of an alternate protocol which can leverage this encapsulation is the FCIP tunneling protocol. FCIP, and other examples of Fibre Channel protocols, are described in the appendicies of this document. 3 iFCP Introduction iFCP is a gateway-to-gateway protocol, which provides Fibre Channel fabric services to FCP-based Fibre Channel devices over a TCP/IP network. iFCP uses TCP to provide congestion control, error detection and recovery. iFCP's primary objective is to allow Monia, Mullendore, Tseng 2 iFCP January 2001 interconnection and networking of existing Fibre Channel devices at wire speeds over an IP network. The protocols and method of frame translation described in this document permit the transparent attachment of Fibre Channel storage devices to an IP-based fabric by means of lightweight gateways. The protocol achieves this transparency through an address translation process that allows normal frame traffic to pass through the gateway directly, with provisions for intercepting and emulating the fabric services required by an FCP device. 3.1 Definitions Terms needed to clarify the concepts presented in this section are presented here. Fibre Channel Network - A fabric and all attached Fibre Channel devices. Fabric - The part of a Fibre Channel network which provides the transport services defined in the FC-FS specification. A fabric may be implemented in the IP framework by means of the architecture and protocols discussed in this document. FC-2 - The Fibre Channel transport services layer described in the FC-FS specification. FCP Portal - An IP-addressable entity representing the point at which a logical or physical iFCP device is attached to the IP network. N_PORT - An iFCP or Fibre Channel entity representing the interface to Fibre Channel device functionality. This interface implements the Fibre Channel N_PORT semantics specified in the FC-FS standard [FC-FS]. N_PORT fabric address - The address of an N_PORT within the Fibre Channel fabric. N_PORT Network Address - The address of an N_PORT in the IP fabric. This address consists of the IP address of the FCP Portal and the N_PORT ID of the directly-attached Fibre Channel device. F_Port - The interface used by an N_PORT to access Fibre Channel fabric and fabric services functionality. iFCP - The protocol discussed in this document. Logical FCP Device - The abstraction representing a single Fibre Channel device as it appears on an iFCP network. Monia, Mullendore, Tseng 3 iFCP January 2001 iSNS - The protocol by which storage name services are implemented. Resolution of Fibre Channel network object names is provided by an iSNS name server. N_PORT Session - An association created when two N_PORTS have executed a PLOGI operation. It is comprised of the N_PORTs and TCP connection that carries traffic between them. iFCP Frame - The frame inserted into the TCP stream which contains the Fibre Channel frame and iFCP header. Port Login (PLOGI) - The Fibre Channel Extended Link Service (ELS) that establishes an N_PORT login session through the exchange of identification and operation parameters between an originating N_PORT and a responding N_PORT. 3.2 The iFCP Network Model The following diagram shows a Fibre Channel fabric with attached devices. These are connected to the fabric through N_PORT and F_PORT interfaces, whose behavior is specified in [FGS]. Within the Fibre Channel device domain, fabric-addressable entities consist of other N_PORTs and devices internal to the fabric that perform the fabric services defined in [FGS]. In this case, the N_PORT Fibre Channel addresses are 24-bit quantities that are unique within the scope of the FC fabric. N_PORTs that perform fabric services are assigned well-known addresses starting at the top end of the 24-bit Fibre Channel address space. Fibre Channel Network +--------+ +--------+ | FC | | FC | | Device | | Device | |........| |........| Fibre Channel | N_PORT |<------>| N_PORT | Device Domain +---+----+ +----+---+ ^ | | | +---+----+ +----+---+ | | F_PORT | | F_PORT | | ==========+========+========+========+============== | Fabric & | | | Fabric Services | v | | Fibre Channel +--------------------------+ Fabric Domain Monia, Mullendore, Tseng 4 iFCP January 2001 An iFCP Network with iFCP Gateways Fibre Channel Devices Fibre Channel Devices +--------+ +--------+ +--------+ +--------+ | FC | | FC | | FC | | FC | | Device | | Device | Fibre | Device | | Device | Fibre |........| |........| Channel |........| |........| Channel | N_PORT | | N_PORT |<--------->| N_PORT | | N_PORT | Device +---+----+ +---+----+ Traffic +----+---+ +----+---+ Domain | | | | ^ +---+----+ +---+----+ +----+---+ +----+---+ | | F_PORT | | F_PORT | | F_PORT | | F_PORT | | =+========+==+====================+========+==+========+========== | iFCP Layer |<--------->| iFCP Layer | | |....................| ^ |....................| | | FCP Portal | | | FCP Portal | v +--------+-----------+ | +----------+---------+ IP Fabric | Control | | Data | | | | | |<------Encapsulated Frames------->| | +------------------+ | | | | | +------+ IP Network +--------+ | | +------------------+ The above diagram shows the simplest implementation of an equivalent iFCP fabric. Here, Fibre Channel devices are directly connected to the iFCP fabric through F_PORTs implemented as part of the edge switch or gateway. At the N_PORT interface on the Fibre Channel side of the gateway, the network appears as a Fibre Channel fabric. Here, the gateway presents remote N_PORTs as directly attached devices. Conversely, on the IP-side, the gateway presents each locally-connected N_PORT as a logical iFCP device on the IP network. An important property of this gateway architecture is that the fabric configuration and topology on the FC-side are opaque to the IP network. Consequently, support for FC fabric topologies, such as switches or loops, becomes a gateway implementation option. In such cases, the gateway incorporates whatever functionality is required to distil and present locally attached N_PORTs (or NL_PORTs) as logical iFCP devices. N_PORT to N_PORT communications that traverse a TCP/IP network require the intervention of the iFCP layer. This is done through the following operations on Fibre Channel frames: a) For outbound frames, translate Fibre Channel fabric addresses imbedded in FC frames to N_PORT Network Addresses. Monia, Mullendore, Tseng 5 iFCP January 2001 b) For inbound frames, translate N_PORT Network Addresses to Fibre Channel fabric addresses. c) Redirect requests for fabric-supplied link services addressed to one of the well-known Fibre Channel N_PORT addresses. d) Generate control data in response to certain link service requests or fabric control operations as described in section 7.1. e) Encapsulate Fibre Channel frames for injection into the TCP/IP network and de-encapsulate Fibre Channel frames received from the TCP/IP network. f) Direct de-encapsulated, FC frames to the appropriate N_PORT. After address translation on outbound FC frames, the emulation process generates two types of encapsulated FC frame traffic: a) FC frames to be passed between N_PORTs unchanged, except for the address field modifications described below. b) Augmented FC frames generated in response to N_PORT Link Service requests. These frames are passed between the iFCP layers. For N_PORT link service requests, iFCP may modify the payload to perform these operations in an IP fabric. Control traffic is also generated in order to perform certain peer- to-peer operations among FCP Portals, such as TCP/IP connection setup and management. 3.3 Fibre Channel Frame Address Translation An iFCP gateway is responsible for the mapping between N_PORT Fibre Channel addresses and N_PORT network addresses in the IP fabric. In the IP fabric, the network address of a N_PORT has two components: the IP address of the FCP Portal to which the Fibre Channel device is attached and an N_PORT ID that is unique within the scope of the FCP Portal. When an FC frame is in transit, the IP components of the source and destination FCP Portals are implied from the TCP/IP connection context. The source and destination N_PORT IDs are stored in the corresponding N_PORT address fields that are part of the FC frame. The iFCP gateway is responsible for assigning Fibre Channel N_PORT IDs to directly-attached N_PORTs. For external N_PORTs, the iFCP gateway is also responsible for assigning an internal key used to look up the N_PORT network address for the external device. To perform this function a gateway or edge switch maintains a table which maps frame traffic to the appropriate TCP/IP connection and N_PORT ID of all external N_PORTs with active sessions. The gateway builds the store of N_PORT network addresses for external devices in the IP fabric by: Monia, Mullendore, Tseng 6 iFCP January 2001 a) Intercepting name service requests issued by directly-attached N_PORTs and redirecting them to the iSNS name server or, b) Intercepting incoming N_PORT login requests from external Fibre Channel devices. The TCP/IP connection context to a given FCP Portal may be established during fabric initialization. Subsequently, in response to name server requests, the iSNS server returns the IP address and N_PORT ID pair. The IP address is mapped to the connection context. After saving the context and N_PORT ID, the iFCP layer then creates a 24-bit key that is returned to the local N_PORT as the Fibre Channel address of the external device. For outbound frames, the table of external N_Port network addresses are referenced to map the Destination N_PORT address key and Source N_PORT ID to a TCP connection identifier and N_PORT ID. The translation process for outbound frames is shown below. Raw Fibre Channel Frame +--------+-----------------------------------+ +--------------+ | | Destination N_PORT Address Key |---->| Lookup TCP | +--------+-----------------------------------+ | connection | | | Source N_PORT ID |---->| and N_PORT ID| +--------+-----------------------------------+ +------+-------+ | | | TCP | Control information | | Conn | and Payload | | & +--------------------------------------------+ | N_PORT | ID | After Address Translation and TCP/IP Encapsulation | +--------------------------------------------+ Conn | | iFCP Encapsulation |<-----------+ | Header | Context | +========+===================================+ | | | Destination N_PORT ID |<-----------+ +--------+-----------------------------------+ | | Source N_PORT ID | +--------+-----------------------------------+ | | | Control information | | and Payload | +--------------------------------------------+ For inbound frames, the store regenerates the internal address key from the TCP connection context and N_PORT ID contained in the encapsulated FC frame. The translation process for inbound frames is shown below. Monia, Mullendore, Tseng 7 iFCP January 2001 Network Format of Inbound Frame +--------------------------------------------+ Conn. +---------+ | iFCP Encapuslation Header |------>| Address | | |Context| Key | +========+===================================+ | Lookup | | | Destination N_PORT ID | | | +--------+-----------------------------------+ | | | | Source N_PORT ID |------>| | +--------+-----------------------------------+ +-----+---+ | | |Address | Control information | | Key | and Payload | | +--------------------------------------------+ | | | | Frame after Address Translation and De-encapsulation | +--------+-----------------------------------+ | | | Destination N_PORT ID | | +--------+-----------------------------------+ | | | Source N_PORT Key |<------------+ +--------+-----------------------------------+ | | | Control information | | and Payload | +--------------------------------------------+ 3.4 iFCP Layered Services The following diagram shows the functional layers for host devices that support FCP. As shown, iFCP provides a set of layered services that transparently provide the transport services required by FCP devices. Using the iFCP framework, any existing host FCP implementation will execute with no modifications required. The iFCP protocol layer consists of the data transport services and iFCP-specific Link Services. This layer provides transport services specific to Fibre Channel devices as specified in [FC-PH], [FC-PH-2], and [FC-PH-3]. This is illustrated in the following diagram, which shows the IP Fabric consisting of the TCP/IP network and the iFCP Layer. The IP Fabric provides the transport services for FCP, and is a direct replacement for the transport services provided by a Fibre Channel fabric. Meanwhile, the components in the Fibre Channel Device Domain remain unchanged. Monia, Mullendore, Tseng 8 iFCP January 2001 +---------------------------------------+ - - - - - - - | Storage & Backup Applications | +---------------------------------------+ | Operating System | Application +--------------------+ | Layer | SCSI | | +--------------------+ | - - - - - - - | FCP | | FC-4 Layer +------------+-------+------------------+ - - - - - - - | | Link Services | | +--------------------------+ FC-2 Layer ^ | | | | N_PORT - F_PORT Interface | Fibre Channel | | Device Domain <=============================================================> | | IP Fabric | iFCP Data Transport Service | | | | v | +---------------+ | |iFCP Specific | iFCP Layer | |Link Services | +-----------------------+---------------+ - - - - - - | | | TCP | Transport | | Layer +---------------------------------------+ - - - - - - | | | IP | Network | | Layer +---------------------------------------+ - - - - - - | | | Physical Transport | Link Layer | | +---------------------------------------+ - - - - - - In the figure shown above, each layer leverages the services of the layer below it. 3.4.1 Application Layer This includes the operating system, Storage and Backup applications, and the SCSI driver. This layer interfaces with FCP and Link Services in the FC-2 and FC-4 layers. 3.4.2 FC-4 Layer (FCP) FCP is the Fibre Channel FC-4 layer application protocol used to communicate with devices implementing the SCSI-3 command set and architectural model. Basically, FCP divides each SCSI I/O operation into a series of information units to be transferred between the initiator and target. Monia, Mullendore, Tseng 9 iFCP January 2001 3.4.3 FC-2 Layer The FC-2 Layer provides the facilities for Link Services and transfer of Fibre Channel information units as described below. 3.4.3.1 Link Service Messages Fibre Channel defines a series of link services defined in Fibre Channel Physical and Signaling Interface specification (FC-PH, FC- PH-2, FC-PH-3). These Link Service Messages provide a set of defined functions that allow a Fibre Channel port to send control information, or to request another port to perform a specific function. Some Link Service messages reference services provided internally within the Fibre Channel fabric. 3.4.3.2 N_PORT Interface This is an interface which provides access to Fibre Channel device functionality. The N_PORT interface is responsible for segmentation and reassembly of information units from Fibre Channel frames. 3.4.3.3 F_PORT Interface This is the interface through which the N_PORT accesses the Fibre Channel fabric. 3.4.4 iFCP Layer The iFCP layer provides three essential services for FCP-based storage products: a) Transport of Fibre Channel frames and Link Service messages between N_PORTs. b) Support for special Link Service messages needed by iFCP to manage the transmission of storage data on a IP network. c) Augmentation of some Link Service messages with additional data needed in the iFCP environment. The iFCP layer maps Fibre Channel frames to a predetermined TCP connection for transport. Additionally, many link service messages can similarly be transported without modification over a TCP connection. 4. iFCP Protocol 4.1 Overview 4.1.1 iFCP Transport Services Monia, Mullendore, Tseng 10 iFCP January 2001 The iFCP transport services map the Fibre Channel frames comprising each FCP IU and Link Service message to a predetermined TCP connection for transport across an IP network. When receiving FCP- based storage data from the network, the iFCP layer transports, and delivers each resulting frame to the appropriate N_PORT via the F_PORT. The iFCP layer never interprets the contents of the frame payload. For incoming iFCP frames with control data, iFCP interprets the augmented information in the iFCP header, modifies the frame content accordingly, and may forward the resulting frame to the N_PORT for further processing. For out-bound Fibre Channel frames that require control data, the iFCP layer creates the augmented information based on frame content, modifies the frame content, then transmits the resulting Fibre Channel frame with augmented iFCP header through the appropriate TCP connection. 4.1.2 iFCP Support for Link Services Some Link Service messages reference constructs which are specific to the Fibre Channel fabric environment, but are irrelevant in the context of an IP fabric. When iFCP encounters such messages, it will augment the information in the payload by adding additional information in the iFCP header. The receiving iFCP layer will reference the augmented information in order to reconstruct the original Link Service message. The reconstructed frames are then forwarded to the receiving N_PORT for further processing. Section 7.1 describes augmented Link Services. 4.3 Mandatory FC-2 Functionality [To be specified] 4.4 FC-2 Functionality Not Supported [To be specified] 4.5 Optional FC-2 Functionality [To be specified] 5. Encapsulation of Fibre Channel frames This section describes the encapsulation method used for iFCP. This encapsulation method may be leveraged for other applications which transport Fibre Channel over TCP. The frame encapsulation format is shown below. The iFCP header encapsulates the iFCP payload containing the Fibre Channel frame. Monia, Mullendore, Tseng 11 iFCP January 2001 The length of the iFCP frame depends on the size of the encapsulated Fibre Channel frame and the amount of augmentation data (if any) in the iFCP header. Each Fibre Channel frame, in turn, is comprised of a Fibre Channel header and payload whose format is specified in the appropriate Fibre Channel standards [FC- PH]. Fibre Channel primitive signals (IDLE, R_RDY, ARB, OPN, CLS, and MRK) and primitive sequences (OLS, NOS, LR, LRR, LIP, LPB, and LPE) are stripped off and not encapsulated by iFCP. Fibre Channel frame delimiters SOF and EOF are preserved by encoding them into a one- byte opcode for transport in the iFCP header. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ - - - - - - - - - - - 0 | iFCP Header | N Bytes ^ +----------------------------------+ - - - - - - - - | | | ^ iFCP N / Fibre Channel Header / 24 Bytes | Frame | (described in section 5.1) | | | +----------------------------------+ Fibre | N+24 | | Channel | / Fibre Channel Payload / L Bytes Frame | | | | | +----------------------------------+ | | 4 | iFCP CRC | | | +----------------------------------+ - - - - - - - - - - - Total Length = 28 + N + L 5.1 iFCP Header The iFCP header for TCP is shown below. |3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1|1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0| |1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6|5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0| +-------+-------+---------------+-------------------------------+ | CLASS | VERS | HEADER MARKER | FLAGS | +-------+-------+---------------+-------------------------------+ | iFCP HEADER LENGTH | iFCP DATA LENGTH | +-------------------------------+-------------------------------+ | HEADER CHECKSUM OR CRC (if present) | +---------------------------------------------------------------+ | iFCP AUGMENTATION DATA (if present) | +---------------------------------------------------------------+ CLASS - This 4-bit field indicates the class of service. Only the values 2 and 3 are allowed and are equivalent to the class of service for Fibre Channel. VERS - This 4-bit field indicates the iFCP protocol version. The version number shall be 1. Monia, Mullendore, Tseng 12 iFCP January 2001 HEADER MARKER - This 8-bit field shall be 0xAA if the HEADER CHECKSUM OR CRC field contains the 32-bit header CRC field, 0xC3 if the HEADER CHECKSUM OR CRC field contains a 32-bit header checksum, or 0x3C if the HEADER CHECKSUM OR CRC field does not exist in the header. Any other value in the HEADER MARKER indicates an error, and shall result termination of the TCP connection in which the error occurred. FLAGS - This 16-bit field contains status flags that indicate various parameters for the data frame. iFCP HEADER LENGTH - This 16-bit field contains the length in 4- byte words of the iFCP HEADER, including the iFCP AUGMENTATION DATA field (if present) and the CHECKSUM OR CRC field (if present). Bit Field iFCP Flag Description --------- --------- ----------- 15 SEQUENCE END Indicates if frame terminates a sequence. For Class 3 sequences, the FCP_Portal sets this bit on the last frame of the sequence. In Class 2 sequences, this bit is set by the recipient in the ACK frame that terminates the sequence. 1=Last frame of sequence, 0=not 14 SEQUENCE START Indicates if frame is the first frame of a sequence. 1=First frame of sequence, 0=not 13 non-iFCP This field indicates if the payload contains a Fibre Channel frame encapsulated for a non-iFCP compatible application. An iFCP-compliant implementation MUST discard the frame if this bit is enabled. 1=non-iFCP compatible 0=iFCP compatible 9-12 RESERVED FOR NON-COMPATIBLE APPLICATIONS (see appendix A) 4-8 RESERVED FOR iFCP 3 iFCP DATA CRC Indicates if a trailing 32-bit CRC is present at the end of the iFCP frame. The CRC shall cover the iFCP payload, with includes the Fibre Channel header and Fibre Channel payload. 1=CRC present, 0=Not Monia, Mullendore, Tseng 13 iFCP January 2001 2 TCP ELS Indicates frame contains a TCP-specific Link Service message. 1=TCP ELS, 0=Not 1 AUGMENTATION Indicates the iFCP Augmentation Data PRESENT field is present in the iFCP header. 1=Present, 0=Not 0 COMPLIANCE This bit must be enabled. 1=Enabled, 0=Not iFCP DATA LENGTH - This 16-bit field contains the length in 4-byte words of the iFCP payload. This includes the FCP or Link Service headers and the iFCP trailing CRC (if present), but does not include the iFCP AUGMENTED DATA field or any other portion of the iFCP Header. HEADER CHECKSUM OR CRC - The presence and contents of this 32-bit field is indicated by the HEADER MARKER field. If this field contains a CRC, it shall be calculated over the iFCP header from the CLASS field to the iFCP DATA LENGTH field (inclusive). The CRC shall not include the iFCP AUGMENTED DATA. The CRC algorithm is the same used for the Frame Check Sequence (FCS) of IEEE 802.3 Ethernet frames. If this field contains a Checksum, it shall be a 32-bit checksum calculated over the iFCP header from the CLASS field to the iFCP DATA LENGTH field (inclusive). The checksum shall not include the iFCP AUGMENTED DATA. iFCP AUGMENTATION DATA - This is a variable-length field containing augmentation data required for transport of some Link Service messages over TCP/IP. 5.2 Fibre Channel Frame Header The Fibre Channel frame header defined in FC-PH is used when transporting FCP and Link Service frames in an IP fabric. Use of the Fibre Channel frame header simplifies the connection of Fibre Channel devices to a iFCP storage network. In iFCP, the only modification to the basic Fibre Channel frame header is that the contents of D_ID and S_ID fields are replaced with Destination and Source N_PORT IDs as described in section 3.3. The figure below shows the format of the header used for both FCP and Link Service frames. The contents of D_ID and S_ID fields have been replaced by the Destination and Source N_PORT IDs. Monia, Mullendore, Tseng 14 iFCP January 2001 3 3 3 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 |1 0 9 8 7 6 5 4|3 2 1 0 9 8 7 6|5 4 3 2 1 0 9 8|7 6 5 4 3 2 1 0| +---------------+-----------------------------------------------+ | R_CTL | Destination N_PORT ID | +---------------+-----------------------------------------------+ | CS_CTL | Source N_PORT ID | +---------------+-----------------------------------------------+ | Type | F_CTL | +---------------+---------------+-------------------------------+ | SEQ_ID | DF_CTL | SEQ_CNT | +---------------+---------------+-------------------------------+ | OX_ID | RX_ID | +-------------------------------+-------------------------------+ | PARAMETER | +---------------------------------------------------------------+ Other than Source and Destination N_PORT ID, descriptions of each of the above fields can be found in [FC-PH]. 5.2.1 Destination N_PORT ID This 24-bit value is stored in the Fibre Channel D_ID field. The destination IP address, combined with the Destination N_PORT ID, specifies the unique N_PORT network address of the device receiving the Fibre Channel frame. 5.2.2 Source N_PORT ID This 24-bit value is stored in the Fibre Channel S_ID field. The source IP address, combined with the Source N_PORT ID specifies the unique N_PORT network address of the device originating the Fibre Channel frame. 5.3 Trailing iFCP CRC This original Fibre Channel CRC shall not be encapsulated into the iFCP message. Rather, iFCP MAY use a 32-bit field containing a CRC calculated over the data contained in the frame from the iFCP AUGMENTED DATA to the iFCP Payload (inclusive). This includes the Fibre Channel header and payload. The iFCP CRC follows the iFCP payload, and the CRC algorithm is the same used for the Frame Check Sequence (FCS) of IEEE 802.3 Ethernet frames. A bit in the iFCP FLAGS field in the iFCP header indicates the presence or absence of this 32-bit trailing CRC. 6. TCP Stream Transport of iFCP Frames TCP connections MAY be established between FCP_Portals that have discovered each other through a naming service or through manual configuration. If a TCP connection is not maintained between the FCP_Portals, then a change in the status of remote N_PORTs must be discovered through a central name server authority. Monia, Mullendore, Tseng 15 iFCP January 2001 Multiple TCP connections may exist between pairs of FCP Portals. Such connections are either "bound" or "unbound". An unbound connection is a TCP connection that is not actively supporting an N_PORT login session. Pre-existing TCP connections between FCP Portals remain unbound and uncommitted until a CBIND message (see section 7.2.2) has been transmitted through them. When the iFCP layer detects a Port Login (PLOGI) message creating a login session between a pair of N_PORTs, it will select an existing TCP connection (bound or unbound) or establish a new TCP connection, and send the CBIND message down that TCP connection. This allocates the TCP connection to that PLOGI login session. Optionally, a TCP connection may be bound to more than one N_PORT login session. 6.1 TCP Session Model iFCP uses a single TCP connection to transport all Fibre Channel frames between unique pairs of N_PORTs. 6.2 TCP Port Numbers An FCP Portal uses a single port number to receive TCP connection requests for iFCP over TCP. All TCP connections established between FCP Portals must be directed to the registered well known port number assigned by the IANA. Note that control and data connections are established to the same port number with CBIND messages used to distinguish the connection type. An FCP Portal may use any TCP port number consistent with its implementation of the TCP/IP stack to initiate a TCP connection, but each port number must be unique. 7. Link Services The link services provide a set of functions that allow a port to send control information or request another port to perform a specific function. Each Link Service message (response and reply) is carried by a Fibre Channel sequence, and can be segmented into multiple frames. The iFCP Layer is responsible for transporting Link Service messages across the IP fabric. This includes mapping Link Service messages appropriately from the domain of the Fibre Channel transport to that of the IP network. This process may involve manipulation of field values as the Link Service message travels to and from the IP and Fibre Channel fabrics. It also may involve the inclusion of additional control data through use of the AUGMENTATION DATA field in the iFCP header in order to make the Link Service message significant in the IP fabric domain. Monia, Mullendore, Tseng 16 iFCP January 2001 [Editor's Note: The method for processing multi-frame Link Service messages will be detailed in a subsequent draft] 7.1 Augmented Link Service Messages The following Link Service Messages must be augmented with additional control data carried in the iFCP header. When the iFCP header encapsulates one of these Link Service messages in the iFCP payload, the AUGMENTATION PRESENT bit must be enabled in the iFCP FLAGS field, and the iFCP HEADER LENGTH must be adjusted appropriately to account for the additional augmentation data in the iFCP header. In addition, many ACC responses to the following Link Service messages must also have control data carried in the encapsulating iFCP header. Link Service Message LS_COMMAND Short Name -------------------- ---------- ---------- Port Login 0x03000000 PLOGI Read Exchange Status Block 0x08000000 RES Read Sequence Status Block 0x09000000 RSS Request Sequence Initiative 0x0A000000 RSI Reinstate Recovery Qualifier 0x12000000 RRQ Read Exchange Concise 0x13000000 REC Third Party Process Logout 0x24 TPRLO Discover Address 0x52000000 ADISC The above Link Service messages use N_PORT fabric addresses in their message format, and must have the corresponding World Wide Port name (WWPN) carried in the AUGMENTATION DATA field of the iFCP header. The N_PORT fabric address field shall then be cleared while the Link Service message is carried in the iFCP frame. Upon receipt of the iFCP frame, the iFCP layer shall use the WWPN data in the iFCP header to replace the original N_PORT fabric address with the appropriate value. The following sections describe the contents and format of the iFCP AUGMENTATION DATA field in the iFCP header. Note that if appropriate, the augmentation may also apply to the corresponding ACC or LS_RJT reply. 7.1.1 Port Login (PLOGI) PLOGI provides the mechanism for establishing a login session between two N_PORTs. The PLOGI request carries information identifying the originating N_PORT, including specification of its capabilities and limitations. If the destination N_PORT accepts the login request, it sends an accept (an ACC frame with PLOGI payload), specifying its capabilities and limitations. This exchange establishes the operating environment for the two N_PORTs. Monia, Mullendore, Tseng 17 iFCP January 2001 The following figure is duplicated from FC-PH, and shows the PLOGI message format for both request and accept (ACC) response. A port will reject a PLOGI request by transmitting an LS_RJT message, which contains no payload. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | COMMON SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 20 | PORT NAME | 8 Bytes +----------------------------------+ 28 | NODE NAME | 8 Bytes +----------------------------------+ 36 | CLASS 1 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 52 | CLASS 2 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 68 | CLASS 3 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 86 | CLASS 4 SERVICE PARAMETERS | 16 Bytes +----------------------------------+ 102 | VENDOR VERSION LEVEL | 16 Bytes +----------------------------------+ Total Length = 118 Details on the above fields, including common and class-based service parameters, can be found in [FC-PH]. The above PLOGI message is transported by the iFCP layer without modification. However, additional accompanying information must be carried in the encapsulating iFCP header. The iFCP AUGMENTED DATA field in the iFCP header shall contain the following information: Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | DEVICE TYPE | 4 Bytes +----------------------------------+ Total Length = 4 DEVICE TYPE - This field contains a value indicating the type of device. The allowed values are shown in the following table. A Parallel SCSI type is assumed to be attached by a SCSI-iFCP Router, while a Fibre Channel device is assumed to be attached by an iFCP Edge Switch. iSCSI and mFCP devices are native IP-based storage devices. Monia, Mullendore, Tseng 18 iFCP January 2001 DEVICE TYPE Bit Field Device Type Values --------- ------------------ 7:4 RESERVED 3 iSCSI DEVICE 2 mFCP DEVICE 1 FIBRE CHANNEL DEVICE 0 PARALLEL SCSI DEVICE 7.1.2 Third Party Process Logout (TPRLO) TPRLO provides a mechanism for an N_PORT (third party) to remove one or more login sessions that exists between the destination N_PORT and other N_PORTs specified in the command. This command includes one or more TPRLO LOGOUT PARAMETER PAGEs, each of which when combined with the destination N_PORT identifies a SCSI login session which shall be terminated by the command. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND | 1 Byte +----------------------------------+ 1 | PAGE LENGTH (0x10) | 1 Byte +----------------------------------+ 2 | PAYLOAD LENGTH (0x14) | 2 Bytes +----------------------------------+ 4 | TPRLO LOGOUT PARAMETER PAGE 1 | 2-4 Bytes +----------------------------------+ | . . . . | M Bytes +----------------------------------+ | TPRLO LOGOUT PARAMETER PAGE N | 2-4 Bytes +----------------------------------+ Total Length = Variable Each TPRLO LOGOUT PARAMETER PAGE identifies a remote N_PORT which when combined with the destination N_PORT identifies a SCSI session to be terminated. The TPRLO LOGOUT PARAMETER PAGE is of the following format: Monia, Mullendore, Tseng 19 iFCP January 2001 Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | TYPE CODE | 1 Byte +----------------------------------+ 1 | TYPE CODE EXTENSION | 1 Byte +----------------------------------+ 2 | TPRLO FLAGS | 2 Bytes +----------------------------------+ 4 | ORIG PROCESS ASSOC (if present) | 4 Bytes +----------------------------------+ 8 | RESP PROCESS ASSOC (if present) | 4 Bytes +----------------------------------+ 12 | RESERVED | 1 Byte +----------------------------------+ 13 | THIRD PARTY ORIGINATOR N_PORT ID | 3 Bytes +----------------------------------+ When the iFCP header contains a TPRLO message (including the ACC response), the iFCP AUGMENTED DATA field will contain the PORT_NAME(s) (WWPN) identifying the N_PORT described by the equivalent TPRLO LOGOUT PARAMETER PAGE(s). If more than one TPRLO LOGOUT PARAMETER PAGE is contained in the Link Service message, equivalent additional PORT_NAME shall also be carried in the iFCP AUGMENTED DATA field. PORT_NAMEs shall be listed in the same order as the equivalent TPRLO LOGOUT PARAMETER PAGEs in the original Link Service message. The following diagram describes the PORT_NAME entries in the iFCP AUGMENTATION DATA field in the iFCP header: Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | 3rd PARTY ORIG PORT NAME 1 | 8 Bytes +----------------------------------+ 8 |3rd PTY ORIG PORT NAME 2 (if pres)| 8 Bytes +----------------------------------+ 16 | . . . . | +----------------------------------+ Additionally, the THIRD PARTY ORIGINATOR N_PORT ID field in each TPRLO LOGOUT PARAMETER PAGE shall be cleared when it is carried by the iFCP header. This applies to both the original Link Service message and the ACC response. When the iFCP layer receives a TPRLO message with AUGMENTATION DATA in the iFCP header, it shall use the latter to replace the THIRD PARTY ORIGINATOR N_PORT ID in the original Link Service message, before forwarding it on to the upper Fibre Channel layers. Additional information on TPRLO can be found in [FC-PH-2]. Monia, Mullendore, Tseng 20 iFCP January 2001 7.1.3 Reinstate Recovery Qualifier (RRQ) RRQ is a request to release resources previously made unavailable as a result of an ABTS or ABTX. The following shows the format of an RRQ request message. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | RESERVED | 1 Bytes +----------------------------------+ 5 | EXCHANGE ORIGINATOR S_ID | 3 Bytes +----------------------------------+ 8 | RRQ OX_ID | 2 Bytes +----------------------------------+ 10 | RRQ RX_ID | 2 Bytes +----------------------------------+ Total Length = 12 Upon transmitting the RRQ Link Service message to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR S_ID field. Furthermore, the iFCP layer shall add the EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. The EXCHANGE ORIGINATOR is a 4-byte field set to 0x00000001 to indicate that the RRQ originator is also the originator of the exchange for which the Recovery Qualifier is being reinstated. This field is set to 0x00000000 to indicate that the RRQ recipient is the originator of the exchange for which the Recovery Qualifier is being reinstated. RRQ OX_ID - The OX_ID of the exchange for which the Recovery Qualifier is being reinstated. RRQ RX_ID - The RX_ID of the exchange for which the Recovery Qualifier is being reinstated. Upon receipt of the RRQ Link Service message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the EXCHANGE ORIGINATOR S_ID field before forwarding the message on to the upper Fibre Channel layers. The ACC reply contains only the LS_COMMAND field as the payload, and does not require additional augmentation data. An LS_RJT reply may be sent in lieu of ACC, to indicate that the RRQ was rejected. Monia, Mullendore, Tseng 21 iFCP January 2001 7.1.4 Read Exchange Concise (REC) REC is a request for information on an exchange. The following shows the format of an REC request. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | RESERVED | 1 Bytes +----------------------------------+ 5 | EXCHANGE ORIGINATOR S_ID | 3 Bytes +----------------------------------+ 8 | REC OX_ID | 2 Bytes +----------------------------------+ 10 | REC RX_ID | 2 Bytes +----------------------------------+ Total Length = 12 Upon transmitting the REC Link Service message to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR S_ID field. Furthermore, the iFCP layer shall add the EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. The EXCHANGE ORIGINATOR is a 4-byte field set to 0x00000001 to indicate that the REC originator is also the originator of the exchange for which status is being requested. This field is set to 0x00000000 to indicate that the REC recipient is the originator of the exchange for which status is being requested. REC OX_ID - The OX_ID of the exchange for which information is being requested. REC RX_ID - The RX_ID of the exchange for which information is being requested. RX_ID may be 0xFFFF, indicating RX_ID is unassigned. Upon receipt of the REC Link Service message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the EXCHANGE ORIGINATOR S_ID field before forwarding the message on to the upper Fibre Channel layers. The following shows the format of the ACC payload in response to REC. Monia, Mullendore, Tseng 22 iFCP January 2001 Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0x02000000) | 4 Bytes +----------------------------------+ 4 | REC OX_ID | 2 Bytes +----------------------------------+ 6 | REC RX_ID | 2 Bytes +----------------------------------+ 8 | RESERVED | 1 Byte +----------------------------------+ 9 | EXCHANGE ORIGINATOR N_PORT ID | 3 Bytes +----------------------------------+ 12 | RESERVED | 1 Byte +----------------------------------+ 12 | EXCHANGE RESPONDER N_PORT ID | 3 Bytes +----------------------------------+ 16 | DATA TRANSFER COUNT | 4 Bytes +----------------------------------+ 20 | E_STAT | 4 Bytes +----------------------------------+ Total Length = 24 Upon transmitting the ACC response to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR N_PORT ID and EXCHANGE RESPONDER N_PORT ID fields when encapsulating the ACC message into iFCP. Furthermore, the iFCP AUGMENTED DATA field in the encapsulating iFCP header shall contain the EXCHANGE RESPONDER field, which is identical to the EXCHANGE ORIGINATOR value used to augment the original REC request message. REC OX_ID - The OX_ID of the exchange for which information is being returned. It should be identical to the REC OX_ID field in the REC request. REC RX_ID - The RX_ID of the exchange for which information is being returned. This field should also be identical to the REC RX_ID field in the REC request. When receiving the ACC response message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and EXCHANGE RESPONDER (in the iFCP header) to replace the EXCHANGE ORIGINATOR N_PORT ID and EXCHANGE RESPONDER N_PORT ID fields before passing the Link Service message on to the upper Fibre Channel layers. DATA TRANSFER COUNT - The number of bytes received by a destination N_PORT for a write command, or the number of bytes received for a read command. E_STAT (Exchange Status) - Of the bits defined in this field, two are particularly significant to iFCP: Monia, Mullendore, Tseng 23 iFCP January 2001 Bit Field Description Significance --------- ----------- ------------ 30 SEQUENCE INITIATIVE 0 = Other port holds sequence init 1 = This port holds sequence init 29 Completion 0 = Open 1 = Closed An LS_RJT reply may be sent in lieu of ACC, to indicate that the REC was rejected. 7.1.5 Discover Address (ADISC) ADISC allows devices to exchange name (Port and Node) information. This allows verification of Port and Node addresses. The following shows the format of the ADISC request message. The ACC response is identical except the command code is replaced with the ACC code (0x02000000), and the PORT_NAME and NODE_NAME fields are those of the responder. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND | 4 Bytes +----------------------------------+ 4 | RESERVED | 1 Byte +----------------------------------+ 5 | HARD ADDRESS | 3 Bytes +----------------------------------+ 8 | ORIGINATOR/RESPONDER PORT NAME | 8 Bytes +----------------------------------+ 16 | ORIGINATOR/RESPONDER NODE NAME | 8 Bytes +----------------------------------+ 24 | RESERVED | 1 Byte +----------------------------------+ 25 | ORIGINATOR N_PORT ID | 3 Bytes +----------------------------------+ Total Length = 28 The HARD ADDRESS field has no meaning for iFCP. The field may contain non-zero values in the request message, but shall be zeroed in the ADISC response. Upon transmission to the IP network, the ORIGINATOR N_PORT ID shall be zeroed in both the request and ACC response messages. Upon receipt of the request or ACC response from the IP network, the iFCP layer shall use the ORIGINATOR/RESPONDER PORT_NAME and NODE_NAME to replace the ORIGINATOR N_PORT ID with the appropriate value, before forwarding the message on to upper Fibre Channel layers. Monia, Mullendore, Tseng 24 iFCP January 2001 The following parameters are used for the ADISC request message. ORIGINATOR PORT NAME - This field contains the World Wide Port Name (WWPN) of the iFCP port from which the ADISC request is being originated. ORIGINATOR NODE NAME - This field contains the WWPN of the N_PORT from which the ADISC request is being originated. 7.1.6 Request Exchange Status (RES) RES requests the status of a specific exchange. The RES request is sent in a new exchange. The following shows the RES message format. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0x08000000) | 4 Bytes +----------------------------------+ 4 | RESERVED | 1 Byte +----------------------------------+ 7 | EXCHANGE ORIGINATOR S_ID | 3 Bytes +----------------------------------+ 8 | RES OX_ID | 2 Bytes +----------------------------------+ 10 | RES RX_ID | 2 Bytes +----------------------------------+ Total Length = 12 Upon transmitting the RES Link Service message to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR S_ID field. Furthermore, the iFCP layer shall add the EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. The EXCHANGE ORIGINATOR is a 4-byte field. When 0x00000001, itindicates that the RES requester is the originating N_PORT of the exchange for which status is being requested. When 0x00000000, indicates that the RES recipient is the originator of the exchange for which status is being requested. RES OX_ID - Specifies the OX_ID of the exchange for which status is being requested. RES_RX_ID - Specifies the RX_ID of the exchange for which status is being requested. If the RES recipient does not have an Exchange Status Block for the specified exchange, it shall respond with an LS_RJT message with a reason code of "UNABLE TO PERFORM COMMAND REQUEST". Monia, Mullendore, Tseng 25 iFCP January 2001 Upon receipt of the RES Link Service message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the EXCHANGE ORIGINATOR S_ID field before forwarding the message on to the upper Fibre Channel layers. The payload for the ACC response is a variable-length block of information called the EXCHANGE STATUS BLOCK. The Exchange Status Block (ESB) is used throughout an exchange to track the exchange's progress and identify which ports holds sequence initiative. The ESB has the following format shown below. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | ESB OX_ID | 1 Byte +----------------------------------+ 2 | ESB RX_ID | 2 Bytes +----------------------------------+ 4 | RESERVED | 1 Byte +----------------------------------+ 5 | ORIGINATOR N_PORT ID | 3 Bytes +----------------------------------+ 4 | RESERVED | 1 Byte +----------------------------------+ 5 | RESPONDER N_PORT ID | 3 Bytes +----------------------------------+ 6 | E_STAT | 4 Bytes +----------------------------------+ | RESERVED | 4 Bytes +----------------------------------+ 8 | SERVICE PARAMETERS | 112 Bytes +----------------------------------+ 88 | OLDEST SHORT SSB | 8 Bytes +----------------------------------+ 96 | INTERMEDIATE SHORT SSB | X*8 Bytes +----------------------------------+ 96 + X*8 | NEWEST SHORT SSB | 8 Bytes +----------------------------------+ Total Length = 104 + X*8 ESB OX_ID - The OX_ID for the exchange status saved in the ESB. ESB RX_ID - The RX_ID for the exchange status saved in the ESB. Upon transmitting the ACC reply to the IP network, the iFCP layer shall clear the ORIGINATOR N_PORT_ID and RESPONDER N_PORT ID fields. Furthermore, the iFCP layer shall add a 4-byte EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. When the EXCHANGE ORIGINATOR field is 0x00000001, this indicates that the port sending the RES message (or receiving the ACC reply) is the originator of the exchange whose status is saved in the ESB. Monia, Mullendore, Tseng 26 iFCP January 2001 When the EXCHANGE ORIGINATOR is 0x00000000, this indicates that the port receiving the RES message (or sending the ACC reply) is the originator of the exchange whose status is saved in the ESB. Upon receipt of the ACC reply from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the ORIGINATOR and RESPONDER N_PORT ID fields before forwarding the message on to the upper Fibre Channel layers. More information on RES and the Exchange Status Block (ESB) can be found in [FC-PH]. 7.1.7 Request Sequence Initiative (RSI) RSI allows a sequence recipient to request sequence initiative be passed for an open exchange. The RSI recipient responds in the following manner if the RSI request identifies an open exchange. The following shows the format of the RSI request message. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0x12000000) | 4 Bytes +----------------------------------+ 4 | RESERVED | 1 Byte +----------------------------------+ 7 | EXCHANGE ORIGINATOR S_ID | 3 Bytes +----------------------------------+ 8 | RSI OX_ID | 2 Bytes +----------------------------------+ 10 | RSI RX_ID | 2 Bytes +----------------------------------+ Total Length = 12 Upon transmitting the RSI Link Service message to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR S_ID field. Furthermore, the iFCP layer shall add a 4-byte EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. The EXCHANGE ORIGINATOR field, when 0x00000001, indicates that the RSI requester is the originator of the exchange for which initiative is being requested. When EXCHANGE ORIGINATOR is set to 0x00000000, this indicates the RSI recipient is the originator of the exchange for which initiative is being requested. RSI OX_ID - Specifies the OX_ID of the exchange for which sequence initiative is being requested. RSI RX_ID - Specifies the RX_ID of the exchange for which sequence initiative is being requested. Monia, Mullendore, Tseng 27 iFCP January 2001 Upon receipt of the RSI message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the EXCHANGE ORIGINATOR S_ID field before forwarding the message on to the upper Fibre Channel layers. An ACC response is sent after the sequence initiative has been transferred, or if it already has been transferred. The ACC response only has the 4-byte LS_COMMAND field as the payload. An LS_RJT response may be sent if the RSI parameters do not specify an open exchange (invalid OX_ID/RX_ID combination). 7.1.8 Read Sequence Status (RSS) RSS requests the status of a specific sequence in an exchange. The following shows the format of the RSS request. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0x09000000) | 4 Bytes +----------------------------------+ 4 | SEQ_ID | 1 Byte +----------------------------------+ 5 | EXCHANGE ORIGINATOR S_ID | 3 Bytes +----------------------------------+ 8 | RSS OX_ID | 2 Bytes +----------------------------------+ 10 | RSS RX_ID | 2 Bytes +----------------------------------+ Total Length = 12 Upon transmitting the RSS Link Service message to the IP network, the iFCP layer shall clear the EXCHANGE ORIGINATOR S_ID field. Furthermore, the iFCP layer shall add a 4-byte EXCHANGE ORIGINATOR to the iFCP AUGMENTED DATA field in the iFCP header. The EXCHANGE ORIGINATOR field, when 0x00000001, indicates that the RSS requester is the originator of the exchange for which sequence status is being requested. When EXCHANGE ORIGINATOR is set to 0x00000000, this indicates the RSS recipient is the originator of the exchange for which sequence status is being requested. SEQ_ID - Specifies the SEQ_ID of the sequence for which status is being requested. RSS OX_ID - Specifies the OX_ID of the exchange for which sequence status is being requested. RSS RX_ID - Specifies the RX_ID of the exchange for which sequence status is being requested. Monia, Mullendore, Tseng 28 iFCP January 2001 Upon receipt of the RSS message from the IP network, the iFCP layer shall use the N_PORT fabric addresses (in the Fibre Channel header) and the EXCHANGE ORIGINATOR value (in the iFCP header) to replace the EXCHANGE ORIGINATOR S_ID field before forwarding the message on to the upper Fibre Channel layers. The ACC response payload for an RSS request consists of a single 16-byte field, the Sequence Status Block (SSB), shown below. If the RSS recipient does not have a Sequence Status Block, it shall respond with an LS_RJT with a reason code of "UNABLE TO PERFORM COMMAND REQUEST". More information on the Sequence Status Block can be found in [FC-PH]. 7.2 TCP Link Service Messages TCP Link Service Messages are used to manage TCP connections. They are passed between peer FCP Portals, and are only processed within the iFCP layer. The response to the TCP Link Service Message (if any) will echo the original request. The LS_COMMAND value for the response remains the same as that used for the request. Additionally, the ABTS request shall never be generated for any TCP Link Service Message. {Editor's note: Since these messages are never passed to the fibre channel device, the use of the FC ELS format is not required. However, leveraging the format may benefit a gateway implementation. Depending on the tradeoffs, therefore, the format may be modified to eliminate use of the ELS as a message template.] The Link Service frame carrying a TCP ELS message is identified by the TCP ELS bit being set in the iFCP FLAGS field of the iFCP header. Additionally, the TYPE field is 0x01 and R_CTL field is 0x22 for the request, and 0x23 for the reply. The following lists the TCP Link Service messages and their corresponding LS_COMMAND values. Request LS_COMMAND Short Name iFCP Support ------- ---------- ---------- ------------ Control Connection Bind 0xE0 CBIND REQUIRED Unbind Connection 0xE4 UNBIND OPTIONAL TCP Message 0xE8 TCPMSG REQUIRED Network Connection Interfaces 0xED NINTF REQUIRED 7.2.1 Network Connection Interfaces (NINTF) NINTF allows an FCP Portal to request information on other network interfaces that may be used to establish connections with the responding gateway implementation. This extended link service will return the number of network interfaces available, and an interface descriptor record for a single interface. Since each NINTF request Monia, Mullendore, Tseng 29 iFCP January 2001 returns information on one interface, multiple NINTF requests are required to obtain information on more than one interface. The following shows the format of the NINTF request message. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xED000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | INTERFACE KEY | 2 Bytes +----------------------------------+ 10 | RESERVED | 2 Bytes +----------------------------------+ Total Length = 12 USER INFO - Contains any data desired by the requester. The value will be echoed by the recipient. INTERFACE KEY - Contains an index to the interface for which the NINTF message is querying. Each interface at the destination shall be sequentially numbered beginning with 1. If the number of interfaces supported by the message recipient is unknown, then this field shall contain 0. In the NINTF response, the recipient will indicate the number of interfaces supported. Each of these interfaces can be referenced in subsequent NINTF messages by the sender by setting the INTERFACE KEY value up to the highest- numbered interface. The following shows the format of the NINTF response. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xED000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | RESERVED | 3 Bytes +----------------------------------+ 11 | INTERFACES AVAILABLE (A) | 1 Byte +----------------------------------+ 12 | INTERFACE RECORDS | X Bytes +----------------------------------+ Total Length = X + 12 USER INFO - The 4-byte field is the same value as the USER INFO in the NINTF request. The recipient echoes this value back to the sender, and does not perform any operation using this field. Monia, Mullendore, Tseng 30 iFCP January 2001 INTERFACES AVAILABLE (A) - This parameter specifies the number of additional network interfaces that may be used to establish TCP connections. The value stored in this field also specifies the number (A) of network interface records that are present at the end of the message. INTERFACE RECORDS - This field contains A interface records describing each of the network interfaces. An interface record consists of 5 parameters as shown in below. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | RECORD LENGTH | 1 Byte +----------------------------------+ 1 | IP ADDRESS TYPE | 1 Byte +----------------------------------+ 2 | INTERFACE HANDLE | 2 Bytes +----------------------------------+ 4 | RESERVED | 4 Bytes +----------------------------------+ 8 | INTERFACE SPEED | 4 Bytes +----------------------------------+ | IP ADDRESS | X-12 Bytes +----------------------------------+ Total Length = X RECORD LENGTH - Defines the total length, in bytes, of the INTERFACE RECORD, including the RECORD LENGTH field. This value shall be a multiple of 4 bytes. IP ADDRESS TYPE - Defines the type of address in the IP ADDRESS field. 0x01 indicates IPv4, 0x02 indicates Ipv6. INTERFACE HANDLE - This 16-bit field contains an identifying number (i.e., handle) assigned to the interface by the destination N_PORT. INTERFACE SPEED - This parameter specifies the data rate of the interface in bits per second. The value in this field is the data rate divided by 1024. For example, a value of 1024 indicates a data rate of 1048576 bits per second. IP ADDRESS - This field contains the IP address of the network interface for which information is being returned. If the address type is N bytes long and the field is larger than N, the address shall be in the first N bytes of the field with the remainder of the field set to 0. 7.2.2 Connection Bind (CBIND) The CBIND message binds an N_PORT login session to a specific TCP connection. In the CBIND request message, the source and destination N_Port is identified by its N_PORT network address. Monia, Mullendore, Tseng 31 iFCP January 2001 The following shows the format of the CBIND request. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xE0000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | SOURCE PORT NAME | 8 Bytes +----------------------------------+ Length = 16 USER INFO - Contains any data desired by the requester. This info is echo-ed back by the recipient. SOURCE PORT NAME - Contains the originating N_PORT's World Wide Port Name (WWPN). The FCP Portal uses this to verify that there is no pre-existing N_PORT session between the source and destination N_PORTs. [The response to this error condition will be handled in a future release of this specification] The following shows the format of the CBIND response. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xE0000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | DESTINATION PORT NAME | 8 Bytes +----------------------------------+ 16 | RESERVED | 2 Bytes +----------------------------------+ 18 | CBIND STATUS | 2 Bytes +----------------------------------+ 20 | RESERVED | 2 Bytes +----------------------------------+ 22 | CONNECTION HANDLE | 4 Bytes +----------------------------------+ Total Length = 26 USER INFO - Contains the same value received in the USER INFO field of the CBIND request message. DESTINATION PORT NAME - Contains the destination N_PORT's World Wide Port Name (WWPN). CBIND STATUS - Indicates success or failure of the CBIND request. CBIND values are shown below. Monia, Mullendore, Tseng 32 iFCP January 2001 Value Description ----- ----------- 0 Successful - No Other Status 1-15 Reserved 16 Failed - Unspecified Reason 17 Failed - No Such device 18 Failed - N_PORT session already exists 19 Failed - Lack of Resources Others Reserved CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the control connection 7.2.4 Unbind Connection (UNBIND) UNBIND is used to release a bound TCP connection and return it to the pool of unbound TCP connections. This message is transmitted in the connection that is to be unbound. The following is the format of the UNBIND request message. Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xE4000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | CONNECTION HANDLE | 4 Bytes +----------------------------------+ 12 | RESERVED | 8 Bytes +----------------------------------+ Total Length = 20 CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the connection The following shows the format of the UNBIND response message. Monia, Mullendore, Tseng 33 iFCP January 2001 Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xE4000000) | 4 Bytes +----------------------------------+ 4 | USER INFO | 4 Bytes +----------------------------------+ 8 | CONNECTION HANDLE | 4 Bytes +----------------------------------+ 16 | RESERVED | 10 Bytes +----------------------------------+ 26 | UNBIND STATUS | 2 Bytes +----------------------------------+ 28 | RESERVED | 2 Bytes +----------------------------------+ Total Length = 26 UNBIND STATUS - Indicates the success or failure of the UNBIND request. Value Description ----- ----------- 0 Successful - No Other Status 1-15 Reserved 16 Failed - Unspecified Reason 17 Failed - No Such device 18 Failed - Connection ID Invalid Others Reserved CONNECTION HANDLE (CHANDLE) - Contains a value assigned by the FCP Portal to identify the unbound connection. 7.2.5 TCP Message (TCPMSG) TCPMSG sends an error message to another iFCP port. TCPMSG differs from other messages in that there is no reply to TCPMSG (both the first and last sequence in a exchange). The primary purpose for TCPMSG is to generate a message informing an iFCP port that a fatal FCP/TCP protocol error was detected, and all connections established with the iFCP port are being closed. TCPMSG can also be used to send "Informative" or "Warning" messages that may be used for debugging or diagnostic purposes. The format of the TCPMSG request message follows. Monia, Mullendore, Tseng 34 iFCP January 2001 Byte MSb LSb Offset 7 6 5 4 3 2 1 0 +----------------------------------+ 0 | LS_COMMAND (0xEE000000) | 4 Bytes +----------------------------------+ 4 | RESERVED | 4 Bytes +----------------------------------+ 8 | ERROR CODE | 2 Bytes +----------------------------------+ 10 | TCPMSG FLAGS | 1 Byte +----------------------------------+ 11 | MSG LENGTH (L) | 1 Byte +----------------------------------+ 12 | MSG | L Bytes +----------------------------------+ Total Length = L + 12 ERROR CODE - Specifies one of the predefined error messages shown in the following table. This field is valid only if the FATAL bit is 1 and MSG LENGTH is 0 in the TCPMSG FLAGS field. Error Code Message ---------- ------- 0x0001 Loss of Synchronization on Connection Others RESERVED TCPMSG FLAGS - This field contains 3 bit flags that specify how the recipient should interpret the received message. Bit Field Flag Description --------- ---- ----------- 7:3 RESERVED 2 INFORMATIVE Indicates the message is informative, usually for debugging purposes. These messages may be discarded. 1 WARNING Indicates the message is a warning. Processing of warning messages is required and implementation-specific. 0 FATAL Indicates a fatal protocol error has been detected. Sender is terminating login sessions with the recipient and closing all TCP connections. The recipient shall implicit logout the sender of the message and close TCP connections to the sender. A WARNING or INFORMATIVE message shall not cause the recipient to alter the operating environment. When more than one TCPMSG FLAG Monia, Mullendore, Tseng 35 iFCP January 2001 bit is set, the message shall be considered Fatal. When no flags are set, the message shall be discarded. MSG LENGTH - Specifies the length in bytes of the MSG field. The length must be a multiple of 4 and can be a value of between 0 and 128. A value of 0 indicates the MSG field is not present. 8 Error Detection and Recovery Procedures for iFCP 8.1 Overview [FCP-2], [FC-PH], and [FC-PH-2] define error detection and recovery procedures. These Fibre Channel-defined mechanisms continue to be available in the iFCP environment. 8.2 Timer Definitions 8.2.1 Error_Detect_Timeout (E_D_TOV) E_D_TOV is "a reasonable timeout value for detection of a response to a time event". The default value specified by FC-PH of 10 seconds will be also used as the iFCP default value. E_D_TOV is the maximum time allowed between the transmission of consecutive data frames within a sequence. For Class 2 service, E_D_TOV specifies the maximum time interval between transmission of a frame, and receipt of the ACK for that frame. [The policy for setting E_D_TOV for an IP fabric is TBS] 8.2.2 Resource Allocation Timeout (R_A_TOV) R_A_TOV is defined in FC-PH-2 as "the maximum transit time within a fabric to guarantee that a lost frame will never emerge from the fabric". A value of 2 x R_A_TOV is the minimum time that the originator of an ELS request or FC-4 ELS request shall wait for the response to that request. [The policy for setting R_A_TOV for an IP fabric is TBS] 8.2.3 Resource Recovery Timer (RR_TOV) [The content of this section is TBD] 8.3 TCP Error Recovery Issues A failed TCP connection will result in a dropped N_PORT session. [The remainder of this section is TBD] 8.4 iFCP Protocol Error Monia, Mullendore, Tseng 36 iFCP January 2001 iFCP protocol errors between FCP Portals shall be considered fatal errors resulting in the termination of the login sessions and closing of the TCP sessions. An iFCP protocol error occurs when Fibre Channel frames are sent on the wrong TCP connection. One example of a protocol error is receiving an FCP_CMND IU on the data connection. If an iFCP port detects an iFCP/TCP protocol error on a connection, the port shall transmit a TCPMSG message on the control connection (if one exists) with the appropriate error code. The FCP_Portal shall then implicitly log out and close all TCP connections established with the iFCP port, and ignore all data received on these TCP connections until they are reopened. [The information returned to the N_PORT upon occurence of an iFCP protocol error will be specified in the next revision of this specification] 9. Security 9.1 Overview As with any other IP-based network, an iFCP storage network has security issues which must be addressed with the appropriate security policies and enforcement resources. There are various levels of security paradigms which when applied appropriately to an iFCP network can provide sufficient levels of security, including data integrity, authentication, and privacy, depending on user needs. 9.2 Physical Security Most existing SCSI and Fibre Channel interconnections are deployed in private, physically isolated environments where hostile entities are not provided access to the SCSI and Fibre Channel interconnects. This is the most basic security mechanism, and may be a sufficient model in some cases for an iFCP network. 9.3 Controlling Access A second level of security is the use of zoning. Zoning specifies which devices are allowed to communicate, and is similar in concept to VLAN (Virtual Local Area Network) technology. Zoning information is maintained in a Name Server. 9.4 Authentication and Encryption Where additional levels of data integrity and privacy are required for iFCP, existing IPSec specifications can be applied to iFCP. Because IPSec is a layer-3 technology and has no knowledge of TCP, UDP, or higher-level protocols such as iFCP and FCP, it can be Monia, Mullendore, Tseng 37 iFCP January 2001 applied transparently to iFCP. The following IETF documents describe the operational framework and automatic keying mechanisms for IPSec. RFC2401 Security Architecture for the Internet Protocol RFC2402 IP Authentication Header RFC2406 IP Encapsulating Security Payload RFC2407 The Internet IP Security Domain of Interpretation for ISAKMP RFC2408 Internet Security Association and Key Management Protocol (ISAKMP) RFC2409 The Internet Key Exchange (IKE) 9.5 Storage Firewalls Firewalls are a common and proven methodology for securing access to IP-based networks, and they can be appropriate for use in IP- based storage networks as well. A firewall is a choke point through which all transit traffic must transit in order to pass between two separate networks. Since all iFCP traffic uses a well- known IANA-assigned TCP port number, it can easily be recognized and inspected. Access to storage resources can be secured by setting up a single gateway through which all outside non-secured traffic must pass through in order to access resources in the storage network. Such a firewall can be a proxy host operating at the session or application layer, requiring authentication before allowing traffic to pass. It can also be a stateful inspection gateway which understands the iFCP protocol, and can passively inspect and discover security threats as they transit the gateway. A third option is to use a standard router access control list to filter authorized traffic based upon static parameters such as IP addresses and TCP port numbers. 10. References 10.1 Relevant SCSI (T10) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854-7179 or (303) 792-2181, Fax: (303) 792-2192 [SAM] SCSI-3 Architecture Model (SAM), ANSI X3.270-1996 [SAM-2] SCSI Architecture Model-2 (SAM-2), Project 1157-D, revision 11 [SPC] SCSI Primary Commands (SPC), ANSI X3.301-1997 [SPC-2] SCSI Primary Commands-2 (SPC-2), Project 1236-D, revision 16 Monia, Mullendore, Tseng 38 iFCP January 2001 [FCP] Fibre Channel Protocol for SCSI (FCP), ANSI X3.269- 1996 [FCP-2] Fibre Channel Protocol for SCSI, Second Revision (FCP- 2), Project 1144D, revision 04 10.2 Relevant Fibre Channel (T11) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854-7179 or (303) 792-2181, Fax: (303) 792-2192 [FC-PH] Fibre Channel Physical and Signaling Interface (FC-PH) Rev 4.3, ANSI X3.230:1994 [FC-PH-2] Fibre Channel Physical and Signaling Interface (FC-PH- 2) Rev 7.4, ANSI X3.297:1997 [FC-PH-3] Fibre Channel Physical and Signaling Interface (FC-PH- 3) Rev 9.4, ANSI X3.303:1998 [FC-FG] Fibre Channel Generic Requirements (FC-FG) Rev 3.5, ANSI X3.289:1996 [FC-GS-2] Fibre Channel Generic Services (FC-GS-2) Rev 5.2, ANSI NCITS 288 [FC-AL] Fibre Channel Arbitrated Loop (FC-AL) Rev 4.5, ANSI X3.272:1996 [FC-AL-2] Fibre Channel Arbitrated Loop (FC-AL-2) Rev 7.0, NCITS 332:1999 [FC-PLDA] Fibre Channel Private Loop SCSI Direct Attachment (FC- PLDA), NCITS TR-19:1998 [FC-FLA] Fibre Channel Fabric Loop Attachment (FC-FLA), NCITS TR-20:1998 [FC-TAPE] Fibre Channel Tape and Tape Medium Changers (FC-TAPE), NCITS TR-24:1999 10.3 Relevant RFC Documents [RFC768] User Datagram Protocol [RFC791] Internet Protocol, DARPA Internet Program Protocol Specification [RFC1146] TCP Alternate Checksum Options Monia, Mullendore, Tseng 39 iFCP January 2001 [RFC2401] Security Architecture for Internet Protocol [RFC2402] IP Authentication Header [RFC2406] Encapsulating Security Protocol (ESP) [RFC2407] The Internet IP Security Domain for ISAKMP [RFC2408] Internet Security Association and Key Management Protocol (ISAKMP) [RFC2409] The Internet Key Exchange (IKE) [RFC2460] Internet Protocol, Version 6 (IPv6) Specification 10.4 Other Reference Documents Fibre Channel, Gigabit Communications and I/O for Computer Networks, Alan F. Beener, McGraw-Hill, ISBN 0-07-005669-2 The Fibre Channel Consultant, A Comprehensive Introduction, Robert W. Kembel, Northwest Learning Associates, ISBN 0-931836-82-6 The Fibre Channel Consultant, Arbitrated Loop, Rober W. Kembel, Connectivity Solutions, a division of Northwest Learning Associates, ISBN 0-931836-84-0 Monia, Mullendore, Tseng 40 iFCP January 2001 11. Author's Addresses Charles Monia Rod Mullendore Josh Tseng Nishan Systems 3850 North First Street San Jose, CA 95134 Phone: 408-519-3986 Email: cmonia@nishansystems.com David Robinson Sun Microsystems Senior Staff Engineer M/S UNWK02-107 901 San Antonio Road Palo Alto, CA 94303-4900 Phone: 510-574-9226 Email: david.robinson@ebay.sun.com Franco Travostino Nortel Networks Director, Content Internetworking Lab 3 Federal Street Billerica, MA 01821 Phone: 978-288-7708 Email: travos@nortelnetworks.com Wayland Jeong Troika Networks Vice President, Hardware Engineering 2829 Townsgate Road Suite 200 Westlake Village, CA 91361 Phone: 805-370-2614 Email: wayland@troikanetworks.com Rory Bolt Quantum/ATL Director, System Design 101 Innovation Drive Irvine, CA 92612 Phone: 949-856-7760 Email: rbolt@atlp.com Monia, Mullendore, Tseng 41 iFCP January 2001 Paul Rutherford ADIC Vice President, Technology & Software 1143 Willows Road N.E. P.O. Box 97057 Redmond, WA 98073-9757 Phone: 425-881-8004 Email: paul.rutherford@adic.com Mark Edwards Senior Systems Architect Eurologic Development, Ltd. 4th Floor, Howard House Queens Ave, UK. BS8 1SD Phone: +44 (0)117 930 9600 Email: medwards@eurologic.com Monia, Mullendore, Tseng 42 iFCP January 2001 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implmentation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." Monia, Mullendore, Tseng 43 iFCP January 2001 Appendix A A. iFCP Transparent Mode [Editor's Note: This section is the initial draft of a proposed architectural extension to iFCP.] This section describes an architectural extension of iFCP which provides for the allocation of fabric-unique N_PORT addresses as alternative to the gateway-local N_PORT addressing method described in the main portion of the document. This alternative may be useful as the basis for iFCP implementations which support non-storage FC-4 protocols. One of the chief attributes of the basic iFCP protocol is the mapping of gateway-assigned Fibre Channel N_PORT IDs to global IPv4 or IPv6 addresses along with the local assignment of N_PORT ID values within the scope of the gateway. This mapping allows the internetworking of a virtually unlimited number of Fibre Channel devices. It also simplifies management of the IP storage fabric configuration by eliminating the need for a centralized address-assignment authority. More importantly, by decoupling N_PORT network addresses from the constraints of Fibre Channel address space management, it improves the scalability of iFCP gateway configurations. These constraints are a result of the manner in which Fibre Channel fabrics manage address assignment. In a switched fabric, this is done hierarchically by subdividing the 24-bit address space into 65K blocks, which are distributed to switch elements. A switch element, in turn, assigns N_PORT addresses out of this 65K pool as attached devices log into the fabric. A fabric using this address allocation scheme is limited to 239 switch elements. This is not a serious limitation for small fabrics. As the system expands, however, fabrics may consist of many switch elements distributed throughout the enterprise, each of which controls a small number of devices. In this case, the limitation in switch count may become a barrier to fully integrating the storage network. The alternative method for N_PORT address assignment described in this section, therefore, is proposed as an alternative where address transparency is considered more important than connectivity. A.1 Definitions Logical iFCP Fabric _ A collection of iFCP gateways that co- ordinate the assignment of N_PORT addresses such that each N_PORT address is unique within the scope of the fabric. Monia, Mullendore, Tseng 44 iFCP January 2001 DOMAIN_ID _ The value contained in the high-order byte of a 24-bit N_PORT address. A.2 Architectural Overview iFCP Transparent Mode is a variant of the basic iFCP protocol described in the main section of this document. A prerequisite for transparent mode operation is to create a logical iFCP fabric and configure it with the gateways to interoperate in transparent mode. This configuration may be done manually or through the services of an iSNS name server. Since a given IP network may contain several logical iFCP fabrics as well as gateways operating in non-transparent mode, it is likely that one or more N_PORTs within the IP network will have the same N_PORT ID. Consequently, it is necessary to prevent data corruption due to extraneous frame traffic entering the transparent-mode gateway from outside the logical fabric. Therefore, a gateway operating in transparent mode: a) MUST only be a member of one logical fabric. b) MUST NOT establish N_PORT login sessions with, send frames to or accept frames from gateways that are not a member of its logical fabric. c) MUST NOT accept or send frames to gateways not in transparent mode. In a logical iFCP fabric, unique N_PORT addresses are obtained by assigning a fabric-unique DOMAIN_ID to each member gateway. The gateway, in turn, assigns fabric-unique N_PORT ID's to each directly attached Fibre Channel device. Once each FC end node is addressed by unique N_PORT ID's, no modification of the S_ID and D_ID is required, and no augmentation of any Link Service Commands is needed. Destination IP addresses can be found with a quick table lookup using the destination N_PORT ID (i.e., D_ID) as the key. The original frame can then be encapsulated without modification and forwarded to the appropriate iFCP Portal. A.3 iFCP Transparent Mode Differences The following is a summary of differences in using Transparent Mode compared to basic iFCP: 1) There is no need to augment any Link Service Commands, as specified in section 7 of the main iFCP document. 2) No address translation of S_ID and D_ID values is needed in the transmission of iFCP frames between iFCP gateways. Monia, Mullendore, Tseng 45 iFCP January 2001 A.4 Important iFCP Transparent Mode Considerations There are important limitations for consideration when using iFCP Transparent Mode. These are as follows: 1) A maximum of 238 DOMAIN_ID values are available for allocation to iFCP gateways. This provides a hard theoretical limit to the maximum size of the iFCP fabric operating in Transparent Mode. 2) N_PORT fabric address space must be allocated in 65,536 address "chunks", likely leading to relatively inefficient use of the available 3-byte address space. 65,536 addresses must be allocated to each iFCP gateway requesting address space, regardless of the number of devices supported by the gateway. 3) iFCP transparent mode increases the dependency on the iSNS, which is the central address-assignment authority. If connectivity with the iSNS is lost, new DOMAIN_ID values cannot be automatically allocated, and new iFCP gateways cannot be automatically added to the fabric. Of course, it is always possible to add and manage additional such gateways manually. 4) Multiple iFCP gateways set up with independently-administered iSNS servers must be completely torn down and slaved under a single iSNS authority, before they can be configured into the same logical fabric. In contrast, the iFCP described in the main section of this document requires only that independent iSNS servers import client attributes from other iSNS servers, before clients under different iSNS authorities can be made to interoperate. 5) iFCP gateways in transparent mode will not interoperate with iFCP gateways that are not in transparent mode. 6) When interoperating with Fibre Channel fabrics, the iFCP gateway MUST assume control of DOMAIN_ID assignments in accordance with the appropriate Fibre Channel standard or specification. DOMAIN_ID values assigned to FC switches in attached fabrics must be acquired from the iSNS. A.5 iFCP Transparent Mode Requirements The following requirements exist for iFCP gateways operating in Transparent Mode: A.5.1 Allocation of DOMAIN_ID Values In Fibre Channel, addresses are 24-bit values. The high-order 8- bits of the address comprise the DOMAIN_ID, and may range in value between 1 and 239. Monia, Mullendore, Tseng 46 iFCP January 2001 In order to assign a fabric-unique value to each N_PORT ID, an iFCP gateway in transparent mode shall acquire a unique DOMAIN_ID by querying the iSNS server or through manual allocation. iFCP gateways in Transparent Mode SHALL only assign to N_PORTs address values which are globally-unique within the iFCP fabric. It accomplishes this by using its assigned DOMAIN_ID and creating addresses with values allocated from the lower-order 16-bits. Each N_PORT interface in the entire iFCP logical fabric SHALL have a unique N_PORT_ID value. A.5.2 Incompatibility with "Main Mode" iFCP iFCP gateways in transparent mode shall not originate or accept frames that do not have bit 9 ("iFCP TRANSPARENT MODE") set to one in the FLAG field. The iFCP gateway shall immediately terminate any N_PORT sessions with the iFCP gateway from which it receives such frames. A.5.3 Encapsulation Header FLAGS Settings The following values shall be considered legal and usable values in the FLAGS field of the encapsulation header: Bit Field FLAG Legal Value for FLAGS field --------- ---- --------------------------- 13 non-iFCP MUST be 1 10-12 RESERVED MUST be 1 9 iFCP TRANSPARENT MODE MUST be 1 4-8 RESERVED for iFCP MUST be 0 3 iFCP DATA CRC MUST be 1 2 TCP ELS MUST be 0 1 AUGMENTATION PRES MUST be 0 0 COMPLIANCE MUST be 1 A.5.4 Fibre Channel CRC The iFCP Transparent Mode frame SHALL include the trailing CRC. This shall be indicated in the encapsulation header by enabling bit 19 in the FLAGS field. Since there is no AUGMENTED DATA field in the encapsulation header, the value in the iFCP CRC field should be a copy of the CRC in the original Fibre Channel frame. Consequently, the original CRC can be used in the iFCP CRC field without recalculation. A.5.5 Applicability of Main iFCP Specification Sections iFCP gateways operating in Transparent Mode shall comply with all of the main sections of this iFCP document except for section 3.3 (Address Translation), and section 7 (Augmented Link Services). In iFCP Transparent Mode, no Fibre Channel Address Translation shall Monia, Mullendore, Tseng 47 iFCP January 2001 take place, and no Link Service Messages shall be augmented with additional information by the iFCP layer. 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 Monia, Mullendore, Tseng 48