IPS Working Group M. Rajagopal
INTERNET-DRAFT Technical Coordinator
<draft-ietf-ips-fcovertcpip-09.txt>
(Expires August, 2002) E. Rodriguez
Category: standards-track ips Liaison
R. Weber
Editor
Fibre Channel Over TCP/IP (FCIP)
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026 [1].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as Reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/lid-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Abstract
Fibre Channel Over TCP/IP (FCIP) describes mechanisms that allow the
interconnection of islands of Fibre Channel storage area networks
over IP-based networks to form a unified storage area network in a
single Fibre Channel fabric. FCIP relies on IP-based network
services to provide the connectivity between the storage area
network islands over local area networks, metropolitan area
networks, or wide area networks.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [2].
Rajagopal, et al. Standards Track [Page 1]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Table Of Contents
1. Editors and Contributors . . . . . . . . . . . . . . . . . . . . 3
2. Purpose, Motivation and Objectives . . . . . . . . . . . . . . . 4
3. Relationship to Fibre Channel Standards . . . . . . . . . . . . 5
3.1 Relevant Fibre Channel Standards . . . . . . . . . . . . . . . 5
3.2 This Specification and Fibre Channel Standards . . . . . . . . 6
4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 6
5. Protocol Summary . . . . . . . . . . . . . . . . . . . . . . . . 7
6. The FCIP Model . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.1 FCIP Protocol Model . . . . . . . . . . . . . . . . . . . . . . 9
6.2 FCIP Link . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.3 FC Entity . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.4 FCIP Entity . . . . . . . . . . . . . . . . . . . . . . . . . 12
6.5 FCIP Link Endpoint (FCIP_LEP) . . . . . . . . . . . . . . . . 13
6.6 FCIP Data Engine (FCIP_DE) . . . . . . . . . . . . . . . . . . 14
6.6.1 FCIP Encapsulation of FC Frames . . . . . . . . . . . . . . 16
6.6.2 FCIP Data Engine Error Detection and Recover . . . . . . . . 19
6.6.2.1 TCP Assistance With Error Detection and Recovery . . . . . 19
6.6.2.2 Errors in FCIP Headers and Discarding FCIP Frames . . . . 19
6.6.2.3 Synchronization Failures . . . . . . . . . . . . . . . . . 21
7. Checking FC Frame Transit Times in the IP Network . . . . . . . 22
8. The FCIP Special Frame . . . . . . . . . . . . . . . . . . . . 23
9. TCP Connection Management . . . . . . . . . . . . . . . . . . . 26
9.1 TCP Connection Establishment . . . . . . . . . . . . . . . . . 26
9.1.1 Connection Establishment Model . . . . . . . . . . . . . . . 26
9.1.2 Creating New TCP Connections . . . . . . . . . . . . . . . . 27
9.1.2.1 Non-Dynamic Creation of New TCP Connections . . . . . . . 27
9.1.2.2 Dynamic Creation of New TCP Connections . . . . . . . . . 27
9.1.2.3 Connection Setup After a Successful TCP Connect Request . 28
9.1.3 Processing Incoming TCP Connect Requests . . . . . . . . . . 30
9.2 Closing TCP Connections . . . . . . . . . . . . . . . . . . . 34
9.3 TCP Connection Parameters . . . . . . . . . . . . . . . . . . 34
9.3.1 TCP Selective Acknowledgement Option . . . . . . . . . . . . 34
9.3.2 TCP Window Scale Option . . . . . . . . . . . . . . . . . . 34
9.3.3 Protection against sequence number wrap . . . . . . . . . . 34
9.3.4 TCP_NODELAY Option . . . . . . . . . . . . . . . . . . . . . 34
9.4 TCP Connection Considerations . . . . . . . . . . . . . . . . 35
9.5 Flow Control Mapping between TCP and FC . . . . . . . . . . . 35
10. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
10.1 Threat Models . . . . . . . . . . . . . . . . . . . . . . . . 36
10.2 FC Fabric and IP Network Deployment Models . . . . . . . . . 37
10.3 FCIP Security Components . . . . . . . . . . . . . . . . . . 37
10.3.1 IPSec ESP Authentication and Confidentiality . . . . . . . 38
10.3.2 Key Management . . . . . . . . . . . . . . . . . . . . . . 38
10.3.3 ESP Replay Protection and Rekeying issues . . . . . . . . . 40
10.4 Secure FCIP Link Operation . . . . . . . . . . . . . . . . . 40
10.4.1 FCIP Link Initialization Steps . . . . . . . . . . . . . . 40
Rajagopal, et al. Standards Track [Page 2]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
10.4.2 TCP Connection Security Associations (SAs) . . . . . . . . 41
10.4.3 Handling data integrity and confidentiality violations . . 41
10.4.4 Handling SA parameter mismatches . . . . . . . . . . . . . 41
11. Performance . . . . . . . . . . . . . . . . . . . . . . . . . 41
11.1 Performance Considerations . . . . . . . . . . . . . . . . . 41
11.2 IP Quality of Service (QoS) Support . . . . . . . . . . . . . 43
12. Normative References . . . . . . . . . . . . . . . . . . . . . 43
13. Informative References . . . . . . . . . . . . . . . . . . . . 45
14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 46
15. Contributors' Addresses . . . . . . . . . . . . . . . . . . . 46
16. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 48
Annex
A IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 48
B FCIP Usage of Addresses and Identifiers . . . . . . . . . . . . 48
C Example of synchronization recovery algorithm . . . . . . . . . 49
D Relationship between FCIP and IP over FC (IPFC) . . . . . . . . 54
E FC Frame Format . . . . . . . . . . . . . . . . . . . . . . . . 54
F FC Encapsulation Format . . . . . . . . . . . . . . . . . . . . 56
G FCIP Requirements on an FC Entity . . . . . . . . . . . . . . . 58
1. Editors and Contributors
During the development of this specification, Murali Rajagopal,
Elizabeth Rodriguez, Vi Chau, and Ralph Weber served consecutively
as editors. Raj Bhagwat contributed substantially to the initial
basic FCIP concepts.
Venkat Rangan contributed the Security section and continues to
coordinate security issues with the ips Working Group and IETF.
Andy Helland contributed a substantial revision of Performance
section, aligning it with TCP/IP QoS concepts.
Dave Peterson contributed the dynamic discovery section and edits
draft-ietf-ips-fcip-slp-___.txt.
Anil Rijhsinghani contributed material related to the FCIP MIB and
edits draft-ietf-ips-fcip-mib-___.txt.
Bob Snively contributed material related to error detection and
recovery including the bulk of the synchronization recovery example
annex.
Lawrence J. Lamers contributed numerous ideas focused on keeping
FCIP compatible with B_Port devices.
Rajagopal, et al. Standards Track [Page 3]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Milan Merhar contributed several of the FCIP conceptual
modifications necessary to support NATs.
Don Fraser contributed material related to link failure detection
and reporting.
Bill Krieg contributed a restructuring of the TCP Connection setup
sections that made them more linear with respect to time and more
readable.
Several T11 leaders supported this effort and advised the editors of
this specification regarding appropriate interfaces to T11
documents. These T11 leaders are: Jim Nelson (Framing and
Signaling), Neil Wanamaker (Framing and Signaling), Craig Carlson
(Generic Services), Ken Hirata (Switch Fabric), Murali Rajagopal
(Backbone), Steve Wilson (Switch Fabric), and Michael O'Donnell
(Security Protocols).
2. Purpose, Motivation and Objectives
Fibre Channel (FC) is a gigabit or multi-gigabit speed networking
technology primarily used to implement Storage Area Networks (SANs).
See section 3 for information about how Fibre Channel is
standardized and the relationship of this specification to Fibre
Channel standards.
This specification describes mechanisms that allow the
interconnection of islands of Fibre Channel SANs over IP Networks to
form a unified SAN in a single Fibre Channel fabric. The motivation
behind defining these interconnection mechanisms is a desire to
connect physically remote FC sites allowing remote disk access, tape
backup, and live mirroring.
Fibre Channel standards have chosen nominal distances between switch
elements that are less than the distances available in an IP
Network. Since Fibre Channel and IP Networking technologies are
compatible, it is logical to turn to IP Networking for extending the
allowable distances between Fibre Channel switch elements.
The fundamental assumption made in this specification is that the
Fibre Channel traffic is carried over the IP Network in such a
manner that the Fibre Channel Fabric and all Fibre Channel devices
on the Fabric are unaware of the presence of the IP Network. This
means that the FC datagrams must be delivered in such time as to
comply with existing Fibre Channel specifications. The FC traffic
may span LANs, MANs and WANs, so long as this fundamental assumption
is adhered to.
Rajagopal, et al. Standards Track [Page 4]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The objectives of this document are to:
1) specify the encapsulation and mapping of Fibre Channel (FC)
frames employing FC Frame Encapsulation [27].
2) apply the mechanism described in 1) to an FC Fabric using an IP
network as an interconnect for two or more islands in an FC
Fabric.
3) address any FC concerns arising from tunneling FC traffic over
an IP-based network, including security, data integrity (loss),
congestion, and performance. This will be accomplished by
utilizing the existing IETF-specified suite of protocols.
4) be compatible with the referenced FC standards. While new work
may be undertaken in T11 [7] to optimize and enhance FC Fabrics,
this specification REQUIRES conformance only to the referenced
FC standards.
5) be compatible with all applicable IETF standards so that the IP
Network used to extend an FC Fabric can be used concurrently for
other reasonable purposes.
3. Relationship to Fibre Channel Standards
3.1 Relevant Fibre Channel Standards
FC is standardized under American National Standard for Information
Systems of the National Committee for Information Technology
Standards (ANSI-NCITS) in its T11 technical committee. T11 has
specified a number of documents describing FC protocols, operations,
and services. T11 documents of interest to readers of this
specification include (but are not limited to):
- FC-BB - Fibre Channel Backbone [3]
- FC-BB-2 - Fibre Channel Backbone -2 [4]
- FC-SW-2 - Fibre Channel Switch Fabric -2 [5]
- FC-FS - Fibre Channel Framing and Signaling [6]
FC-BB and FC-BB-2 describe the relationship between an FC Fabric and
interconnect technologies not defined in by Fibre Channel standards
(e.g., ATM and SONET). FC-BB-2 is the natural Fibre Channel home for
describing relationships to TCP/IP and FCIP.
FC-SW-2 describes the switch components of an FC Fabric and FC-FS
describes the FC Frame format and basic control features of Fibre
Channel.
Rajagopal, et al. Standards Track [Page 5]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Additional information regarding T11 activities is available on the
committee's web site [7].
3.2 This Specification and Fibre Channel Standards
When considering the challenge of transporting FC Frames over an IP
Network, it is logical to divide the standardization effort between
TCP/IP requirements and Fibre Channel requirements. This
specification covers the TCP/IP requirements for transporting FC
Frames and the Fibre Channel documents described in section 3.1
cover the Fibre Channel requirements.
This specification addresses only the requirements necessary to
properly utilize an IP Network as a conduit for FC Frames. The
result is a specification for an FCIP Entity (see section 6.4).
A product that tunnels an FC Fabric through an IP Network MUST
combine the FCIP Entity with an FC Entity (see section 6.3) using an
implementation specific interface. The requirements placed on an FC
Entity by this specification to achieve proper delivery of FC Frames
are summarized in annex G. More information about FC Entities can be
found in the Fibre Channel standards and an example of an FC Entity
can be found in FC-BB-2 [4].
No attempt is being made to define a specific API between an FCIP
Entity and an FC Entity at this time because doing so risks
compromising the performance and efficacy of the resulting products.
Current experience in this area is simply insufficient to guide
definition of the interface appropriately.
The objectives and motivations of this specification are not
impacted by the decision not to standardize a specific API between
FCIP Entities and FC Entities because fully functional and compliant
products can be built provided they contain both an FCIP Entity and
an FC Entity. The only products that cannot be built are those that
contain only one or the other.
4. Terminology
Terms needed to clarify the concepts presented in FCIP are defined
here.
FC End Node - A FC device that uses the connection services provided
by the FC Fabric.
FC Entity - The Fibre Channel specific element that combines with an
FCIP Entity to form an interface between an FC Fabric and an IP
Network (see section 6.3).
Rajagopal, et al. Standards Track [Page 6]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
FC Fabric - An entity that interconnects various Nx_Ports (see [6])
attached to it, and is capable of routing FC Frames using only the
destination ID information in a FC Frame header (see annex E).
FC Frame - The basic unit of Fibre Channel data transfer (see annex
E).
FC Receiver Portal - The access point through which an FC Frame and
time stamp enters an FCIP Data Engine from the FC Entity.
FC Transmitter Portal - The access point through which a
reconstituted FC Frame and time stamp leaves an FCIP Data Engine to
the FC Entity.
FCIP Data Engine (FCIP_DE) - The component of an FCIP Entity that
handles FC Frame encapsulation, de-encapsulation, and transmission
FCIP Frames through a single TCP Connection (see section 6.6).
FCIP Entity - The principal FCIP interface point to the IP Network
(see section 6.4).
FCIP Frame - An FC Frame plus the FC Frame Encapsulation [27] header
and encoded EOF that contains the FC Frame (see section 6.6.1).
FCIP Link - One or more TCP Connections that connect one FCIP_LEP to
another (see section 6.2).
FCIP Link Endpoint (FCIP_LEP) - The component of an FCIP Entity that
contains one or more FCIP_DEs (see section 6.5).
Encapsulated Frame Receiver Portal - The TCP access point through
which an FCIP Frame is received from the IP Network by an FCIP Data
Engine.
Encapsulated Frame Transmitter Portal - The TCP access point through
which an FCIP Frame is transmitted to the IP Network by an FCIP Data
Engine.
Special Frame (SF) - A specially formatted FCIP frame containing
information used by the FCIP protocol (see section 8).
5. Protocol Summary
The FCIP protocol is summarized as follows:
1) The primary function of an FCIP Entity is forwarding FC Frames,
employing FC Frame Encapsulation described in [27].
Rajagopal, et al. Standards Track [Page 7]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
2) Viewed from the IP Network perspective, FCIP Entities are peers
and communicate using TCP/IP. Each FCIP Entity is a TCP endpoint
in the IP-based network.
3) Viewed from the FC Fabric perspective, pairs of FCIP Entities,
in combination with their associated FC Entities, serve as an FC
Frame transmission component of the FC Fabric. The FC End Nodes
are unaware of the existence of the FCIP Link.
4) FC Primitive Signals, Primitive Sequences, and Class 1 FC Frames
are not transmitted across an FCIP Link because they cannot be
encoded using FC Frame Encapsulation [27].
5) The path (route) taken by an encapsulated FC Frame follows the
normal routing procedures of the IP Network.
6) An FCIP Entity MAY contain multiple FCIP Link Endpoints, but
each FCIP Link Endpoint (FCIP_LEP) communicates with exactly one
other FCIP_LEP.
7) When multiple FCIP_LEPs with multiple FCIP_DEs are in use,
selection of which FCIP_DE to use for encapsulating and
transmitting a given FC Frame is outside the scope of this
document. FCIP Entities do not actively participate in FC Frame
routing.
8) The FCIP Control & Services function MAY use TCP/IP quality of
service features (see section 11.2) to support Fibre Channel
capabilities.
9) Each FCIP Entity is statically or dynamically configured with a
list of IP addresses and TCP port numbers corresponding to
participating FCIP Entities. If dynamic discovery of
participating FCIP Entities is supported, the function SHALL be
performed using the Service Location Protocol (SLPv2) [25]. It
is outside the scope of this specification to describe any
static configuration method for participating FCIP Entity
discovery. Refer to section 9.1.2.2 for a detailed description
of dynamic discovery of participating FCIP Entities using SLPv2.
10) Before creating a TCP Connection to a peer FCIP Entity, the FCIP
Entity attempting to create the TCP connection SHALL statically
or dynamically determine the IP address, TCP port, expected FC
Fabric Entity World Wide Name, TCP Connection Parameters, and
Quality of Service Information.
Rajagopal, et al. Standards Track [Page 8]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
11) FCIP Entities do not actively participate in the discovery of FC
source and destination identifiers. Discovery of FC addresses
(accessible via the FCIP Entity) is provided by techniques and
protocols within the FC architecture as described in FC-FS [6]
and FC-SW-2 [5].
12) To support IP Network security (see section 10), FCIP Entities
MUST:
1) implement cryptographically protected authentication and
cryptographic data integrity keyed to the authentication
process, and
2) implement data confidentiality security features.
13) On a given TCP Connection, this specification relies on TCP/IP
to deliver a byte stream in the same order that it was sent.
14) This specification defines only limited error detection and
recovery mechanisms and relies on both TCP and FC to handle data
loss and corruption within the IP Network.
6. The FCIP Model
6.1 FCIP Protocol Model
The relationship between FCIP and other protocols is illustrated in
figure 1.
+------------------------+ FCIP Link +------------------------+
| FCIP |===========| FCIP |
+--------+------+--------+ +--------+------+--------+
| FC-2 | | TCP | | TCP | | FC-2 |
+--------+ +--------+ +--------+ +--------+
| FC-1 | | IP | | IP | | FC-1 |
+--------+ +--------+ +--------+ +--------+
| FC-0 | | LINK | | LINK | | FC-0 |
+--------+ +--------+ +--------+ +--------+
| | PHY | | PHY | |
| +--------+ +--------+ |
| | | |
| | IP Network | |
V +--------------------+ V
to Fibre to Fibre
Channel Channel
Environment Environment
Fig. 1 FCIP Protocol Stack Model
Rajagopal, et al. Standards Track [Page 9]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Note that the objective of the FCIP Protocol is creation and
maintenance of one or more FCIP Links to transport data.
6.2 FCIP Link
The FCIP Link is the basic unit of service provided by the FCIP
Protocol to an FC Fabric. As shown in figure 2, an FCIP Link
connects two portions of an FC Fabric using an IP Network as a
transport to form a single FC Fabric.
/\/\/\/\/\/\ /\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / \ IP / \ FC /
/ Fabric \=========/ Network \=========/ Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/ \/\/\/\/\/\/
| |
|<--------- FCIP Link -------->|
Fig. 2 FCIP Link Model
At the points where the ends of the FCIP Link meet portions of the
FC Fabric, an FCIP Entity (see section 6.4) combines with an FC
Entity as described in section 6.3 to serve as the interface between
FC and IP.
An FCIP Link SHALL contain at least one TCP Connection and MAY
contain more than one TCP Connection. The endpoints of a single TCP
Connection are FCIP Data Engines (see section 6.6). The endpoints of
a single FCIP Link are FCIP Link Endpoints (see section 6.5).
Rajagopal, et al. Standards Track [Page 10]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
6.3 FC Entity
A product that tunnels an FC Fabric through an IP Network MUST
combine an FC Entity with an FCIP Entity (see section 6.4) to form a
complete interface between the FC Fabric and IP Network as shown in
figure 3.
|<--------- FCIP Link -------->|
| |
+----------+ /\/\/\/\/\/\ +----------+
| FCIP | \ IP / | FCIP |
| Entity |=========/ Network \=========| Entity |
+----------+ \/\/\/\/\/\/ +----------+
| FC | | FC |
| Entity | | Entity |
+----------+ +----------+
| |
/\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / \ FC /
/ Fabric \ / Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/
Fig. 3 FC Entity and FCIP Entity Model
In general, the combination of an FCIP Link and FC and FCIP Entities
is intended to replace a Fibre Channel defined connection between
Fibre Channel components. For example, this combination can be used
to replace a hard-wire connection between two Fibre Channel
switches. There are limitations on the generally intended usage of
the combination shown in figure 3. As another example, the
combination cannot be used to replace cable connections in a Fibre
Channel Arbitrated Loop because loop primitive signals cannot be
encapsulated for transmission over TCP.
The interface between the FC and FCIP Entities is implementation
specific. The minimum requirements placed on an FC Entity by this
specification are listed in annex G. More information about FC
Entities can be found in the Fibre Channel standards and an example
of an FC Entity can be found in FC-BB-2 [4].
Rajagopal, et al. Standards Track [Page 11]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
6.4 FCIP Entity
The model for an FCIP Entity is shown in figure 4.
.......................................................
: FCIP Entity :
: :
: +-----------+ :
: | FCIP | :
: | Control & |------------------------------------+ :
: | Services | | :
: | Module | | :
: +-----------+ | :
: | +--------------------+ | :
: | +-------+--------------------+|----+ | :
: | |+-----+--------------------+|----+| | :
: | ||+----| FCIP Link Endpoint |----+|| | :
: | ||| +--------------------+ ||| | :
:.............................................|||.....:
| ||| ||| |
| ||| ||| o<--+
| ||| unique TCP ||| | |
| ||| connections-->||| | |
| ||| ||| | |
+----------+ /\/\/\/\/\/\ |
| FC | \ IP / |
| Entity | / Network \ |
+----------+ \/\/\/\/\/\/ |
| |
/\/\/\/\/\/\ +------------------+
\ FC / +->TCP connect request
/ Fabric \ IP Address and port
\/\/\/\/\/\/
Fig. 4 FCIP Entity Model
The FCIP Entity is the connection interface point for the IP Network
and is the owner of the IP Address and Well Known Port used to form
TCP Connections. An FC Fabric to IP Network interface product SHALL
provide each FC Entity FCIP Entity pair contained in the product
with a unique combination of FC Fabric Entity World Wide Identifier
and FC/FCIP Entity Identifier values (see section 8).
An FCIP Entity contains an FCIP Control & Services Module to control
FCIP link initialization, FCIP link dissolution, and to provide the
FC Entity with an interface to key IP Network features. The
interfaces to the IP Network features is implementation specific,
Rajagopal, et al. Standards Track [Page 12]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
however, to maintain interoperability, the notable TCP/IP mechanisms
used are specified in this document as follows:
- TCP Connections - see section 9
- Security - see section 10
- Performance - see section 11
- Dynamic Discovery - see section 9.1.2.2
The FCIP Link Endpoints in an FCIP Entity provide the FC Frame
encapsulation and transmission features of FCIP.
6.5 FCIP Link Endpoint (FCIP_LEP)
As shown in figure 5, the FCIP Link Endpoint contains one FCIP Data
Engine for each TCP Connection in the FCIP Link.
................................................
: FCIP Link Endpoint :
: +------------------+ :
: +-------+------------------+|----+ :
: |+-----+------------------+|----+| :
: ||+----| FCIP Data Engine |----+|| :
: ||| +------------------+ ||| :
:..............................................:
||| |||
+----------+ /\/\/\/\/\/\
| FC | \ IP /
| Entity | / Network \
+----------+ \/\/\/\/\/\/
|
/\/\/\/\/\/\
\ FC /
/ Fabric \
\/\/\/\/\/\/
Fig. 5 FCIP Link Endpoint Model
Each time a TCP Connection is formed with a new FCIP Entity FC
Entity pair (including all the actions described in section 9.1),
the FCIP Entity SHALL create a new FCIP Link Endpoint containing one
FCIP Data Engine.
An FCIP_LEP is a transparent data translation point between an FC
Entity and an IP Network. A pair of FCIP_LEPs communicating over one
or more TCP Connections create an FCIP Link to join two islands of a
FC Fabric, producing a single FC Fabric.
Rajagopal, et al. Standards Track [Page 13]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The IP Network over which the two FCIP_LEPs communicate is not aware
of the FC payloads that it is carrying. Likewise, the FC End Nodes
connected to the FC Fabric are unaware of the TCP/IP based transport
employed in the structure of the FC Fabric.
An FCIP_LEP uses normal TCP based flow control mechanisms for
managing its internal resources and matching them with the
advertised TCP Receiver Window Size (see section 9.5). An FCIP_LEP
MAY communicate with its FC Entity counterpart to coordinate flow
control.
6.6 FCIP Data Engine (FCIP_DE)
The model for one of the multiple FCIP_DEs that MAY be present in an
FCIP_LEP is shown in figure 6.
+--------------------------------+
| |
F |-+ +------------------+ +-|
C |p| | Encapsulation | |p| N
-->|1|--->| Engine |--->|2|--> e
E |-+ +------------------+ +-| t
n | | I w
t |-+ +------------------+ +-| P o
i |p| | De-Encapsulation | |p| r
t <--|4|<---| Engine |<---|3|<-- k
y |-+ +------------------+ +-|
| |
+--------------------------------+
Fig. 6 FCIP Data Engine Model
Data enters and leaves the FCIP_DE through four portals (p1 - p4).
The portals do not process or examine the data that passes through
them. They are only the named access points where the FCIP_DE
interfaces with external world. The names of the portals are as
follows:
p1) FC Receiver Portal - The interface through which an FC Frame and
time stamp enters an FCIP_DE from the FC Entity.
p2) Encapsulated Frame Transmitter Portal - The TCP interface
through which an FCIP Frame is transmitted to the IP Network by
an FCIP_DE.
p3) Encapsulated Frame Receiver Portal - The TCP interface through
which an FCIP Frame is received from the IP Network by an FCIP_DE.
Rajagopal, et al. Standards Track [Page 14]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
p4) FC Transmitter Portal - The interface through which a
reconstituted FC Frame and time stamp exits an FCIP_DE to the FC
Entity.
The work of the FCIP_DE is done by the Encapsulation and De-
Encapsulation Engines. The Engines have two functions:
1) Encapsulating and de-encapsulating FC Frames using the
encapsulation format described in FC Frame Encapsulation [27]
and in section 6.6.1 of this document, and
2) Detecting some data transmission errors and performing minimal
error recovery as described in section 6.6.2.
Data flows through the FCIP_DE in the following seven steps:
1) An FC Frame and time stamp arrives at the FC Receiver Portal and
is passed to the Encapsulation Engine. The FC Frame is assumed
to have been processed by the FC Entity according to the
applicable FC rules and is not validated by the FCIP_DE. If the
FC Entity is in the Unsynchronized state with respect to a time
base as described in the FC Frame Encapsulation [27]
specification, the time stamp delivered with the FC Frame SHALL
be zero.
2) In the Encapsulation Engine, the encapsulation format described
in FC Frame Encapsulation [27] and in section 6.6.1 of this
document SHALL be applied to prepare the FC Frame and associated
time stamp for transmission over the IP Network.
3) The entire encapsulated FC Frame (a.k.a. the FCIP Frame) SHALL
be passed to the Encapsulated Frame Transmitter Portal where it
SHALL be inserted in the TCP byte stream.
4) Transmission of the FCIP Frame over the IP Network follows all
the TCP rules of operation. This includes but is not limited to
the in-order delivery of bytes in the stream, as specified by
TCP [8].
5) The FCIP Frame arrives at the partner FCIP Entity where it
enters the FCIP_DE through the Encapsulated Frame Receiver
Portal and is passed to the De-Encapsulation Engine for
processing.
Rajagopal, et al. Standards Track [Page 15]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
6) The De-Encapsulation Engine SHALL validate the incoming TCP byte
stream as described in section 6.6.2 and SHALL de-encapsulate
the FC Frame and associated time stamp according to the
encapsulation format described in FC Frame Encapsulation [27]
and in section 6.6.1 of this document.
7) In the absence of errors, the de-encapsulated FC Frame and time
stamp SHALL be passed to the FC Transmitter Portal for delivery
to the FC Entity.
Every FC Frame that arrives at the FC Receiver Portal SHALL be
transmitted on the IP Network as described in steps 1 through 4
above. In the absence of errors, data bytes arriving at the
Encapsulated Frame Receiver Portal SHALL be de-encapsulated and
forwarded to the FC Transmitter Portal as described in steps 5
through 7.
6.6.1 FCIP Encapsulation of FC Frames
The FCIP encapsulation of FC Frames employs FC Frame Encapsulation
[27].
The features from FC Frame Encapsulation that are unique to
individual protocols SHALL be applied as follows for the FCIP
encapsulation of FC Frames.
The Protocol# field SHALL contain 1 in accordance with the IANA
Considerations annex of FC Frame Encapsulation [27].
The Protocol Specific field SHALL have the format shown in figure 7.
Note: the word numbers in figure 7 are relative to the complete FC
Frame Encapsulation header, not to the Protocol Specific field.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
+---------------------------------------------------------------+
1| replication of encapsulation word 0 |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
+---------------+---------------+---------------+---------------+
Fig. 7 FCIP Usage of FC Frame Encapsulation Protocol Specific
field
Word 1 of the Protocol Specific field SHALL contain an exact copy of
word 0 in FC Frame Encapsulation [27].
Rajagopal, et al. Standards Track [Page 16]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The pFlags (protocol specific flags) field provides information
about the protocol specific usage of the FC Encapsulation Header.
Figure 8 shows the defined pFlags bits.
|----------------Bit--------------------|
| |
| 31 30 29 28 27 26 25 24 |
+----+-----------------------------+----+
| Ch | Reserved | SF |
+----+-----------------------------+----+
Fig. 8 pFlags Field Bits
The SF (Special Frame) bit indicates whether the FCIP Frame is an
encapsulated FC Frame or an FCIP Special Frame (see section 8). When
the FCIP Frame contains an encapsulated FC Frame the SF bit SHALL be
0. When the FCIP Frame is an FCIP Special Frame the SF bit SHALL be 1.
The FCIP Special Frame SHALL only be sent as the first bytes
transmitted in each direction on a newly formed TCP Connection and
only one FCIP Special Frame SHALL be transmitted in each direction
at that time (see section 9.1). After that all FCIP Frames SHALL
have the SF bit set to 0.
The Ch (Changed) bit indicates whether an echoed Special Frame has
been intentionally altered (see section 9.1.3). The Ch bit SHALL be
0 unless the Special Frame bit is 1. When the initial TCP Connection
Special Frame is sent, the Ch bit SHALL be 0. If the recipient of a
TCP connect request echoes the Special Frame without any changes,
then the Ch bit SHALL continue to be 0. If the recipient of a TCP
connect request alters the Special Frame before echoing it, then the
Ch bit SHALL be changed to 1.
Rajagopal, et al. Standards Track [Page 17]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Table 1 summarizes the usage of the pFlags SF and Ch bits.
+----+----+------------+--------------------------------------+
| | | Originated | |
| SF | Ch | or Echoed | Validitity/Description |
+----+----+------------+--------------------------------------+
| 0 | 0 | n/a | Encapsulated FC Frame |
+----+----+------------+--------------------------------------+
| 0 | 1 | n/a | Always Illegal |
+----+----+------------+--------------------------------------+
| 1 | 0 | Originated | Originated Special Frame |
+----+----+------------+--------------------------------------+
| 1 | 1 | Originated | Always Illegal |
+----+----+------------+--------------------------------------+
| 1 | 0 | Echoed | Echoed Special Frame without changes |
+----+----+------------+--------------------------------------+
| 1 | 1 | Echoed | Echoed Special Frame with changes |
+----+----+------------+--------------------------------------+
| Note 1: Echoed Special Frames may contain changes resulting |
| from transmission errors, necessitating the comparison |
| between sent and recieved Special Frame bytes by the |
| Special Frame originator described in section 9.1.2.3. |
| |
| Note 2: Column positions in this table do not reflect the |
| bit positions of the SF and Ch bits in the pFlags field. |
+-------------------------------------------------------------+
Table 1 pFlags SF and Ch bit usage summary
The Reserved pFlags bits SHALL be 0.
The Reserved field (bits 23-16 in word 2): SHALL contain 0.
The -Reserved field (bits 7-0 in word 2): SHALL contain 255 (or 0xFF).
The CRCV (CRC Valid) Flag SHALL be set to 0.
The CRC field SHALL be set to 0.
Rajagopal, et al. Standards Track [Page 18]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Table 2 shows the SOF and EOF code values that are legal in FCIP
Frames. This list may be a subset of the SOF and EOF codes listed in
the FC Frame Encapsulation [27].
+-------+----------+ +-------+----------+
| FC | | | FC | |
| SOF | SOF Code | | SOF | SOF Code |
+-------+----------+ +-------+----------+
| SOFf | 0x28 | | SOFi4 | 0x29 |
| SOFi2 | 0x2D | | SOFn4 | 0x31 |
| SOFn2 | 0x35 | | SOFc4 | 0x39 |
| SOFi3 | 0x2E | +-------+----------+
| SOFn3 | 0x36 |
+-------+----------+
+-------+----------+ +--------+----------+
| FC | | | FC | |
| EOF | EOF Code | | EOF | EOF Code |
+-------+----------+ +--------+----------+
| EOFn | 0x41 | | EOFdt | 0x46 |
| EOFt | 0x42 | | EOFdti | 0x4E |
| EOFni | 0x49 | | EOFrt | 0x44 |
| EOFa | 0x50 | | EOFrti | 0x4F |
+-------+----------+ +--------+----------+
Table 2 Valid FCIP SOF and EOF codes
6.6.2 FCIP Data Engine Error Detection and Recover
6.6.2.1 TCP Assistance With Error Detection and Recovery
TCP [8] requires in order delivery, generation of TCP checksums, and
checking of TCP checksums. Thus, the byte stream passed from TCP to
the FCIP_LEP will be in order and free of errors detectable by the
TCP checksum. If TCP did not perform these functions, the FCIP_LEP
would have to.
6.6.2.2 Errors in FCIP Headers and Discarding FCIP Frames
Bytes delivered through the Encapsulated Frame Receiver Portal that
are not correctly delimited as defined by the FC Frame Encapsulation
[27] are considered to be in error.
Further, some errors in the encapsulation will result in the FCIP_DE
losing synchronization with the FCIP frames in the byte stream
entering through the Encapsulated Frame Receiver Portal.
Rajagopal, et al. Standards Track [Page 19]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The Frame Length field in the FC Frame Encapsulation header is used
to determine where in the data stream the next FC Encapsulated
Header is located. The following tests SHALL be performed to verify
synchronization with the byte stream entering the Encapsulated Frame
Receiver Portal, and synchronization SHALL be considered lost if any
of the tests fail:
1) Length field validation -- 15 < Length < 545;
2) Comparison of Length field to its ones complement; and
3) A valid EOF is found in the word preceding the start of the next
FCIP header as indicated by the Frame Length field, to be tested
as follows:
1) Bits 24-31 and 16-23 contain identical legal EOF values (the
list of legal EOF values is in the FC Frame Encapsulation
[27]); and
2) Bits 8-15 and 0-7 contain the ones complement of the EOF
value found in bits 24-31.
If synchronization is lost, the frame SHALL NOT be forwarded on to
the FC Entity and further recovery SHALL be handled as defined by
section 6.6.2.3.
In addition to the tests above, the validity and positioning of the
following FCIP Frame information SHOULD be used to detect
encapsulation errors that may or may not affect synchronization:
a) Protocol # field and its ones complement (2 tests);
b) Version field and its ones complement (2 tests);
c) Replication of encapsulation word 0 in word 1 (1 test);
d) Reserved field and its ones complement (2 tests);
e) Flags field and its ones complement (2 tests);
f) CRC field is equal to zero (1 test);
g) SOF fields and ones complement fields (4 tests);
h) Format and values of FC header (1 test);
i) CRC of FC Frame (2 tests);
j) FC Frame Encapsulation header information in the next FCIP Frame
(1 test).
At least 5 of the 18 tests listed above SHALL be performed. Failure
of any of the above tests actually performed SHALL indicate an
encapsulation error and the frame SHALL NOT be forwarded on to the
FC Entity. Further, such errors SHOULD be considered carefully,
since some may be synchronization errors.
Whenever an FCIP_DE discards bytes delivered through the
Encapsulated Frame Receiver Portal, it SHALL cause the FCIP Entity
to notify the FC Entity of the condition and provide a suitable
description of the reason bytes were discarded.
Rajagopal, et al. Standards Track [Page 20]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The burden for recovering from discarded data falls on the FC Entity
and other components of the FC Fabric and is outside the scope of
this specification.
6.6.2.3 Synchronization Failures
If an FCIP_DE determines that it cannot find the next FCIP Frame
header in the byte stream entering through the Encapsulated Frame
Receiver Portal, the FCIP_DE SHALL either:
a) close the TCP Connection [8] [9] and notify the FC Entity with
the reason for the closure;
b) recover synchronization by searching the bytes delivered by the
Encapsulated Frame Receiver Portal for a valid FCIP Frame header
having the correct properties, and discarding bytes delivered by
the Encapsulated Frame Receiver Portal until a valid FCIP Frame
header is found; or
c) attempt to recover synchronization as described in b) and if
synchronization cannot be recovered close the TCP Connection as
described in a) including notification of the FC Entity with the
reason for the closure.
If the FCIP_DE attempts to recover synchronization, the
resynchronization algorithm used SHALL meet the following
requirements:
a) discard or identify with an EOFa (see annex section E.1) those
FC Frames and fragments of FC Frames identified before
synchronization has again been completely verified. The number
of FC Frames not forwarded may vary based on the algorithm used;
b) return to sending valid FC Frames only after synchronization has
been verified; and
c) close the TCP/IP connection if the algorithm ends without
verifying successful synchronization. The probability of failing
to synchronize successfully and the time necessary to determine
whether or not synchronization was successful may vary with the
algorithm used.
An example algorithm meeting these requirements can be found in
annex C.
The burden for recovering from the discarding of FCIP Frames during
the optional resynchronization process described in this section
falls on the FC Entity and other components of the FC Fabric and is
outside the scope of this specification.
Rajagopal, et al. Standards Track [Page 21]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
7. Checking FC Frame Transit Times in the IP Network
The FC Entity MUST implement the measurement of Fibre Channel frame
IP Network transit time as described in the FC Frame Encapsulation
[27] specification. The choice to place this implementation
requirement in the FC Entity is based on a desire to include the
transit time through the FCIP Entities when computing the IP Network
transit time experienced by the FC Frames.
Each FC Frame that enters the FCIP_DE through the FC Receiver Portal
SHALL be accompanied by a time stamp value that the FCIP_DE SHALL
place in the Time Stamp [integer] and Time Stamp [fraction] fields
of the encapsulation header of the FCIP Frame that contains the FC
Frame. If no synchronized time stamp value is available to accompany
the entering FC Frame a value of zero SHALL be supplied.
Each FC Frame that exits the FCIP_DE through the FC Transmitter
Portal SHALL be accompanied by the time stamp value taken from the
FCIP Frame that encapsulated the FC Frame.
The FC Entity SHALL use suitable internal clocks and either Fibre
Channel services or an SNTP Version 4 server [13] to establish and
maintain the required synchronized time value. The FC Entity SHALL
verify that the FC Entity it is communicating with on an FCIP Link
is using the same synchronized time source as it is, either Fibre
Channel services or SNTP server.
Note that since the FC Fabric is expected to have a single
synchronized time value throughout, reliance on the Fibre Channel
services means that only one synchronized time value is needed for
all FCIP_DEs regardless of their connection characteristics.
Rajagopal, et al. Standards Track [Page 22]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
8. The FCIP Special Frame
Figure 9 shows the FCIP Special Frame format.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
+---------------+---------------+---------------+---------------+
0| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
1| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
| | (0x00) | | (0xFF) |
+-----------+---+---------------+-----------+---+---------------+
3| Flags | Frame Length | -Flags | -Frame Length |
| (0b000000)| (0b0000010010) | (0b111111)| (0b1111101101) |
+-----------+-------------------+-----------+-------------------+
4| Time Stamp [integer] |
+---------------------------------------------------------------+
5| Time Stamp [fraction] |
+---------------------------------------------------------------+
6| CRC (Reserved in FCIP) |
| (0x00-00-00-00) |
+-------------------------------+-------------------------------+
7| Reserved | -Reserved |
| (0x00-00) | (0xFF-FF) |
+-------------------------------+-------------------------------+
8| |
+----- Source FC Fabric Entity World Wide Name -----+
9| |
+---------------------------------------------------------------+
10| |
+----- Source FC/FCIP Entity Identifier -----+
11| |
+---------------------------------------------------------------+
12| |
+----- Connection Nonce -----+
13| |
+---------------------------------------------------------------+
(Continued)
Fig. 9 FCIP Special Frame Format (part 1 of 2)
Rajagopal, et al. Standards Track [Page 23]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
| |
| (Concluded) |
+---------------+---------------+-------------------------------+
14| Connection | Reserved | Connection Usage Code |
| Usage Flags | (0x00) | <defined in FC-BB-2> |
+---------------+---------------+-------------------------------+
15| |
+----- Destination FC Fabric Entity World Wide Name -----+
16| |
+-------------------------------+-------------------------------+
17| Reserved | -Reserved |
| (0x00-00) | (0xFF-FF) |
+-------------------------------+-------------------------------+
Fig. 9 FCIP Special Frame Format (part 2 of 2)
The FCIP Special Frame SHALL only be sent as the first bytes
transmitted in each direction on a newly formed TCP Connection and
only one FCIP Special Frame SHALL be transmitted in each direction.
The contents of the FCIP Special Frame SHALL be as described for
encapsulated FC Frames, except for the fields described in this
section.
All FCIP Special Frames SHALL have the pFlags SF bit set to 1 (see
section 6.6.1).
The Source FC Fabric Entity World Wide Name field SHALL contain the
Fibre Channel Name_Identifier [6] for the FC Fabric entity
associated with the FC Entity FCIP Entity pair that generates (as
opposed to echoes) the Special Frame. For example, if the FC Fabric
entity is a FC Switch, the FC Fabric Entity World Wide Name field
SHALL contain the Switch_Name [5]. The Source FC Fabric Entity World
Wide Name SHALL be world wide unique.
The Source FC/FCIP Entity Identifier field SHALL contain a unique
identifier for the FC Entity FCIP Entity pair that generates (as
opposed to echoes) the Special Frame. The value is assigned by the
FC Fabric entity whose world wide name appears in the Source FC
Fabric Entity World Wide Name field.
Note: The combination of the Source FC Entity World Wide Name and
Source FCIP Entity Identifier fields uniquely identifies every FC
Entity FCIP Entity pair in the IP Network.
Rajagopal, et al. Standards Track [Page 24]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The Connection Nonce field shall contain a 64-bit random number
generated to uniquely identify a single TCP connect request. In
order to provide sufficient security for the connection nonce, the
Randomness Recommendations for Security [12] SHOULD be followed.
The Connection Usage Flags field identifies the types of SOF values
[27] to be carried on the connection as shown in figure 10.
|------------------------------Bit------------------------------|
| |
| 31 30 29 28 27 26 25 24 |
+-------+-------+-------+-------+-------------------------------+
| SOFf | SOF?2 | SOF?3 | SOF?4 | Reserved |
+-------+-------+-------+-------+-------------------------------+
Fig. 10 Connection Usage Flags Field Format
If the SOFf bit is one, then FC Frames containing SOFf are intended
to be carried on the connection.
If the SOF?2 bit is one, then FC Frames containing SOFi2 and SOFn2
are intended to be carried on the connection.
If the SOF?3 bit is one, then FC Frames containing SOFi3 and SOFn3
are intended to be carried on the connection.
If the SOF?4 bit is one, then FC Frames containing SOFi4, SOFn4, and
SOFc4 are intended to be carried on the connection.
All or none of the SOFf, SOF?2, SOF?3, and SOF?4 bits MAY be set to
one. If all of the SOFf, SOF?2, SOF?3, and SOF?4 bits are zero, then
the types of FC Frames intended to be carried on the connection has
no specific relationship to SOF code.
The FCIP Entity SHALL NOT enforce the SOF usage described by the
Connection Usage Flags field and SHALL only use the contents of the
field as described below.
The Connection Usage Code field contains Fibre Channel defined
information regarding the intended usage of the connection as
specified in FC-BB-2 [4].
The FCIP Entity SHALL use the contents of the Connection Usage Flags
and Connection Usage Code fields to locate appropriate QoS settings
in the "shared" database of TCP Connection information (see section
9.1.1) and apply those settings to a newly formed connection.
Rajagopal, et al. Standards Track [Page 25]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The Destination FC Fabric Entity World Wide Name field MAY contain
the Fibre Channel Name_Identifier [6] for the FC Fabric entity
associated with the FC Entity FCIP Entity pair that echoes (as
opposed to generates) the Special Frame.
For each new incoming TCP connect request and subsequent Special
Frame received, the FCIP Entity SHALL send the contents of the
Source FC Fabric Entity World Wide Name, Source FC/FCIP Identifier,
Connection Usage Flags and Connection Usage Code fields to the FC
Entity along with the other connection information (e.g., FCIP_LEP
and FCIP_DE information).
9. TCP Connection Management
9.1 TCP Connection Establishment
9.1.1 Connection Establishment Model
The description of the connection establishment process in section
9.1 is a model for the interactions between an FC Entity and an FCIP
Entity during TCP Connection establishment. The model is written in
terms of a "shared" database that the FCIP Entity consults to
determine the properties of the TCP Connections to be formed
combined with routine calls to the FC Entity when connections are
successfully established. Whether the FC Entity contributes
information to the "shared" database is not critical to this model.
What is important is the fact that the FCIP Entity MAY consult the
database at anytime to determine its actions relative to TCP
Connection establishment.
It is important to remember that this description is only a model
for the interactions between an FC Entity and an FCIP Entity. Any
implementation that has the same effects on the FC Fabric and IP
Network as those described in the model meets the requirements of
this specification. For example, an implementation might replace the
"shared" database with a routine interface between the FC and FCIP
Entities.
Rajagopal, et al. Standards Track [Page 26]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
9.1.2 Creating New TCP Connections
9.1.2.1 Non-Dynamic Creation of New TCP Connections
When an FCIP Entity discovers that a new TCP Connection needs to be
established, it SHALL determine the IP Address to which the TCP
Connection is to be made and establish all enabled IP security
features for that IP Address as described in section 10. Then the
FCIP Entity SHALL determine the following information about the new
connection in addition to the IP Address:
- The expected Destination FC Fabric Entity World Wide Name of the
FC Entity FCIP Entity pair to which the TCP Connection is being
made
- TCP Connection Parameters (see section 9.3)
- Quality of Service Information (see section 11)
Based on this information, the FCIP Entity SHALL generate a TCP
connect request [8] to the FCIP Well-Known Port of 3225 (or other
configuration specific port number) at the specified IP Address. If
the TCP connect request is rejected, the FCIP Entity SHALL act to
limit unnecessary repetition of attempts to establish similar
connections. If the TCP connect request is accepted, the FCIP Entity
SHALL follow the steps described in section 9.1.2.3 to complete the
establishment of a new FCIP_DE.
It is recommended that an FCIP Entity not initiate TCP connect
requests to another FCIP Entity if incoming TCP connect requests
from that FCIP Entity have already been accepted.
9.1.2.2 Dynamic Creation of New TCP Connections
If dynamic discovery of participating FCIP Entities is supported the
function SHALL be performed using the Service Location Protocol
(SLPv2) [25] in the manner defined for FCIP usage [28].
Upon discovering that dynamic discovery is to be used, the FCIP
Entity SHALL enable IP security features for the SLP discovery
process as described in [28] and then:
1) Determine the one or more FCIP Discovery Domain(s) to be used in
the dynamic discovery process;
2) Establish an SLPv2 Service Agent to advertise the availability
of this FCIP Entity to peer FCIP Entities in the identified FCIP
Discovery Domain(s); and
Rajagopal, et al. Standards Track [Page 27]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
3) Establish an SLPv2 User Agent to locate service advertisements
for peer FCIP Entities in the identified FCIP Discovery Domain(s).
For each peer FCIP Entity dynamically discovered through the SLPv2
User Agent, the FCIP Entity SHALL establish all enabled IP security
features for the discovered IP Address as described in section 10
and then determine the following information about the new connection:
- The expected Destination FC Fabric Entity World Wide Name of the
FC Entity FCIP Entity pair to which the TCP Connection is being
made
- TCP Connection Parameters (see section 9.3)
- Quality of Service Information (see section 11)
Based on this information, the FCIP Entity SHALL generate a TCP
connect request [8] to the FCIP Well-Known Port of 3225 (or other
configuration specific port number) at the IP Address specified by
the service advertisement. If the TCP connect request is rejected,
act to limit unnecessary repetition of attempts to establish similar
connections. If the TCP connect request is accepted, the FCIP Entity
SHALL follow the steps described in section 9.1.2.3 to complete the
establishment of a new FCIP_DE.
It is recommended that an FCIP Entity not initiate TCP connect
requests to another FCIP Entity if incoming TCP connect requests
from that FCIP Entity have already been accepted.
9.1.2.3 Connection Setup After a Successful TCP Connect Request
Whether Non-Dynamic TCP Connection creation (see section 9.1.2.1) or
Dynamic TCP Connection creation (see section 9.1.2.2) is used, the
steps described in this section SHALL be followed to take the TCP
Connection setup process to completion.
After the TCP connect request has been accepted, the FCIP Entity
SHALL send an FCIP Special Frame (see section 8) as the first bytes
transmitted on the newly formed connection and retain a copy of
those bytes for later comparisons. All fields in the FCIP Special
Frame SHALL be filled in as described in section 8, particularly:
- The Source FC Fabric Entity World Wide Name field SHALL contain
the FC Fabric Entity World Wide Name for the FC Entity FCIP
Entity pair that is originating the TCP connect request;
- The Source FC/FCIP Entity Identifier field SHALL contain a
unique identifier that is assigned by the FC Fabric entity whose
world wide name appears in the Source FC Fabric Entity World
Wide Name field;
Rajagopal, et al. Standards Track [Page 28]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
- The Connection Nonce field SHALL contain a 64-bit random number
that differs in value from any recently used Connection Nonce
value. In order to provide sufficient security for the
connection nonce, the Randomness Recommendations for Security
[12] SHOULD be followed; and
- The Destination FC Fabric Entity World Wide Name field SHALL
contain 0 or the expected FC Fabric Entity World Wide Name for
the FC Entity FCIP Entity pair that is destination the TCP
connect request.
After the FCIP Special Frame bytes are sent on the newly formed
connection, the FCIP Entity SHALL wait for the FCIP Special Frame to
be echoed as the first bytes received on the newly formed connection.
The FCIP Entity MAY apply a timeout of not less than 90 seconds to
the waiting for the echoed FCIP Special Frame bytes and if the
timeout expires the FCIP Entity SHALL close the TCP Connection and
notify the FC Entity with the reason for the closure.
If the echoed FCIP Special Frame bytes do not exactly match the FCIP
Special Frame bytes sent (words 7 through 17 inclusive), the FCIP
Entity SHALL close the TCP Connection and notify the FC Entity with
the reason for the closure.
The remaining steps in this section SHALL be performed only if the
echoed FCIP Special Frame bytes exactly match the FCIP Special Frame
bytes sent (words 7 through 17 inclusive).
If the IP Address and TCP Port to which the TCP Connection was made
is not associated with any other FCIP_LEP, the FCIP Entity SHALL:
1) Instantiate the appropriate Quality of Service (see section 11)
conditions on the newly created TCP Connection,
2) Create a new FCIP_LEP for the new FCIP Link,
3) Create a new FCIP_DE within the newly created FCIP_LEP to
service the new TCP Connection, and
4) Inform the FC Entity of the new FCIP_LEP, FCIP_DE, Destination
FC Fabric Entity World Wide Name, Connection Usage Flags and
Connection Usage Code.
If an existing FCIP_LEP is associated with the IP Address and TCP
Port to which the TCP Connection was made, the FCIP Entity SHALL:
1) Instantiate the appropriate Quality of Service (see section 11)
conditions on the newly created TCP Connection,
Rajagopal, et al. Standards Track [Page 29]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
2) Create a new FCIP_DE within the existing FCIP_LEP to service the
new TCP Connection, and
3) Inform the FC Entity of the FCIP_LEP, Destination FC Fabric
Entity World Wide Name, Connection Usage Flags, Connection Usage
Code and new FCIP_DE.
9.1.3 Processing Incoming TCP Connect Requests
The FCIP Entity SHALL listen for new TCP Connection requests [8] on
the FCIP Well-Known Port (3225). An FCIP Entity MAY also accept and
establish TCP Connections to a TCP port number other than the FCIP
Well-Known Port, as configured by the network administrator.
The FCIP Entity SHALL determine the following information about the
requested connection:
- Whether the requested connection is allowed
- Whether IP security setup has been performed for the IP security
features enabled on the connection (see section 10)
If the requested connection is not allowed, the FCIP Entity SHALL
abort the TCP connect request [8]. If the requested connection is
allowed, the FC Entity SHALL ensure that required IP security
features are enabled and accept the TCP connect request.
After the TCP connect request has been accepted, the FCIP Entity
SHALL wait for the FCIP Special Frame sent by the originator of the
TCP connect request as the first bytes received on the accepted
connection.
The FCIP Entity MAY apply a timeout of not less than 90 seconds to
the waiting for the FCIP Special Frame bytes and if the timeout
expires the FCIP Entity SHALL close the TCP Connection and notify
the FC Entity with the reason for the closure.
Note: One method for attacking the security of the FCIP Link
formation process (detailed in section 10.1) depends on keeping a
TCP connect request open without sending an FCIP Special Frame.
Implementations should bear this in mind in the handling of TCP
connect requests where the FCIP Special Frame is not sent in a
timely manner.
Rajagopal, et al. Standards Track [Page 30]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Upon receipt of the FCIP Special Frame sent by the originator of the
TCP connect request, the FCIP Entity SHALL inspect the contents of
the following fields:
- Connection Nonce,
- Destination FC Fabric Entity World Wide Name,
- Connection Usage Flags, and
- Connection Usage Code.
If the Connection Nonce field contains a value identical to the most
recently received Connection Nonce from the same IP Address, the
FCIP Entity SHALL close the TCP Connection and notify the FC Entity
with the reason for the closure.
If an FCIP Entity receives a duplicate FCIP Short Frame during the
FCIP Link formation process, it SHALL close that TCP Connection and
notify the FC Entity with the reason for the closure.
If the Destination FC Fabric Entity World Wide Name contains 0, the
FCIP Entity SHALL take one of the following three actions:
1) Leave the Destination FC Fabric Entity World Wide Name field and
Ch bit both 0;
2) Change the Destination FC Fabric Entity World Wide Name field to
match FC Fabric Entity World Wide Name associated with the FCIP
Entity that received the TCP connect request and change the Ch
bit to 1; or
3) Close the TCP Connection without sending any response.
The choice between the above actions depends on the anticipated
usage of the FCIP Entity and is outside the scope of this
specification. The FCIP Entity may consult the "shared" database
when choosing between the above actions.
If:
a) The Destination FC Fabric Entity World Wide Name contains a non-
zero value that does not match the FC Fabric Entity World Wide
Name associated with the FCIP Entity that received the TCP
connect request, or
b) The contents of the Connection Usage Flags, and Connection Usage
Code fields is not acceptable to the FCIP Entity that received
the TCP connect request,
then the FCIP Entity SHALL take one of the following two actions:
1) Change the contents of the unacceptable fields to correct/
acceptable values and set the Ch bit to 1; or
2) Close the TCP Connection without sending any response.
Rajagopal, et al. Standards Track [Page 31]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
If the FCIP Entity makes any changes in the content of the FCIP
Special Frame, it SHALL also set the Ch bit to 1.
If any changes have been made in the received FCIP Special Frame
during the processing described above, the following steps SHALL be
performed:
1) The changed FCIP Special Frame SHALL be echoed to the originator
of the TCP connect request as the only bytes transmitted on the
accepted connection;
2) The TCP Connection SHALL be closed (the FC Entity need not be
notified of the TCP Connection closure in this case because it
is not indicative of an error); and
3) All of the additional processing described in this section SHALL
be skipped.
The remaining steps in this section SHALL be performed only if the
FCIP Entity has not changed the contents of the above mentioned
fields to correct/acceptable values.
If the Source FC Fabric Entity World Wide Name and Source FC/FCIP
Entity Identifier field values in the FCIP Special Frame do not
match the Source FC Fabric Entity World Wide Name and Source FC/FCIP
Entity Identifier associated with any other FCIP_LEP, the FCIP
Entity SHALL:
1) Echo the unchanged FCIP Special Frame to the originator of the
TCP connect request as the first bytes transmitted on the
accepted connection;
2) Instantiate the appropriate Quality of Service (see section 11)
conditions on the newly created TCP Connection, considering the
Connection Usage Flags and Connection Usage Code fields and
"shared" database information (see section 9.1.1) as appropriate,
3) Create a new FCIP_LEP for the new FCIP Link,
4) Create a new FCIP_DE within the newly created FCIP_LEP to
service the new TCP Connection, and
5) Inform the FC Entity of the new FCIP_LEP, FCIP_DE, Source FC
Fabric Entity World Wide Name, Source FC/FCIP Entity Identifier,
Connection Usage Flags and Connection Usage Code.
Rajagopal, et al. Standards Track [Page 32]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
If the Source FC Fabric Entity World Wide Name and Source FC/FCIP
Entity Identifier field values in the FCIP Special Frame match the
Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity
Identifier associated with an existing FCIP_LEP, the FCIP Entity
SHALL:
1) Request that the FC Entity authenticate the source of TCP
connect request, providing the following information to the FC
Entity for authentication purposes:
a) Source FC Fabric Entity World Wide Name,
b) Source FC/FCIP Entity Identifier, and
c) Connection Nonce.
The FCIP Entity SHALL wait indefinitely for the FC Entity to
authenticate source of the TCP connect request and SHALL not use
the new TCP Connection for any purpose until the FC Entity
completes the authentication. If the FC Entity indicates that
the TCP connect request cannot be properly authenticated, the
FCIP Entity SHALL close the TCP Connection and skip all of the
remaining steps in this section.
Warning: The authentication mechanism described here and in FC-
BB-2 [4] is not designed to thwart sophisticated security
threats. The IP security mechanisms described in section 10
should be enabled in environments where security threats are
suspected.
2) Echo the unchanged FCIP Special Frame to the originator of the
TCP connect request as the first bytes transmitted on the
accepted connection;
3) Instantiate the appropriate Quality of Service (see section 11)
conditions on the newly created TCP Connection, considering the
Connection Usage Flags and Connection Usage Code fields and
"shared" database information (see section 9.1.1) as appropriate,
4) Create a new FCIP_DE within the existing FCIP_LEP to service the
new TCP Connection, and
5) Inform the FC Entity of the FCIP_LEP, Source FC Fabric Entity
World Wide Name, Source FC/FCIP Entity Identifier, Connection
Usage Flags, Connection Usage Code and new FCIP_DE.
Note that the originator of TCP connect requests uses IP Address and
TCP Port to identify which TCP Connections belong to which FCIP_LEPs
while the recipient of TCP connect requests uses the Source FC
Fabric Entity World Wide Name, Source FC/FCIP Entity Identifier
fields from the FCIP Special Frame to identify which TCP Connection
belong to which FCIP_LEPs. For this reason, an FCIP Entity that both
Rajagopal, et al. Standards Track [Page 33]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
originates and receives TCP connect requests is unable to match the
FCIP_LEPs associated with originated TCP connect requests to the
FCIP_LEPs associated with received TCP connect requests.
9.2 Closing TCP Connections
The FCIP Entity SHALL provide a mechanism with acknowledgement by
which the FC Entity is able to cause the closing of an existing TCP
Connection at anytime. This allows the FC Entity to close TCP
Connections that are producing too many errors, etc.
9.3 TCP Connection Parameters
In order to provide efficient management of FCIP_LEP resources as
well as FCIP Link resources, consideration of certain TCP Connection
parameters is RECOMMENDED.
9.3.1 TCP Selective Acknowledgement Option
The Selective Acknowledgement option RFC 2883 [26] allows the
receiver to acknowledge multiple lost packets in a single ACK,
enabling faster recovery. An FCIP Entity MAY negotiate use of TCP
SACK and use it for faster recovery from lost packets and holes in
TCP sequence number space.
9.3.2 TCP Window Scale Option
This option allows TCP window sizes larger than 16-bit limits to be
advertised by the receiver. It is necessary to allow data in long
fat networks to fill the available pipe. This also implies buffering
on the TCP sender that matches the (bandwidth*delay) product of the
TCP Connection. An FCIP_LEP uses locally available mechanisms to set
a window size that matches the available local buffer resources and
the desired throughput.
9.3.3 Protection against sequence number wrap
It is RECOMMENDED that FCIP Entities implement protection against
sequence number wrap. It is quite possible that within a single
connection, TCP sequence numbers wrap within a timeout window.
9.3.4 TCP_NODELAY Option
FCIP Entities SHALL set the TCP_NODELAY option to one. This will
disable the Nagle Algorithm that is designed for usage in a telnet
environment.
Rajagopal, et al. Standards Track [Page 34]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
9.4 TCP Connection Considerations
In idle mode, a TCP Connection "keep alive" option of TCP is
normally used to keep a connection alive. However, this timeout is
fairly large and may prevent early detection of loss of
connectivity. In order to facilitate faster detection of loss of
connectivity, FC Entities SHOULD implement some form of Fibre
Channel connection failure detection (see FC-BB-2 [4]).
When an FCIP Entity discovers that TCP connectivity has been lost,
the FCIP Entity SHALL notify the FC Entity of the failure including
information about the reason for the failure.
9.5 Flow Control Mapping between TCP and FC
The FCIP Entity and FC Entity are connected to the IP Network and FC
Fabric, respectively, and they need to follow the flow control
mechanisms of both TCP and FC, which work independent of each other.
This section provides guidelines as to how the FCIP Entity can map
TCP flow control to status notifications to the FC Entity.
There are two scenarios when the flow control management becomes
crucial:
1) When there is line speed mismatch between the FC and IP
interfaces.
Even though it is RECOMMENDED that both the FC and IP interfaces
to the FC Entity and FCIP Entity, respectively, be of comparable
speeds, it is possible to carry FC traffic over an IP Network
that has a different line speed and bit error rate.
2) When the FC Fabric or IP Network encounters congestion.
Even when both the FC Fabric or IP network are of comparable
speeds, during the course of operation the FC Fabric or the IP
Network could encounter congestion due to transient conditions.
The FC Entity uses Fibre Channel mechanisms for flow control at the
FC Receiver Portal based on information supplied by the FCIP Entity
regarding flow constraints at the Encapsulated Frame Transmitter
Portal. The FCIP Entity uses TCP mechanisms for flow control at the
Encapsulated Frame Receiver Portal portal based on information
supplied by the FC Entity regarding flow constraints at the FC
Transmitter Portal.
Rajagopal, et al. Standards Track [Page 35]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Coordination of these flow control mechanisms one of which is credit
based and the other of which is window based depends on painstaking
design that is outside the scope of this specification.
10. Security
10.1 Threat Models
Using a general purpose, wide-area network such as an IP Network as
a substitute for physical cabling introduces some security problems
not normally encountered in Fibre Channel Fabrics. FC interconnect
cabling typically is protected physically from outside access.
Public IP Networks allow hostile parties to impact the security of
the transport infrastructure.
The general effect is that the security of the entire FC Fabric is
only as good as the security of the entire IP Network through which
it tunnels. The following broad classes of attacks are possible:
1) Unauthorized Fibre Channel elements can gain access to resources
through normal Fibre Channel Fabric and processes. Although this
is a valid threat, securing the Fibre Channel Fabrics is outside
the scope of this document. Securing the IP Network is the issue
considered in this specification.
2) Unauthorized agents can monitor and manipulate Fibre Channel
traffic flowing over physical media used by the IP Network and
under control of the agent.
3) TCP Connections may be hijacked and used to instantiate an
invalid FCIP Link between two peer FCIP Entities.
4) Valid and invalid FCIP Encapsulated frames may be injected on
the TCP Connections.
5) The payload of an FCIP Encapsulated frame may be altered or
transformed in such a way that it preserves the TCP Checksum
transform while altering content.
6) Unauthorized agents can masquerade as a valid FCIP Entities and
disturb proper operation of the Fibre Channel Fabric.
7) Denial of Service attacks can be mounted by injecting TCP
Connection requests and other resource exhaustion operations.
8) An attacker may exploit the FCIP Special Frame (SF)
authentication mechanism of the FCIP Link formation process (see
section 9.1.3). The attacker could observe the SF contents sent
Rajagopal, et al. Standards Track [Page 36]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
on an initial connection of an FCIP Link and use the observed
nonce, Source FC/FCIP Entity Identifier and other SF contents to
form an FCIP Link using attacker's own previously established
connection, while resetting/blocking the observed connection.
Although the use of timeout for reception of Special Frame
reduces the risk of this attack, such an attack is possible. See
section 10.3.1 to protect against this specific attack.
The existing IPSec Security Architecture and protocol suite [14]
offers protection from these threats. An FCIP Entity MUST implement
portions of the IPSec protocol suite as described in this section.
10.2 FC Fabric and IP Network Deployment Models
In the context of enabling a secure FCIP tunnel between FC SANs, the
following characteristics of the IP Network deployment are useful to
note.
1) The FCIP Entities share a peer-to-peer relationship. Therefore,
the administration of security policies applies to all FCIP
Entities in an equal manner. This varies from a true Client-
Server relationship, where there is an inherent difference in
how security policies are administered.
2) Policy administration as well as security deployment and
configuration are constrained to the set of FCIP Entities,
thereby posing less of a requirement on a scalable mechanism.
For example, the validation of credentials can be relaxed to the
point where deploying a set of pre-shared keys is a viable
technique.
3) TCP Connections and the IP Network are terminated at the FCIP
Entity. The granularity of security implementation is at the
level of the FCIP tunnel endpoint (or FCIP Entity), unlike other
applications where there is a user-level termination of TCP
Connections. User-level objects are not controllable by or
visible to FCIP Entities. All user-level security related to
FCIP is the responsibility of the Fibre Channel standards [7]
and outside the scope of this specification.
10.3 FCIP Security Components
FCIP Security compliant implementations MUST implement IPSec
Protocol Suite based cryptographic authentication and data integrity
[14], as well as confidentiality using algorithms and transforms as
described in this section. Also, FCIP implementations MUST meet the
secure key management requirements of IPSec protocol suite.
Rajagopal, et al. Standards Track [Page 37]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
10.3.1 IPSec ESP Authentication and Confidentiality
FCIP Entities MUST implement IPSec ESP [16] in Tunnel Mode for
providing Data Integrity and Confidentiality. FCIP Entities MAY
implement IPSec ESP in Transport Mode, if deployment considerations
require use of Transport Mode.
If Confidentiality is not enabled but Data Integrity is enabled, ESP
with NULL Encryption [19] MUST be used.
IPSec ESP for message authentication computes a cryptographic hash
over the payload that is protected. While IPSec ESP mandates
compliant implementations to support certain algorithms for deriving
this hash, FCIP implementations:
- MUST implement HMAC with SHA-1 [15]
- SHOULD implement AES in CBC MAC mode with XCBC extensions [30]
- DES in CBC mode SHOULD NOT be used due to inherent weaknesses
For ESP Confidentiality, FCIP Entities:
- MUST implement 3DES in CBC mode
- SHOULD implement AES in CTR mode [29]
- MUST implement NULL Encryption [19]
When AES is used, the key size SHALL be at least 128-bits and the
cipher block size SHALL be at least 128-bits.
10.3.2 Key Management
FCIP Entities MUST support IKE [18] for peer authentication,
negotiation of Security Associations (SA) and Key Management using
the IPSec DOI [17]. Manual keying for establishing SA is not
permitted since it does not provide the necessary elements for
rekeying (see section 10.3.3).
IKE Phase 1 establishes a secure, MAC-authenticated channel for
communications for use by IKE Phase 2. FCIP Entities MUST support
"Main Mode" operation in Phase 1 and MAY support "Aggressive Mode"
if identity protection is not required.
FCIP Entities negotiate parameters for SA during IKE Phase 2 only
using "Quick Mode". For FCIP Entities engaged in IKE "Quick Mode",
there is no requirement for PFS (Perfect Forward Secrecy). FCIP
Entities engaged in IKE "Quick Mode" are not required to transmit a
Key Exchange (KE) payload.
Rajagopal, et al. Standards Track [Page 38]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
For a given pair of FCIP Entities, the same IKE Phase 1 negotiation
can be used for all Phase 2 negotiations; i.e., all TCP Connections
that are bundled into the single FCIP Link can share the same Phase
1 results.
Repeated rekeying using "Quick Mode" on the same shared secret will
over time, reduce the cryptographic properties of that secret. To
overcome this, Phase 1 MAY be invoked periodically to create a new
set of IKE shared secrets and related security parameters.
IKE Phase 1 establishment requires key distribution, and FCIP
Entities:
- MUST support pre-shared IKE keys.
- MAY support certificate-based peer authentication using digital
signatures.
- Peer authentication using the public key encryption methods
outlined in sections 5.2 and 5.3 of [18] SHOULD NOT be used.
When pre-shared keys are used, IKE Aggressive Mode SHOULD be used
and Main Mode SHOULD NOT be used. When Digital Signatures are used,
either IKE Main Mode or IKE Aggressive Mode may be used. In all
cases, access to locally stored secret information (pre-shared key,
or private key for digital signing) MUST be suitably restricted,
since compromise of secret information nullifies the security
properties of IKE/IPSec protocols. Such mechanisms are outside the
scope of this document. Support for IKE Oakley Groups is not required.
For the purposes of establishing a secure FCIP Link, the two
participating FCIP Entities consult a Security Policy Database
(SPD). FCIP Entities may have more than one interface and IP
Address, and it is possible for an FCIP Link to contain multiple TCP
connections whose FCIP endpoint IP Addresses are different. In this
case, an IKE Phase 1 SA is established for each FCIP endpoint IP
Address pair. For the purposes of establishing IKE Phase 1 SA,
static IP Addresses are typically used for identification.
At the end of successful IKE negotiations both FCIP Entities store
the SA parameters in their SA database (SAD). The SAD contains the
set of active SA entries, each entry containing Sequence Counter
Overflow, Sequence Number Counter, Anti-replay Window and the
Lifetime of the SA. FCIP Entities SHALL employ a default SA Lifetime
of one hour and a default Anti-replay window of 32 sequence numbers.
When a TCP Connection is established between two FCIP_DEs, two
unidirectional SAs are created for that connection and each SA is
identified in the form of a Security Parameter Index (SPI). One SA
is associated with the incoming traffic flow and the other SA is
Rajagopal, et al. Standards Track [Page 39]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
associated with the outgoing traffic flow. The FCIP_DEs at each end
of the TCP connection MUST maintain the SPIs for both its incoming
and outgoing FCIP Encapsulated Frames.
FCIP Entities MAY provide administrative management of
Confidentiality usage. These management interfaces SHOULD be
provided in a secure manner, so as to prevent an attacker from
subverting the security process by attacking the management interface.
10.3.3 ESP Replay Protection and Rekeying issues
FCIP Entities MUST implement Replay Protection against ESP Sequence
Number wrap, as described in [18]. In addition, based on the cipher
algorithm and the number of bits in the cipher block size, the
validity of the key may become compromised. In both cases, the SA
needs to be reestablished.
FCIP Entities MUST use the results of IKE Phase 1 negotiation for
initiating an IKE Phase 2 "Quick Mode" exchange and establish new SAs.
To enable smooth transition of SAs, it is RECOMMENDED that both FCIP
Entities refresh the SPI when sequence number counter reaches 2^31
(i.e., half the sequence number space). It also is RECOMMENDED that
the receiver operate with multiple SPIs for the same TCP Connection
for a period of 2^31 sequence number packets before aging out an SPI.
When a new SPI is created for the outgoing direction, the sending
side SHALL begin using it for all new FCIP Encapsulated Frames.
Frames that are either in-flight, or resent due to TCP
retransmissions etc. MAY use either the new SPI or the one being
replaced.
10.4 Secure FCIP Link Operation
10.4.1 FCIP Link Initialization Steps
When an FCIP Link is initialized, before any FCIP TCP Connections
are established, the local SPD is consulted to determine if IKE
Phase 1 has been completed with the FCIP Entity in the peer FCIP
Entity, as identified by the WWN.
If Phase 1 is already completed, IKE Phase 2 proceeds. Otherwise,
IKE Phase 1 MUST be completed before IKE Phase 2 can start. Both IKE
Phase 1 and Phase 2 transactions use UDP Port 500. If IKE Phase 1
fails, the FCIP Link initialization terminates. Otherwise, the FCIP
Link initialization moves to TCP Connection Initialization.
Rajagopal, et al. Standards Track [Page 40]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
As described in section 9.1, FCIP Entities exchange an FCIP Special
Frame, for forming an FCIP Link. The use of ESP Confidentiality is
an effective countermeasure against any perceived security risks of
FCIP Special Frame.
10.4.2 TCP Connection Security Associations (SAs)
For a TCP Connection establishment, IKE Phase 2 is employed,
resulting in an SA, identified by an SPI. All IP datagrams of the
TCP Connection MUST carry an ESP header with a valid SPI and
Sequence Number to be accepted as valid by the receiving peer.
An implementation is free to perform several IKE Phase 2
negotiations and cache them in its local SPIs, although entries in
such a cache can be flushed per current SA Lifetime settings.
When a TCP Connection is terminated or closed, all SAs associated
with it MUST be removed from the local SAD.
10.4.3 Handling data integrity and confidentiality violations
Upon datagram reception, when the ESP packet fails an integrity
check, the receiver MUST drop the datagram, which will trigger TCP
retransmission. If many such datagrams are dropped, a receiving FCIP
Entity MAY close the TCP Connection and notify the FC Entity with
the reason for the closure.
An implementation MAY audit such events as a diagnostic aid.
Confidentiality checks MUST be performed if Confidentiality is
enabled.
10.4.4 Handling SA parameter mismatches
When SA parameters do not match, the TCP Connection may reach a
point where no traffic moves, or there are excessive TCP
retransmissions. In such a case, either side MAY take one of the
following actions:
a) Reestablish another set of SA parameters; or
b) Close the TCP Connection and notify the FC Entity with the
reason for the closure.
11. Performance
11.1 Performance Considerations
Traditionally, the links between FC Fabric components have been
characterized by low latency and high throughput. The purpose of
Rajagopal, et al. Standards Track [Page 41]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
FCIP is to replace some of these links with an IP Network, where low
latency and high throughput are not as certain. It follows that FCIP
Entities and their counterpart FC Entities probably will be
interested in optimal use of the IP Network.
Many options exist for ensuring high throughput and low latency
appropriate for the distances involved in an IP Network. For
example, a private IP Network might be constructed for the sole use
of FCIP Entities. The options that are within the scope of this
specification are discussed here.
One option for increasing the probability that FCIP data streams
will experience low latency and high throughput is the IP QoS
techniques discussed in section 11.2. This option can have value
when applied to a single TCP Connection. Depending on the
sophistication of the FC Entity, further value may be obtained by
having multiple TCP Connections with differing QoS characteristics.
There are many reasons why an FC Entity might request creation of
multiple TCP Connections within an FCIP_LEP. These reasons include a
desire to provide differentiated service for different TCP data
connections between FCIP_LEPs or a preference to separately queue
different streams of traffic not having a common in-order delivery
requirement.
At the time a new TCP Connection is created, the FC Entity SHALL
specify to the FCIP Entity the QoS characteristics (including but
not limited to IP per-hop-behavior) to be used for the lifetime of
that connection. This MAY be achieved by having:
a) only one set of QoS characteristics for all TCP Connections;
b) a default set of QoS characteristics that the FCIP Entity
applies in the absence of differing instructions from the FC
Entity; or
c) a sophisticated mechanism for exchanging QoS requirements
information between the FC Entity and FCIP Entity each time a
new TCP Connection is created.
Once established, the QoS characteristics of a TCP Connection SHALL
NOT be changed, since this specification provides no mechanism for
the FC Entity to control such changes. The mechanism for providing
different QoS characteristics in FCIP is the establishment of a
different TCP Connections and associated FCIP_DEs.
When FCIP is used with a network with a large (bandwidth*delay)
product, it is RECOMMENDED that FCIP_LEPs use the TCP mechanisms
(window scaling and wrapped sequence protection) for Long Fat
Networks (LFNs) as defined in RFC 1323 [10].
Rajagopal, et al. Standards Track [Page 42]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
11.2 IP Quality of Service (QoS) Support
Many methods of providing QoS have been devised or proposed. These
include (but are not limited to) the following:
- Multi-Protocol Label Switching (MPLS)
- Differentiated Services Architecture (diffserv) -- RFC 2474
[21], RFC 2475 [22], RFC 2597 [23], and RFC 2598 [24] -- and
other forms of per-hop-behavior (PHB)
- Integrated Services, RFC 1633 [11]
- IEEE 802.1p
The purpose of this specification is not to specify any particular
form of IP QoS but rather to specify only those issues that must be
addressed in order to maximize interoperability between FCIP
equipment that has been manufactured by different vendors.
It is RECOMMENDED that some form of preferential QoS be used for
FCIP traffic to minimize latency and drop precedence. No particular
form of QoS is recommended.
If a PHB IP QoS is implemented, it is RECOMMENDED that it
interoperate with diffserv (see RFC 2474 [21], RFC 2475 [22], RFC
2597 [23], and RFC 2598 [24]).
If diffserv/PHB QoS is NOT implemented, the DSCP field for all IP
packets SHALL be set to '000000'.
12. Normative References
The references in this section were current as of the time this
specification was approved. This specification is intended to
operate with newer version of the referenced documents and looking
for newer reference documents is recommended.
[1] Bradner, S., "The Internet Standards Process -- Revision 3", BCP
9, RFC 2026, October 1996.
[2] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[3] Fibre Channel Backbone (FC-BB), ANSI NCITS.342:200x, March 5,
2001 (http://www.t11.org/t11/docreg.nsf/ldl/fc-bb).
[4] Fibre Channel Backbone -2 (FC-BB-2), T11 Project 1466-D, (http://
www.t11.org/t11/docreg.nsf/ldl/fc-bb-2).
Rajagopal, et al. Standards Track [Page 43]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
[5] Fibre Channel Switch Fabric -2 (FC-SW-2), ANSI NCITS.355:200x,
May 23, 2001 (http://www.t11.org/t11/docreg.nsf/ldl/fc-sw-2).
[6] Fibre Channel Framing and Signaling (FC-FS), T11 Project 1331-D,
Rev 1.2, February 16, 2001 (http://www.t11.org/t11/docreg.nsf/
ldl/fc-fs).
[7] http://www.t11.org
[8] "Transmission Control Protocol", RFC 793, Sept. 1981.
[9] Braden, R., "Requirements for Internet Hosts -- Communication
Layers", RFC 1122, October 1989
[10] Jacobson, V., Braden, R. and Borman, D., "TCP Extensions for
High Performance", RFC 1323, May 1992.
[11] R. Braden, et. al., ISI, "Integrated Services in the Internet
Architecture: an Overview", RFC 1633, June 1994
[12] Eastlake, D., Crocker, S., and Schiller, J., "Randomness
Recommendations for Security", RFC 1750, Dec. 1994.
[13] Mills, D., "Simple Network Time Protocol (SNTP) Version 4 for
IPv4, IPv6 and OSI", RFC 2030, October 1996.
[14] Kent, S. and Atkinson, R., "Security Architecture for the
Internet Protocol", RFC 2401, Nov. 1998.
[15] Krawczyk, H., Bellare, M., and Canetti, R., "HMAC: Keyed-
Hashing for Message Authentication", RFC 2104, February 1997.
[16] Kent, S. and Atkinson, R., "IP Encapsulating Security Payload
(ESP)", RFC 2406, Nov. 1998.
[17] Piper, D., "The Internet IP Security Domain of Interpretation
of ISAKMP", RFC 2407, November 1998.
[18] Harkins, D. and Carrel, D., "The Internet Key Exchange (IKE)",
RFC 2409, November 1998.
[19] Glenn, R., Kent, S., "The NULL Encryption Algorithm and Its Use
With IPsec", RFC 2410, Nov. 1998
[20] Thayer, R., Glenn, R., and Doraswamy, N., "IP Security Document
Roadmap", RFC 2411, Nov. 1998.
Rajagopal, et al. Standards Track [Page 44]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
[21] Nichols, K., Blake, S., Baker, F. and D. Black, "Definition of
the Differentiated Services Field (DS Field) in the IPv4 and
Ipv6 Headers", RFC 2474, December 1998.
[22] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., Weiss,
W., "An Architecture for Differentiated Services", RFC 2475,
Dec. 1998.
[23] Heinanen, J., Baker, F., Weiss, W., Wroclawski, J., "An Assured
Forwarding PHB", RFC 2597, June 1999.
[24] Jacobson, V., Nichols, K., Poduri, K., "An Expedited Forwarding
PHB Group", RFC 2598, June 1999.
[25] E.Guttman, C. Perkins, J. Veizades, M. Day. Service Location
Protocol, version 2, RFC 2608, July, 1999.
[26] Floyd, et al, "SACK Extension", RFC 2883, July 2000.
[27] Weber, Rajagopal, Travostino, Chau, O'Donnell, Monia Merhar,
"FC Frame Encapsulation", draft-ietf-ips-fcencapsulation-__.txt
(RFC reference and date to be added during standards action).
[28] Peterson, "Finding FCIP Entities Using SLP", draft-ietf-ips-
fcip-slp-___.txt (RFC reference and date to be added during
standards action).
[29] Walker, J., Moskowitz, R., "The AES128 CTR Mode of Operation
and Its Use with IPsec", Internet draft (work in progress),
draft-moskowitz-aes128-ctr-00.txt, September 2001.
[30] Frankel, S., Kelly, S., Glenn, R., "The AES Cipher Algorithm
and Its Use with IPsec", Internet draft (work in progress),
draft-ietf-ipsec-ciph-aes-cbc-01.txt, May 2001.
13. Informative References
The following references may prove informative to readers unfamiliar
with Fibre Channel.
Kembel, R., "The Fibre Channel Consultant: A Comprehensive
Introduction", Northwest Learning Associates, 1998
Rajagopal, et al. Standards Track [Page 45]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
14. Acknowledgments
Funding for the RFC Editor function is currently provided by the
Internet Society.
15. Contributors' Addresses
Murali Rajagopal Vi Chau
USA USA
Phone: +1 949 280 6516 Email: vchau1@cox.net
Email: muralir@cox.net
Elizabeth G. Rodriguez Neil Wanamaker
USA Akara
Phone: +1 214 495 8712 10624 Icarus Court
Fax: +1 214 495 8712 Austin, TX 78726
Email: USA
ElizabethRodriguez@ieee.org Phone: +1 512 257 7633
Fax: +1 512 257 7877
Email: nwanamaker@akara.com
Ralph Weber Steve Wilson
ENDL Texas, representing Brocade Brocade Comm. Systems, Inc.
Suite 102 PMB 178 1745 Technology Drive
18484 Preston Road San Jose, CA. 95110
Dallas, TX 75252 USA
USA Phone: +1 408 487 8128
Phone: +1 214 912 1373 Fax: +1 408 487 8101
Email: roweber@acm.org email: swilson@brocade.com
Bob Snively David Peterson
Brocade Comm. Systems, Inc. Cisco Systems - SRBU
1745 Technology Drive 6450 Wedgwood Road
San Jose, CA 95110 Maple Grove, MN 55311
USA USA
Phone: +1 408 487 8135 Phone: +1 763 398 1007
Email: rsnively@brocade.com Cell: +1 612 802 3299
Email: dap@cisco.com
Donald R. Fraser R. Andy Helland
Compaq Computer Corporation LightSand Communications, Inc.
301 Rockrimmon Blvd., Bldg. 5 375 Los Coches Street
Colorado Springs, CO 80919 Milpitas, CA 95035
USA USA
Phone: +1 719 548 3272 Phone: +1 408 404 3119
Email: don.fraser@compaq.com Fax: +1 408 941 2166
Email: andyh@lightsand.com
Rajagopal, et al. Standards Track [Page 46]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Raj Bhagwat Bill Krieg
LightSand Communications, Inc. Lucent Technologies
24411 Ridge Route Dr. 200 Lucent Lane
Suite 135 Cary, NC 27511
Laguna Hills, CA 92653 USA
USA Phone: +1 919 463 4020
Phone: +1 949 837 1733 x104 Fax: +1 919 463 4041
Email: rajb@lightsand.com Email: bkrieg@lucent.com
Michael E. O'Donnell Anil Rijhsinghani
McDATA Corporation McDATA Corporation
310 Interlocken Parkway 5 Brickyard lane
Broomfield, Co. 80021 Westboro, MA 01581
USA USA
Phone: +1 303 460 4142 Phone: +1 508 870 6593
Fax: +1 303 465 4996 Email:
Email: modonnell@mcdata.com anil.rijhsinghani@mcdata.com
Milan J. Merhar Craig W. Carlson
43 Nagog Park QLogic Corporation
Pirus Networks 6321 Bury Drive
Acton, MA 01720 Eden Prairie, MN 55346
USA USA
Phone: +1 978 206 9124 Phone: +1 952 932 4064
Email: Milan@pirus.com Email: craig.carlson@qlogic.com
Venkat Rangan Lawrence J. Lamers
Rhapsody Networks Inc. SAN Valley Systems, Inc.
3450 W. Warren Ave. 6320 San Ignacio Ave.
Fremont, CA 94538 San Jose, CA 95119-1209
USA USA
Phone: +1 510 743 3018 Phone: +1 408 234 0071
Fax: +1 510 687 0136 Email: ljlamers@ieee.org
Email: venkat@rhapsodynetworks.com
Ken Hirata Jim Nelson
Vixel Corporation Vixel Corporation
15245 Alton Parkway, Suite 100 15245 Alton Parkway, Suite 100
Irvine, CA 92618 Irvine, CA 92618
USA USA
Phone: +1 949 788 6368 Phone: +1 949 450 6159
Fax: +1 949 753 9500 Fax: +1 949 753 9500
Email: ken.hirata@vixel.com Email: Jim.Nelson@vixel.com
Rajagopal, et al. Standards Track [Page 47]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
16. Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
ANNEX A - IANA Considerations
IANA has made the following port assignments to FCIP:
- fcip-port 3225/tcp FCIP
- fcip-port 3225/udp FCIP
ANNEX B - FCIP Usage of Addresses and Identifiers
In support of network address translators, FCIP does not use IP
Addresses to identify FCIP Entities or FCIP_LEPs. The only use of IP
Addresses for identification occurs when initiating new TCP connect
requests (see section 9.1.2.3) where the IP Address destination of
the TCP connect request is used to answer the question: "Have
previous TCP connect requests been made to the same destination FCIP
Entity?" The correctness of this assumption is further checked by
sending the Destination FC Fabric Entity World Wide Name in the
Special Frame and having the value checked by the FCIP Entity that
Rajagopal, et al. Standards Track [Page 48]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
receives the TCP connect request and Special Frame (see section
9.1.3).
For the purposes of processing incoming TCP connect requests, the
source FCIP Entity is identified by the Source FC Fabric Entity
World Wide Name and Source FC/FCIP Entity Identifier fields in the
Special Frame sent from the TCP connect requestor to the TCP connect
recipient as the first bytes following the TCP connect request (see
section 9.1.2.3 and section 9.1.3).
FC-BB-2 [4] provides the definitions for each of the following
Special Frame fields:
- Source FC Fabric Entity World Wide Name,
- Source FC/FCIP Entity Identifier, and
- Destination FC Fabric Entity World Wide Name.
As described in section 9.1.3, FCIP Entities segregate their
FCIP_LEPs between:
- Connections resulting from TCP connect requests initiated by the
FCIP Entity, and
- Connections resulting from TCP connect requests received by the
FCIP Entity.
Within each of these two groups, the following information is used
to further identify each FCIP_LEP:
- Source FC Fabric Entity World Wide Name,
- Source FC/FCIP Entity Identifier, and
- Destination FC Fabric Entity World Wide Name.
ANNEX C - Example of synchronization recovery algorithm
The contents of this annex are informative.
Synchronization may be recovered as specified in section 6.6.2.3. An
example of an algorithm for searching the bytes delivered to the
Encapsulated Frame Receiver Portal for a valid FCIP Frame header is
provided in this annex.
This resynchronization uses the principle that a valid FCIP data
stream must contain at least one valid header every 2176 bytes (the
maximum length of an encapsulated FC Frame). Although other data
patterns containing apparently valid headers may be contained in the
stream, the FC CRC or FCIP Frame validity of the data patterns
contained in the data stream will always be either interrupted by or
resynchronized with the valid FCIP Frame headers.
Rajagopal, et al. Standards Track [Page 49]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Consider the case shown in figure 11. A series of short FCIP Frames,
perhaps from a trace, are embedded in larger FCIP Frames, say as a
result of a trace file being transferred from one disk to another.
The headers for the short FCIP Frames are denoted SFH and the long
FCIP Frame headers are marked as LFH.
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
|L| |S| |S| |S| |S| |L| |S|
|F| |F| |F| |F| |F| |F| |F|...
|H| |H| |H| |H| |H| |H| |H|
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
| |
|<---------2176 bytes-------->|
Fig. 11 Example of resynchronization data stream
A resynchronization attempt that starts just to the right of an LFH
will find several SFH FCIP Frames before discovering that they do
not represent the transmitted stream of FCIP Frames. Within 2176
bytes plus or minus, however, the resynchronization attempt will
encounter an SFH whose length does not match up with the next SFH
because the LFH will fall in the middle of the short FCIP Frame
pushing the next header farther out in the byte stream.
Note that the resynchronization algorithm cannot forward any
prospective FC Frames to the FC Transmitter Portal because until
synchronization is completely established there is no certainty that
anything that looked like an FCIP Frame really was one. For example,
an SFH might fortuitously contain a length that points exactly to
the beginning of an LFH. The LFH would identify the correct
beginning of a transmitted FCIP Frame, but that in no way guarantees
that the SFH was also a correct FCIP Frame header.
There exist some data streams that cannot be resynchronized by this
algorithm. If such a data stream is encountered, the algorithm
causes the TCP Connection to be closed.
The resynchronization assumes that security and authentication
procedures outside the FCIP Entity are protecting the valid data
stream from being replaced by an intruding data stream containing
valid FCIP data.
Rajagopal, et al. Standards Track [Page 50]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The following steps are one example of how an FCIP_DE might
resynchronize with the data stream entering the Encapsulated Frame
Receiver Portal.
1) Search for candidate and strong headers:
The data stream entering the Encapsulated Frame Receiver Portal
is searched for 12 bytes in a row containing the required values
for:
a) Protocol field,
b) Version field,
c) ones complement of the Protocol field,
d) ones complement of the Version field,
e) replication of encapsulation word 0 in word 1, and
f) Reserved field and its ones complement.
If such a 12-byte grouping is found, the FCIP_DE assumes that it
has identified bytes 0-2 of a candidate FCIP encapsulation header.
All bytes up to and including the candidate header byte are
discarded.
If no candidate header has been found after searching a
specified number of bytes greater than some multiple of 2176
(the maximum length of an FCIP Frame), resynchronization has
failed and the TCP/IP connection is closed.
Word 3 of the candidate header contains the Frame Length and
Flags fields and their ones complements. If the fields are
consistent with their ones complements, the candidate header is
considered a strong candidate header. The Frame Length field is
used to determine where in byte stream the next strong candidate
header should be and processing continues at step 2).
2) Use multiple strong candidate headers to locate a verified
candidate header:
The Frame Length in one strong candidate header is used to skip
incoming bytes until the expected location of the next strong
candidate header is reached. Then the tests described in step 1)
are applied to see if another strong candidate header has
successfully been located.
All bytes skipped and all bytes in all strong candidate headers
processed are discarded.
Strong candidate headers continue to be verified in this way for
at least 4352 bytes (twice the maximum length of an FCIP Frame).
Rajagopal, et al. Standards Track [Page 51]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
If at anytime a verification test fails, processing restarts at
step 1 and a retry counter is incremented. If the retry counter
exceeds 3 retries, resynchronization has failed and the TCP
Connection is closed.
After strong candidate headers haves been verified for at least
4352 bytes, the next header identified is a verified candidate
header and processing continues at step 3).
Note: If a strong candidate header was part of the data content
of an FCIP Frame, the FCIP Frame defined by that or a subsequent
strong candidate header will eventually cross an actual header
in the byte stream. As a result it will either identify the
actual header as a strong candidate header or it will lose
synchronization again because of the extra 28 bytes in the
length, returning to step 1 as described above.
3) Use multiple strong candidate headers to locate a verified
candidate header:
Incoming bytes are skipped and discarded until the next verified
candidate header is reached. Each verified candidate header is
tested against the full collection of tests listed in section
6.6.2.2 as would normally be the case.
Verified candidate headers continue to be located and tested in
this way for a minimum of 4352 bytes (twice the maximum length
of an FCIP Frame). If all verified candidate headers encountered
are valid, the last verified candidate header is a valid header.
At this point the FCIP_DE stops discarding bytes and begins
normal FCIP de-encapsulation begins, including for the first
time since synchronization was lost, delivery of FC frames
through the FC Transmitter Portal according to normal FCIP rules.
If any verified candidate headers are invalid but meet all the
requirements of a strong candidate header, increment the retry
counter and return to step 2). If any verified candidate headers
are invalid and fail to meet the tests for a strong candidate
header, increment the retry counter and return to step 1. If the
retry counter exceeds 4 retries, resynchronization has failed
and the TCP/IP connection is closed.
A flowchart for this algorithm can be found in figure 12.
Rajagopal, et al. Standards Track [Page 52]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
Synchronization is lost
|
_____________v_______________
| |
| Search for candidate header |
+----------->| |
| | Found Not Found |
| | (Strong candidate) |
| |_____________________________|
| | |
| | + --------->close TCP
| _______v_____________________ Connection
| | | and notify
| | Enough strong candidate | the FC Entity
| +---->| headers identified? | with the reason
| | | | for closure
| | | No Yes |
| | | (Verified candidate) |
| | |_____________________________|
|___________________| |
^ | |
| | |
| | _______________________v_____
| | | |
| | | Enough verified candidate |
| | | headers validated? |
| | | |
| | | No Yes |
| | | (Resynchronized) |
| | |_____________________________|
| | | |
| | ______v__________ | Resume
| | | | + ---> Normal
| | | Synchronization | De-encapsulation
| | | Lost? |
| | | |
| | | No Yes |
| | |_________________|
| | | |
| |________| |
|___________________________|
Fig. 12 Flow diagram of simple synchronization example
Rajagopal, et al. Standards Track [Page 53]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
ANNEX D - Relationship between FCIP and IP over FC (IPFC)
The contents of this annex are informative.
IPFC (RFC 2625) describes the encapsulation of IP packets in FC
Frames. It is intended to facilitate IP communication over an FC
network.
FCIP describes the encapsulation of FC Frames in TCP segments which
in turn are encapsulated inside IP packets for transporting over an
IP network. It gives no consideration to the type of FC Frame that
is being encapsulated. Therefore, the FC Frame may actually contain
an IP packet as described in the IP over FC specification (RFC
2625). In such a case, the data packet would have:
- Data Link Header
- IP Header
- TCP Header
- FCIP Header
- FC Header
- IP Header
Note: The two IP headers would not be identical to each other. One
would have information pertaining to the final destination while the
other would have information pertaining to the FCIP Entity.
The two documents focus on different objectives. As mentioned above,
implementation of FCIP will lead to IP encapsulation within IP.
While perhaps inefficient, this should not lead to issues with IP
communication. One caveat: if a Fibre Channel device is
encapsulating IP packets in an FC Frame (e.g. an IPFC device), and
that device is communicating with a device running IP over a non-FC
medium, a second IPFC device may need to act as a gateway between
the two networks. This scenario is not specifically addressed by FCIP.
There is nothing in either of the specifications to prevent a single
device from implementing both FCIP and IP-over-FC (IPFC), but this
is implementation specific, and is beyond the scope of this document.
ANNEX E - FC Frame Format
The contents of this annex are informative.
All FC Frames have a standard format (see FC-FS [6]) much like LAN's
802.x protocols. However, the exact size of each FC Frame varies
depending on the size of the variable fields. The size of the
Rajagopal, et al. Standards Track [Page 54]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
variable field ranges from 0 to 2112-bytes as shown in the FC Frame
Format in figure 13 resulting in the minimum size FC Frame of 36
bytes and the maximum size FC Frame of 2148 bytes. Valid FC Frame
lengths are always a multiple of four bytes.
+------+--------+-----------+----//-------+------+------+
| SOF |Frame |Optional | Frame | CRC | EOF |
| (4B) |Header |Header | Payload | (4B) | (4B) |
| |(24B) |<----------------------->| | |
| | | Data Field = (0-2112B) | | |
+------+--------+-----------+----//-------+------+------+
Fig. 13 FC Frame Format
E.1 SOF and EOF Delimiters
On an FC link, Start-of-Frame (SOF) and End-Of-Frame (EOF) are
called Ordered Sets and are sent as special words constructed from
the 8B/10B comma character (K28.5) followed by three additional 8B/
10B data characters making them uniquely identifiable in the data
stream.
On an FC link the SOF delimiter serves to identify the beginning of
an FC Frame and prepares the receiver for FC Frame reception. The
SOF contains information about the FC Frame's Class of Service,
position within a sequence, and in some cases, connection status.
The EOF delimiter identifies the end of the FC Frame and the final
FC Frame of a sequence. In addition, it serves to force the running
disparity to negative. The EOF is used to end the connection in
connection-oriented classes of service.
A special EOF delimiter called EOFa (End Of Frame - Abort) is used
to terminate a partial FC Frame resulting from a malfunction in a
link facility during transmission. Since an FCIP Entity functions
like a transmission link with respect to the rest of the FC Fabric,
FCIP_DEs may use EOFa in their error recovery procedures.
It is therefore important to preserve the information conveyed by
the delimiters across the IP-based network, so that the receiving
FCIP Entity can correctly reconstruct the FC Frame in its original
SOF and EOF format before forwarding it to its ultimate FC
destination on the FC link.
When an FC Frame is encapsulated and sent over a byte-oriented
interface, the SOF and EOF delimiters are represented as sequences
of four consecutive bytes, which carry the equivalent Class of
Service and FC Frame termination information as the FC ordered sets.
Rajagopal, et al. Standards Track [Page 55]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The representation of SOF and EOF in an encapsulation FC Frame is
described in FC Frame Encapsulation [27].
E.2 Frame Header
The FC Frame Header is transparent to the FCIP Entity. The FC Frame
Header is 24 bytes long and has several fields that are associated
with the identification and control of the payload. Current FC
Standards allow up to 3 Optional Header fields [6]:
- Network_Header (16-bytes)
- Association_Header (32-bytes)
- Device_Header (up to 64-bytes).
E.3 Frame Payload
The FC Frame Payload is transparent to the FCIP Entity. An FC
application level payload is called an Information Unit at the FC-4
Level. This is mapped into the FC Frame Payload of the FC Frame. A
large Information Unit is segmented using a structure consisting of
FC Sequences. Typically, a Sequence consists of more than one FC
Frame. FCIP does not maintain any state information regarding the
relationship of FC Frames within a FC Sequence.
E.4 CRC
The FC CRC is 4 bytes long and uses the same 32-bit polynomial used
in FDDI and is specified in ANSI X3.139 Fiber Distributed Data
Interface. This CRC value is calculated over the entire FC header
and the FC payload; it does not include the SOF and EOF delimiters.
Note: When FC Frames are encapsulated into FCIP Frames, the FC Frame
CRC is untouched by the FCIP Entity.
ANNEX F - FC Encapsulation Format
This annex contains a reproduction of the FC Encapsulation Format
[27] as it applies to FCIP Frames that encapsulate FC Frames. The
information in this annex is not intended to represent the FCIP
Special Frame that is described in section 8.
The information in this annex was correct as of the time this
specification was approved. The information in this annex is
informative only.
Rajagopal, et al. Standards Track [Page 56]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
If there are any differences between the information here and the FC
Encapsulation Format specification [27], the FC Encapsulation Format
specification takes precedence.
If there are any differences between the information here and the
contents of section 6.6.1, then the contents of section 6.6.1 take
precedence.
Figure 14 applies the requirements stated in section 6.6.1 and in
the FC Encapsulation Frame format resulting in a summary of the FCIP
frame format. Where FCIP requires specific values, those values are
shown in hexadecimal in parentheses. Detailed requirements for the
FCIP usage of the FC Encapsulation Format are in section 6.6.1.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|
+---------------+---------------+---------------+---------------+
0| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
1| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
| (0x00) | (0x00) | (0xFF) | (0xFF) |
+-----------+---+---------------+-----------+---+---------------+
3| Flags | Frame Length | -Flags | -Frame Length |
| (0x00) | | (0x3F) | |
+-----------+-------------------+-----------+-------------------+
4| Time Stamp [integer] |
+---------------------------------------------------------------+
5| Time Stamp [fraction] |
+---------------------------------------------------------------+
6| CRC (Reserved in FCIP) |
| (0x00-00-00-00) |
+---------------+---------------+---------------+---------------+
7| SOF | SOF | -SOF | -SOF |
+---------------+---------------+---------------+---------------+
8| |
+----- FC frame content (see annex E) -----+
| |
+---------------+---------------+---------------+---------------+
n| EOF | EOF | -EOF | -EOF |
+---------------+---------------+---------------+---------------+
Fig. 14 FCIP Frame Format
Rajagopal, et al. Standards Track [Page 57]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
The names of fields are generally descriptive on their contents and
the FC Encapsulation Format specification [27] is referenced for
details. Field names preceded by a minus sign are one's complement
values of the named field.
Note: Figure 14 does not represent the FCIP Special Frame that is
described in section 8.
ANNEX G - FCIP Requirements on an FC Entity
The contents of this annex are informative for FCIP but might be
considered normative on FC-BB-2.
The capabilities that FCIP requires of an FC Entity include:
1) The FC Entity must deliver FC Frames to the correct FCIP Data
Engine (in the correct FCIP Link Endpoint).
2) Each FC Frame delivered to an FCIP_DE must be accompanied by a
time value synchronized with the clock maintained by the FC
Entity at the other end of the FCIP Link (see section 7). If a
synchronized time value is not available, a value of zero must
accompany the FC Frame.
3) When FC Frames exit FCIP Data Engine(s) via the FC Transmitter
Portal(s), the FC Entity should forward them to the FC Fabric.
However, before forwarding a FC Frame the FC Entity must compute
the end-to-end transit time for the FC Frame using the time
value supplied by the FCIP_DE (taken from the FCIP header) and a
synchronized time value (see section 7). If the end-to-end
transit time exceeds the requirements of the FC Fabric, the FC
Entity is responsible for discarding the FC Frame.
4) The only delivery ordering guarantee provided by FCIP is
correctly ordered delivery of FC Frames between a pair of FCIP
Data Engines. FCIP expects the FC Entity to implement all other
FC Frame delivery ordering requirements.
5) When a TCP connect request is received and that request would
add a new TCP Connection to an existing FCIP_LEP, the FC Entity
must authenticate the source of the TCP connect request before
use of the new TCP connection is allowed.
6) The FC Entity may participate in determining allowed TCP
Connections, TCP Connection parameters, quality of service
usage, and security usage by modifying interactions with the
Rajagopal, et al. Standards Track [Page 58]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
FCIP Entity that are modelled as a "shared" database in section
9.1.1.
7) The FC Entity may require the FCIP Entity to perform TCP close
requests.
8) The FC Entity may recover from connection failures.
9) The FC Entity must recover from events that the FCIP Entity
cannot handle, such as:
a) loss of synchronization with FCIP Frame headers from the
Encapsulated Frame Receiver Portal requiring resetting the
TCP Connection; and
b) recovering from FCIP Frames that are discarded as a result
of synchronization problems (see section 6.6.2.2 and section
6.6.2.3).
10) The FC Entity must work cooperatively with the FCIP Entity to
manage flow control problems in either the IP Network or FC
Fabric.
11) The FC Entity may test for failed TCP Connections.
Note that the Fibre Channel standards must be consulted for a
complete understanding of the requirements placed on an FC Entity.
The following table shows the explicit interactions between the FCIP
Entity and the FC Entity.
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| 6.6 | FC Frame ready | | Provide FC |
| FCIP Data | for IP transfer | | Frame and |
| Engine | | | time stamp at |
| | | | FC Receiver |
| | | | Portal |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Rajagopal, et al. Standards Track [Page 59]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 6.6 | FCIP Frame | Provide FC | |
| FCIP Data | received from | Frame and | |
| Engine | IP Network | time stamp at | |
| | | FC Transmitter | |
| | | Portal | |
+-------------+-----------------+-----------------+-----------------+
| 6.6.2.2 | FCIP_DE | Inform FC | |
| Errors | discards bytes | Entity that | |
| in FCIP | delivered | bytes have been | |
| Headers and | through | discarded with | |
| Discarding | Encapsulated | reason | |
| FCIP Frames | Frame Receiver | | |
| | Portal | | |
+-------------+-----------------+-----------------+-----------------+
| 6.6.2.3 | FCIP Entity | Inform FC | |
| Synchron- | closes TCP | Entity that TCP | |
| ization | Connection due | Connection has | |
| Failures | to synchron- | been closed | |
| | ization failure | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| 9.1.2.3 | Receipt of the | Inform FC | |
| Connection | echoed SF takes | Entity that TCP | |
| Setup | too long or the | Connection has | |
| Following a | SF contents | been closed | |
| Successful | have changed | with reason | |
| TCP Connect | | for closure | |
| Request | | | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Rajagopal, et al. Standards Track [Page 60]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 9.1.2.1 | New TCP | Inform FC | |
| Non-Dynamic | Connection | Entity of | |
| Creation of | created based | new or existing | |
| a New TCP | on "shared" | FCIP_LEP and | |
| Connections | database | new FCIP_DE | |
| | information | along with | |
| | | Destination FC | |
| | | Fabric Entity | |
| | | WWN, Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| 9.1.2.2 | New TCP | Inform FC | |
| Dynamic | Connection | Entity of | |
| Creation of | created based | new or existing | |
| a New TCP | on SLP service | FCIP_LEP and | |
| Connections | advertisement | new FCIP_DE | |
| | and "shared" | along with | |
| | database | Destination FC | |
| | information | Fabric Entity | |
| | | WWN, Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Rajagopal, et al. Standards Track [Page 61]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 9.1.3 | New TCP | Inform FC | |
| Processing | Connection | Entity of | |
| Incoming | created based | new or existing | |
| TCP Connect | on incoming TCP | FCIP_LEP and | |
| Requests | Connect request | new FCIP_DE | |
| | and "shared" | along with | |
| | database | Source FC | |
| | information | Fabric Entity | |
| | | WWN, Source | |
| | | FC/FCIP Entity | |
| | | Identifier, | |
| | | Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| 9.1.3 | TCP Connect | Request FC | Yes or No |
| Processing | Request wants | Entity to | answer about |
| Incoming | to add a new | authenticate | whether the |
| TCP Connect | TCP Connection | the source of | source of the |
| Requests | to an existing | the TCP Connect | TCP Connect |
| | FCIP_LEP | Request | Request can be |
| | | | authenticated |
+-------------+-----------------+-----------------+-----------------+
| 9.1.3 | Receipt of the | Inform FC | |
| Processing | SF takes too | Entity that TCP | |
| Incoming | long or | Connection has | |
| TCP Connect | duplicate | been closed | |
| Requests | Connection | with reason | |
| | Nonce value | for closure | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Rajagopal, et al. Standards Track [Page 62]
Internet-Draft Fibre Channel Over TCP/IP (FCIP) February, 2002
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| concluded |
+-------------+-----------------+-----------------+-----------------+
| 9.2 | FC Entity | Acknowledgement | Identification |
| Closing TCP | determines | of TCP | of the FCIP_DE |
| Connections | that a TCP | Connection | whose TCP |
| | Connection | closure | Connection |
| | needs to be | | needs to be |
| | closed | | closed |
+-------------+-----------------+-----------------+-----------------+
| 9.5 | Discovery that | Inform FC | |
| TCP | TCP connectiv- | Entity that TCP | |
| Connection | ity has been | Connection has | |
| Considera- | lost | been closed | |
| tions | | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| 10.4.3 | Excessive | Inform FC | |
| Handling | numbers of | Entity that TCP | |
| data | dropped | Connection has | |
| integrity | datagrams | been closed | |
| and confi- | detected and | with reason | |
| dentiality | TCP Connection | for closure | |
| violations | closed | | |
+-------------+-----------------+-----------------+-----------------+
| 10.4.4 | TCP Connection | Inform FC | |
| Handling SA | closed due to | Entity that TCP | |
| parameter | SA parameter | Connection has | |
| mismatches | mismatch | been closed | |
| | problems | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
Rajagopal, et al. Standards Track [Page 63]