Internet Draft Alex Audu
Expiration: May 2004 Alcatel USA Inc.
File: draft-gopal-forces-fact-06.txt
Working Group: ForCES Ram Gopal
Nokia
Hormuzd Khosravi
Intel
Chaoping Wu
Azanda Network Devices
November 2003
ForwArding and Control ElemenT protocol (FACT)
draft-gopal-forces-fact-06.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of [1]. Internet-Drafts are
working documents of the Internet Engineering Task Force (IETF),
its areas, and its working groups. Note that other groups may
also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as ``work in
progress.''
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in [2].
Abstract
This document defines the FACT protocol that is designed for
communicating between Forwarding Element and Control Elements inside
a network element. This protocol addresses all the requirements
described in the Forces [3] requirements document. This document
also specifies architectural attributes necessary in an
implementation of FACT to ensure correct and secure protocol
operation.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 1]
Internet Draft Forwarding and Control Element protocol November 2003
Table of Content
1. Definitions.....................................................3
2. Introduction....................................................5
3. Protocol Overview...............................................5
3.1. Independence from Interconnect................................7
3.2. Reliability...................................................7
3.3. Separate Control and Data channels............................8
3.4. Fail-Over Model...............................................9
4. FACT Message Overview...........................................9
4.1. Protocol Message Header structure............................10
4.1.1. Version....................................................10
4.1.2. Message Classes and Types..................................10
4.1.3. Length.....................................................12
4.1.4. CE Tag.....................................................12
4.1.5. FE Identifier..............................................13
4.1.6. Priority Bits..............................................13
4.1.7. Transaction Sequence Number (TSN)..........................13
4.2. Service Data or Payload Structure............................13
5. FACT Messages..................................................14
5.1. Association and Connection (CA) Messages.....................14
5.1.1. Join Request...............................................14
5.1.2. Join Response..............................................15
5.1.3. Leave Request..............................................17
5.1.4. Leave Response.............................................17
5.2. Capabilities Control (CAPCO) Messages........................18
5.2.1. Capability Request.........................................18
5.2.2. Capability Response........................................18
5.2.3. Configure Request..........................................19
5.2.4. Configure Response.........................................21
5.2.5. Query Request..............................................22
5.2.6. Query Response.............................................23
5.2.7. Error TLV..................................................24
5.3. PE State Maintenance Messages................................25
5.3.1. Protocol Element UP (PEUP).................................25
5.3.2. Protocol Element Up Acknowledge (PEUP-ACK).................25
5.3.3. Protocol Element Active (PEACT)............................25
5.3.4. Protocol Element Active Acknowledgement (PEACT-ACK)........26
5.3.5. Protocol Element Inactive (PEINACT)........................26
5.3.6. Protocol Element Inactive Acknowledgement (PEINACT-ACK)....27
5.3.7. Protocol Element Down (PEDOWN).............................27
5.3.8. Protocol Element Down Ack (PEDOWN-ACK).....................28
5.3.9. Heartbeat..................................................28
5.3.10. Heartbeat Acknowledgement (HB-ACK)........................28
5.4. PE Traffic Maintenance (PETM) Messages.......................28
5.4.1. Control Packet Redirect to CE..............................29
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 2]
Internet Draft Forwarding and Control Element protocol November 2003
5.4.2. Control Packet Forwarding to FE............................29
5.5. Event Notification Messages..................................30
5.5.1. Event Register.............................................30
5.5.2. Event Register Response....................................31
5.5.3. Event De-Register..........................................31
5.5.4. Event De-Register Response.................................32
5.5.5. Asynchronous FE Event Notification.........................32
5.6. Vendor Specific Messages.....................................33
5.6.1. Vendor Specific Data (VS-DATA Request).....................33
5.6.2. Vendor Specific Data Response (VS-Data Response)...........33
6. Procedures for FACT Protocol...................................33
6.1. CE and FE State Maintenance..................................33
6.1.1. CE and FE States...........................................34
6.2. State Maintenance Procedures.................................35
6.2.1. Protocol Element Up........................................35
6.2.2. Protocol Element Down......................................36
6.2.3. Protocol Element ACTIVE....................................37
6.2.4. Protocol Element Inactive..................................37
7. Example Scenarios..............................................37
7.1. Establishment of Association.................................37
7.2. Steady State Communication...................................38
7.3. CE Fail-over Scenarios.......................................39
8. Security Considerations........................................41
8.1. TLS Usage with FACT..........................................41
9. Architecture support for FACT protocol.........................42
9.1. Configurable parameters......................................42
10. IANA Considerations...........................................42
11. References....................................................42
11.1. Normative References........................................42
11.2. Informative References......................................43
12. Acknowledgments...............................................44
13. Authors' Addresses............................................44
1. Definitions
The following definitions are taken from [3],[5]
Forwarding Element (FE) - A logical entity that implements the
ForCES protocol. FEs use the underlying hardware to provide per-
packet processing and handling as directed by a CE via the ForCES
protocol. FEs may use PFE partitions, whole PFEs, or multiple PFEs.
Control Element (CE) - A logical entity that implements the ForCES
protocol and uses it to instruct one or more FEs how to process
packets. CEs handle functionality such as the execution of control
and signaling protocols.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 3]
Internet Draft Forwarding and Control Element protocol November 2003
Pre-association Phase - The period of time during which a FE Manager
(see below) and a CE Manager (see below) are determining which FE
and CE should be part of the same network element and delivering
that information to the FE and CE.
Post-association Phase - The period of time during which a FE has
the information specifying what CE is to control it and vice versa,
including the time during which the CE and FE are establishing
communication with one another.
ForCES Protocol - While there may be multiple protocols used within
the overall ForCES architecture, the term "ForCES protocol" refers
only to the ForCES post-association phase protocol (see below).
ForCES Post-Association Phase Protocol - The protocol used for post-
association phase communication between CEs and FEs. This protocol
does not apply to CE-to-CE communication, FE-to-FE communication, or
to communication between FE and CE managers. The ForCES protocol is
a master-slave protocol in which FEs are slaves and CEs are masters.
This protocol includes both the management of the communication
channel (e.g., connection establishment, heartbeats) and the control
messages themselves. The term ForCES protocol may refer to a suite
of protocols that are used to exchange control information as well
as redirect data packets between the CEs and FEs.
FE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which CE(s) a FE should
communicate. This process is called CE discovery and may involve
the FE manager learning the capabilities of available CEs. A FE
manager may use anything from a static configuration to a pre-
association phase protocol (see below) to determine which CE(s) to
use. Being a logical entity, a FE manager might be physically
combined with any of the other logical entities mentioned in this
section.
CE Manager - A logical entity that operates in the pre-association
phase and is responsible for determining to which FE(s) a CE should
communicate. This process is called FE discovery and may involve
the CE manager learning the capabilities of available FEs. A CE
manager may use anything from a static configuration to a pre-
association phase protocol (see below) to determine which FE to use.
Being a logical entity, a CE manager might be physically combined
with any of the other logical entities mentioned in this section.
Pre-association Phase Protocol - A protocol between FE managers and
CE managers that is used to determine which CEs or FEs to use. A
pre-association phase protocol may include a CE and/or FE capability
discovery mechanism. Note that this capability discovery process is
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 4]
Internet Draft Forwarding and Control Element protocol November 2003
wholly separate from (and does not replace) that used within the
ForCES protocol. However, the two capability discovery mechanisms
may utilize the same FE model.
ForCES Network Element (NE) - An entity composed of one or more CEs
and one or more FEs. To entities outside a NE, the NE represents a
single point of management. Similarly, a NE usually hides its
internal organization from external entities.
ForCES Protocol Element (PE) - A Forwarding Element or a Control
Element that speaks the ForCES protocol.
LFB (Logical Function Block) class (or type) -- A generic template
representing a fine-grained, logically separable and well-defined
packet processing operation in the datapath. LFB class is the basic
building block of the FE model.
FE Model (FEM) ¡ Modeling/Organization of LFBs in the Forwarding
plane.
2. Introduction
Network elements such as routers play an important role in
transporting IP packets in the Internet. QoS aware routing, policy
based routing and middle-box functions such as firewall, NAT, and
proxies put heavy requirements on per-hop behavior treatment for IP
packets. This complicates the network element.
Routers have emerged from simple monolithic software to a
distributed routing complex that interconnects different networks
and distributes the routing and forwarding logic to line cards and
control cards. A line card does basic forwarding function and a
control card runs all the control and management functions.
Forces [3][4] defines both architectural and protocol requirements
for the communication between CE and FE. The Forwarding and Control
ElemenT (FACT) protocol addresses all the requirement for a Forces
protocol and is described in this document.
3. Protocol Overview
ForCES is a framework consisting of set of protocols representing the
forwarding and control elements in the form of an extensible model
[4][5]. CEs handle control, signaling and management protocols,
while FEs perform forwarding functions. CEs control the behavior of
FEs in a master/slave fashion.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 5]
Internet Draft Forwarding and Control Element protocol November 2003
The FACT protocol is logically separated into a base control
protocol and LFBs service functions. This is similar to SNMP [16]
where the SNMP protocol provides the base protocol for exchange of
messages between SNMP manager and agent(s) and defines the concept
of MIBS and SMI which are used for the actual payload communicated
in the form of OIDs between them.
Similarly FACT provides the base ForCES protocol functionality based
on the requirements [3] and framework [4] with an extensible payload
which can be used to carry the FE/LFB data being defined by the FE
Model [5]. The FACT protocol has a fixed size common header and a
variable size payload field; the payload field carries data that may
contain one of the following
* Control packets that are received by FE through external ports
* Packets that are to be sent by a FE to peer network elements via
external ports
* LFBs details which are represented in FE Modeling draft [5]
* Other configuration information required according to [3], [4]
The FACT protocol is flexible enough to allow any arbitrary query
and configuration of FE capabilities. These queries can be encoded
using simple TLVs, OIDs, or XML. All these encoding schemes have
their own advantages and limitations. FACT protocol is independent
of type of encoding. However, it is recommended to choose one
encoding scheme to enable interoperability. We will defer the
selection of the encoding scheme that will be used for the FACT
payload transmitted on the wire till the FE data model is finalized.
The FACT protocol supports different types of messages, including
synchronous messages like simple request/reply, asynchronous
messages to notify CEs of status changes, and transaction oriented
synchronous messages that support two phase commit operations. These
messages provide basic functionality such as dynamic association,
configuration, topology exchange, packet redirection and command
bundling defined in the Requirements [3]. All FACT messages are 32-
bit aligned.
The FACT protocol follows the basic design principles of simplicity,
reuse of existing mechanisms and enabling easy interoperability. In
this respect, FACT reuses existing transport and security protocols
which are widely available and avoids building such mechanisms into
the protocol which can increase complexity. It also mandates single
transport, security mechanisms and payload encapsulation which help
with enabling easy interoperability. The clean separation of FACT
base protocol and data model also helps with simplicity and
extensibility.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 6]
Internet Draft Forwarding and Control Element protocol November 2003
3.1.Independence from Interconnect
The FACT protocol is independent of the Interconnect Layer. It makes
no assumptions about the interconnect layer and uses interconnect
independent addressing (FE Identifier, CE tag) in its common header.
For non-IP interconnects, such as ATM, an interconnect specific
encapsulation will have to be defined to carry the FACT messages,
similar to the approach used in GSMP [13]. For IP interconnects,
FACT MUST use TCP as the transport protocol [see section 3.2].
For non-IP interconnects, which do not provide reliability, the
interconnect specific encapsulation might consist of an optional
checksum and any other fields to help build reliability. FACT also
provides protocol level Responses/Acks and Sequence Numbers which
can be used to build reliability. However, as a general principle
FACT recommends reusing existing mechanisms such as reliable
transport protocols to provide reliability. For example, in case of
unreliable interconnects, Congestion Manager [17] can be used to
provide congestion control or TIPC [15] which is an existing open
source transport protocol that runs over Layer 2, could be used to
provide reliability.
3.2.Reliability
The FACT protocol MUST use a reliable transport protocol/mechanism,
which will meet all the reliability requirements stated in [Reqs]
Section 7 #6, for carrying all its (control) messages. The use of a
reliable transport simplifies the protocol design considerably. This
also makes the FACT protocol easily deployable in both single hop
and multi-hop scenarios (for multi-hop scenario, [Reqs] mandates the
use of RFC 2914 [12] based transport i.e. TCP/SCTP). In addition,
the congestion control provided by reliable transport protocols such
as TCP/SCTP [10]is important and required to meet the scalability
requirements (hundreds of FEs and thousands of ports), specially in
the case when the FACT control messages and the data traffic being
forwarded between the FEs use the same backplane interconnect.
In case of IP interconnection between NE elements, FACT protocol
MUST use TCP as the transport protocol. TCP meets all the
requirements (no losses, no data corruption, no re-ordering of data)
for the ForCES protocol and also provides congestion control
mechanism which is important to meet the scalability requirement. In
addition, it helps with interoperability since TCP is a well
understood, widely deployed transport protocol. Having a single
transport protocol (for IP interconnection) helps FACT protocol to
work seamlessly in single hop and multihop scenarios; this would be
difficult if different transport mechanisms were used.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 7]
Internet Draft Forwarding and Control Element protocol November 2003
It is also possible to use other reliable transports such as SCTP
[10] or Reliable Multicast [14] as optional transport protocols for
(control channel in) FACT. However in order to promote
interoperability, TCP is mandated in all implementations of FACT
over IP interconnection.
Note: The authors see very limited applicability of Reliable
Multicast as transport for FACT and need to do more research on this
topic.
3.3.Separate Control and Data channels
The ForCES NEs are subject to Denial of Service (DoS) attacks
[Requirements Section 7 #15]. A malicious system in the network can
flood a ForCES NE with bogus control packets such as spurious RIP or
OSPF packets in an attempt to disrupt the operation of and the
communication between the CEs and FEs. In order to protect against
this situation, the FACT protocol uses separate control and data
channels for communication between the CEs and FEs.
The data channel carries the control protocol packets such as RIP,
OSPF messages as outlined in Requirements [3] Section 7 #10, which
are carried in FACT PE Traffic Maintenance messages (section 5.4),
between the CEs and FEs. All the other FACT messages, which are used
for configuration/capability exchanges, event notification, etc, are
carried over the control channel. The data channel is set up only
after the control channel is set up and the capability exchange has
successfully taken place between the FEs and CEs. The CE signals the
FE to establish the data channel at the appropriate time and
provides it with the channel addressing information, such as, port
number in case of TCP (see section 5.3). By default, the data
channel is established on the CE control channel port number +1.
The reliability requirements for the data channel messages are
different from that of the control messages [Reqs] i.e. they don?t
require strict reliability in terms of retransmission, etc. However
congestion control is important for the data channel because in case
of DoS attacks, if an unreliable transport such as UDP is used for
the data traffic, it can more easily overflow the physical
connection, overwhelming the control traffic with congestion. Thus
we need a transport protocol which provides congestion control but
does not necessarily provide full reliability. Datagram Congestion
Control Protocol (DCCP) [11] which is currently being defined, is a
transport protocol which exactly meets this requirement. However
since it is currently not an IETF standard RFC, and the authors are
unaware of any existing implementations, the FACT protocol MUST use
TCP as transport protocol for the data channel (for IP
interconnect). TCP provides the congestion control mechanism
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 8]
Internet Draft Forwarding and Control Element protocol November 2003
required for the data channel and its wide deployment eases
interoperability. In future versions of this draft, if DCCP becomes
an IETF standard RFC with available implementations, the authors
would like to consider using it as the Data channel transport
protocol. Furthermore, for the data channel requirements, DCCP
without mobility, ECN support, partial checksum support, with TCP-
like congestion control would suffice.
The priority bits in the FACT common header used to carry the
network control protocol packets over the data channel in FACT PE
Traffic Maintenance messages can be used to distinguish between
different types of data traffic on the data channel. For example,
OSPF packets (encoded as FACT PE Traffic Maintenance messages) could
be given higher priority than ping packets on the data channel. The
use of separate control and data channels along with rate limiting
mechanisms on the FE provides robustness to the FACT protocol
against DoS attacks.
3.4.Fail-Over Model
The FACT protocol provides CE fail-over functions in order to
support high availability of the network element [3].
The CE-SET (see section 4.1.4) is a list of CEs that reside within a
Network Element (NE) as a cooperating unit. Likewise, the FE-SET is
a list of FEs that reside in an NE as a cooperating unit.
The following are the high-availability mechanisms that are provided
by FACT protocol.
(1) Strong Consistency: In strong consistency mode, the FE sends
all asynchronous notifications/ control protocol packets to the
primary and backup CEs in the CE set. By doing this, the FE
enables both the primary and backup CEs to keep the state
synchronized.
(2) Weak Consistency (Fail-over): In this mode, the FE communicates
directly with the primary CE. If the primary CE fails, the FE
starts communicating with the backup CE. The FACT protocol
specifies that selection of primary and backup CEs be done
during pre-association phase.
In all the above cases, CE (including primary and backup CEs) and
FEs are pre-configured to perform such activities as part of pre-
association phase.
4. FACT Message Overview
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 9]
Internet Draft Forwarding and Control Element protocol November 2003
FACT protocol messages are made up of two parts: the common header,
and the message body or service payload part. This section describes
the details of the common header and payload structure.
4.1. Protocol Message Header structure
FACT protocol Header is fixed size (12 bytes) and contains the
following fields.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| MsgCls|Msg-Type | P| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CE-Tag | FE-Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transaction sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4.1.1. Version
The version field (4-bits) contains the version of the FACT protocol
supported by the implementation. The current supported version is :
value 0x01
Protocol elements implementing the FACT protocol SHOULD provide
backwards compatibility with prior versions of the protocol.
4.1.2. Message Classes and Types
The messages are grouped into classes, with each class consisting
of a set of related message types.
The valid message classes are:
Message Class: 4 bits (unsigned integer)
0 Reserved
1 PE Connection and association (CA) Messages
2 Capabilities Control (CAPCO) Messages
3 PE State Maintenance (PESM) Message
4 PE Traffic Maintenance (PETM) Messages
5 Event Notification (EN) Messages
6 Vendor Specific (VS) Messages
7-15 Reserved by IETF for future use
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 10]
Internet Draft Forwarding and Control Element protocol November 2003
The message types (5 bits) for the defined message classes are as
follows:
Message Type for Connection and Association (CA) Message Class
0 Reserved
1 Join Request
2 Join Response
3 Leave Request
4 Leave Response
5-31 Reserved by IETF for future use
Message Type for Capabilities Control (CAPCO) Message Class
0 Reserved
1 Capability Request
2 Capability Response
3 Configure Request
4 Configure Response
7 Query Request
8 Query Response
9-31 Reserved by IETF for future use
Message Types for PE State Maintenance (PESM) Message Class
0 Reserved
1 PE Up
2 PE Up Ack
3 PE Down
4 PE Down Ack
5 PE Active
6 PE Active ACK
7 PE Inactive
8 PE Inactive ACK
9 Heartbeat
10 Heartbeat Ack
11-31 Reserved by IETF for future use
Message Types for PE Traffic Maintenance (PETM) Message Class
0 Reserved
1 Control Packet Redirect
2 Control Packet Forward
3-31 Reserved by IETF for future use
Message Types for Event Notification (EN) Message Class
0 Reserved
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 11]
Internet Draft Forwarding and Control Element protocol November 2003
1 Event Register
2 Event Register Response
3 Event De-register
4 Event De-register Response
5 Asynchronous FE Event Notification
6-31 Reserved by IETF for future use
Message Types for Vendor Specific Message Class
0 Reserved
1 VS-Data Request
2 VS-Data Response
3-31 Reserved for other vendor specific messages
Vendor specific message types interpretation is beyond the scope of
this protocol.
4.1.3. Length
The Message Length (16-bits ) denotes the length of the message in
octets, including the Message Header. All FACT messages are 32-bit
aligned with padding at the end if needed.
4.1.4. CE Tag
During the pre-association phase, the CEs are configured by the CE-
Manager. In a network element, there may be many CEs; one or more CEs
can be grouped together to form a CE-set. A CE-set is unique in one
network element and is identified by an 8-bit number. To identify a
CE inside a CE-set, the CE identifier is used which is also an 8-bit
field. One of the advantage of grouping such CE and FEs to is to
provide scalability when managing hundreds of FE?s and CE?s [3].
Figure 1 shows the CE-Tag fields, CE-Tag is a 16-bit integer
which has two portions, the higher order 8-bit describes the CE-set
number, and the lower 8-bit describes the CE-identification number.
This field provides interconnect independent identification for the
CEs and is unique for each CE within a NE. This field is mandatory
for all message types.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CE-Set Number | CE-Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 CE-Tag fields
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 12]
Internet Draft Forwarding and Control Element protocol November 2003
4.1.5. FE Identifier
This is a 16-bit field that is used to uniquely identify the FE in a
network element. This field?s purpose is to provide interconnect
independent identification of FEs and is a unique number for each FE
within a NE. This field is present in all type of messages. The
system administrator will have to make sure that the CEs assign
unique FE identifiers to multiple FEs in the system.
Note: The range of values from 0 to 1000 for this field is reserved
to address FE groups. The values of 1001 to 64K will be used as
unique FE identifiers. This allows the CE to address a message to a
group of FEs as well as individual FEs.
4.1.6. Priority Bits
This is a 3-bit field which is used to assign different priorities
to the FACT messages. If this is set to 3 then the message is of
normal priority. Priority value of 7 indicates the highest priority
and value of 0 indicates the lowest priority.
4.1.7. Transaction Sequence Number (TSN)
This 32-bit field uniquely identifies a transaction between the FE
and CE. When an endpoint makes a request it generates a TSN to send
in the request message; the other endpoint copies this same TSN
number in its reply/response message.
4.2. Service Data or Payload Structure
FACT protocol messages consist of the Common Message Header
described in the previous section, followed by zero or more variable
length parameters, as defined by the message type. This constitutes
the Payload or Service Data. All FACT messages are 32-bit aligned.
Examples of the service data are the following:
. LFB configuration
. LFB status and events
. FE capability and topology information
. Control protocol packets
The variable length parameters in the payload are defined in a Tag-
Length-Value (TLV) format as shown below:
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 13]
Internet Draft Forwarding and Control Element protocol November 2003
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Parameter Tag | Parameter Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
\ \
/ Parameter Value /
\ \
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Mandatory parameters MUST be placed before optional parameters in
a message.
Parameter Tag: 16 bits
The Tag field is a 16-bit unique identifier of the type of the
parameter. It takes a value of 0 to 65534. Appendix-1 lists all
used values of the Tag and related messages. Values other than those
defined in specific parameter description are reserved for use by the
IETF.
Parameter Length: 16 bits
The Parameter Length field contains the size of the parameter in
bytes, including the Parameter Tag, Parameter Length, and Parameter
Value fields. The Parameter Length does not include any padding
bytes.
Parameter Value: variable-length
The Parameter Value field contains the actual information to be
transferred in the parameter.
The total length of a parameter (including Tag, Parameter Length and
Value fields) MUST be a multiple of 4 bytes. If the length of the
parameter is not a multiple of 4 bytes, the sender pads the
parameter at the end (i.e., after the Parameter Value field) with
all zero bytes. The length of the padding is NOT included in the
parameter length field. A sender MUST NEVER pad the parameter with
more than 3 bytes. The receiver MUST ignore the padding bytes.
5. FACT Messages
This section defines the messages and their parameter contents.
5.1. Association and Connection (CA) Messages
5.1.1. Join Request
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 14]
Internet Draft Forwarding and Control Element protocol November 2003
After the pre-association phase [section 9], the FEs can join or
leave any CE in a CE-set. A FE uses the Join Request message to
initiate association with a CE in a CE-set. The message contains the
requester?s identity that was configured during pre-association.
After a successful join process, FEs can report their capabilities
to the CE.
At a given point, CEs from one CE set can communicate with a FE. FE
has to know which CE?s requests it can accept. This information is
configured during the pre-association phase, during which a list of
CEs allowed to control the FE is configured in the FE. The FE uses
this CE-list to send the join request. It first tries one of the
CE?s in the list and if it not successful, it tries the next CE in
the list. If all of the CEs in the list are tried without success,
the FE should start over again until Retry timer expires.
The format of the JOIN Request Message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x01) | Length (8) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Address: The IP Address or interconnect dependant unique address
of the FE. The tag value determines whether it is the
IPv4 (tag 0x01)or IPv6 (tag 0x02)or some other type
of address such as Layer 2 address.
5.1.2. Join Response
CE after receiving a join request message performs the following
operations.
(1) Checks the FE association ID (FE identifier) in the
request message and if it equals zero, then the CE
generates a unique identifier for that FE and allocates
resources for its attributes.
(2) If the FE association ID (FE identifier) field is not
zero, then the CE checks up its previously stored
configuration for that FE. If it has any, it will use
that to configure the FE in subsequent messages. This
is warm restart operation.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 15]
Internet Draft Forwarding and Control Element protocol November 2003
(3) If the CE needs to reject the join request for some
reason, it sends a Leave Response Message as indicated
later.
After a successful JOIN operation, the CE should initiate the next
phase of the association establishment process by querying the FE
for its capabilities, its topologies, etc, and then configuring the
FE for the functions it is to perform in the NE.
The format for the JOIN RESPONSE message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x03) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FE identifier | FE Behavior |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Heartbeat Interval |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Association Expiry Timer |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FE Behavior Expiry Timer (optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
FE Behavior: This defines the FE behavior when all the CEs are
down. A value of 1 indicates that the FE should
continue forwarding packets, a value of 2 indicates
that FE should stop forwarding packets when CEs are
down.
Heartbeat Interval: This gives the time interval for the
heartbeat messages sent from the CE to the FE in
milliseconds.
Association Expiry Timer: This gives the timer value in
milliseconds after which if the FE does not receive
any heartbeat messages from the CE it should consider
the association with the CE to have expired (CE
down). This can be a multiple of the heartbeat
interval.
FE Behavior Expiry Timer: This is an optional timer value, which
applies in case the FE behavior is to continue
forwarding packets when CEs are down. This value
indicates the time in seconds for which the FE should
continue forwarding packets without associations with
any CEs.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 16]
Internet Draft Forwarding and Control Element protocol November 2003
5.1.3. Leave Request
The FE can leave the association by sending a Leave request to the
CE. The CE?s upon receiving such request releases the associated
resources assigned for the FE and sends a Leave response. The CE can
also send a Leave request to the FE to leave the association.
The LEAVE Request message contains the following parameters:
Reason
Info String (optional)
The format for the LEAVE Request Message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x04) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reason |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x05) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> INFO String* <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The reason parameter indicates the reason the PE is leaving the NE.
Valid values are as follows:
Value Description
0x1 Management Inhibit (Manual Removal)
0x2 Invalid CE group (for leave response only)
0x3 Max number of FEs reached (for leave response only)
0x4 Wrong FE ID assigned (for leave response only)
The optional INFO String parameter can be any meaningful 8-bit
character string, up to 255 characters in length. This may be used
for debugging purposes.
5.1.4. Leave Response
When a FE is leaving the NE, the CE generates a Leave response to
the leaving FE. Also, the FE generates a Leave response in response
to a Leave request from the CE.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 17]
Internet Draft Forwarding and Control Element protocol November 2003
If a CE needs to reject a join request from a FE for some reason, it
sends a Leave Response Message to the FE as well (Refer to Section
5.1.2).
The LEAVE Response message contains the following parameters:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x04) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reason |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The format for the Reason parameters is same as in LEAVE Request
message (See 5.1.3).
5.2.Capabilities Control (CAPCO) Messages
This is the next phase of the JOIN process. After a FE has been
successfully accepted into a CE-SET, the CE initiates the next phase
of the association establishment process by querying the FE for its
capabilities, its topology, etc, and then configuring the FE for the
functions it is to perform in the NE.
5.2.1. Capability Request
This message is used to request the FE to report its Capabilities.
The message consists of the FACT header with the message type set to
Capability Request.
The FE may have one or more LFBs on the forwarding plane like meter,
shaper, egress port etc. CE may configure or query these components
and their status at any time. In order to do this, CE needs to know
the LFBs placement and sequence in the forwarding data path. FE
Model [5] describes the FE Capabilities.
5.2.2. Capability Response
This is used by the FE to report its capabilities to the CE as per
CE?s request. This message structure might change depending on the
FE Model [5].
The format for the Capability Response is as follows:
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 18]
Internet Draft Forwarding and Control Element protocol November 2003
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x06) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> FE Caps <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The format for the FE capability will be defined in [5].
5.2.3.Configure Request
This message is used by the CE to configure the FE. The FE consists
of LFBs that could be configured to achieve a desired behavior of
the FE in the network. Some configurable attributes of the FE
include LFB parameters. Examples of such attributes are:
. Port attributes (direction, bandwidth,..)
. routing table functions
. High Touch functions
. Off-Load functions
. Filter functions
This message can be used by CE to configure both LFB and FE
attributes. One example of FE attribute is the LFB topology. This
message can thus be to configure the LFB topology on the FEs that
support dynamic reconfiguration of LFB topology. The Tag value in
the message will indicate whether it is being used to configure FE
or LFB attributes.
The CE might also have received a command (either through CLI or
through SNMP etc) to setup some tunnels or path or some
configuration. This may sometimes involve configuring more than one
FE. For example, packets may arrive through one FE and egress the NE
through another. In this situation, the CE has to configure both
FEs? LFBs. If one of the configuration operation fails, the CE has
to perform rollback operation and issue a command failure
notification to the management station or CLI or SNMP etc.
This operation is called 2-phase commit. To perform this, CE sends
series of commands to each FE with command bundling bit set. Each FE
after receiving the command will have to save the current
configuration and check whether it can program the requested
configuration. A status message should be sent back to the CE. Once
CE receives all the status messages, it can then send an execute
command with same transaction sequence number, signaling the FEs to
now switch to the new configuration.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 19]
Internet Draft Forwarding and Control Element protocol November 2003
Command bundling refers to the ability to send an ordered set of
commands to the FE. FACT supports command bundling via multiple TLVs
in its payload as described in section 4.2. Each command is
formatted as a TLV structure shown below, and multiple commands are
sent to the FE in a single Configure Request message.
The format of the Configure Request message (for LFB attributes) is
as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x07) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| C-Operation| C-Command | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Command Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
C-Operation:
FEs and CEs may engage in two-phase commit operation. This
field provides the stage of such transactions.
0x00 - Command is single operation
0x01 ¡ This command is a two-phase operation, FE needs to
save the previous state if rollback operation may be
performed later by FE.
0x10 ¡ Rollback to the previous state.
0x11 ¡ Execute and complete the command.
During this operation the unique TSN value in the common header is
the same and is used to identify the transaction. Note that TSNs are
only unique in combination with a CE identifier, that is, two
separate CEs using the same TSN are considered different
transactions.
Configuration-command:
This field defines the command type. The valid values for this
field are
0 Reserved
1 NULL
2 Add
3 Update
4 Delete
5 Delete All (Flush or Purge)
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 20]
Internet Draft Forwarding and Control Element protocol November 2003
LFB Handle:
This field defines the LFB handle for which this command is
being issued.
Command Data:
This is the variable length configuration data for the LFB.
This can be encapsulated using TLV or OID or XML formats.
Note: The Configure Request message format for FE attributes will be
slightly different and will be defined in conjunction with the FE
Model [5].
5.2.4. Configure Response
This is sent by the FE to CE to acknowledge configuration request by
CE. The tag value will indicate whether the response is for LFB or
FE attributes.
The format of the Configure Response (for LFB attributes) is as
follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x08) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| C-Operation| C-Command | G.Result |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Command Result <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The response contains the information from the command, but with
?Global Result? field filled in to indicate success or failure.
Result Value Meaning
CONFIG-OK 0x0 Success
CONFIG-BADCMD 0x1 Bad or unsupported configuration command.
CONFIG-BADPRM 0x2 Bad configuration parameter.
LFB Handle:
This field defines the LFB handle or identifier for which this
command is being issued.
Command Result:
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 21]
Internet Draft Forwarding and Control Element protocol November 2003
This is the variable length configuration result for the LFB.
This can be encapsulated using TLV or OID or XML formats.
Note: The Configure Response message format for FE attributes will
be slightly different and will be defined in conjunction with the FE
Model [5].
5.2.5.Query Request
CE may be interested in querying the properties of FEs, LFBs or
collecting statistical information from FE, including that of
its LFBs. In this case, a CE sends a Query Request Message to a
FE and expects a Query Response Message from it. This message
is also used to query the inter-FE topology or the LFB topology
(Intra-FE) on the FE. This information is useful to the CE in
order to configure the FEs correctly.
The format for the Query Request Message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0A) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type of Info |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Query Specific Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: :
| LFB HandleN |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
There can be multiple LFB IDs in one message. The number of IDs is
derived from the Length field. In case of querying FE properties or
topology, the LFB Handle field is not needed.
Valid values of the Info Type are:
LFB-PROPS (0x1) ¡ LFB Properties
FE-PROPS (0x2) ¡ FE Properties
LFB-TOPO (0x3) ¡ LFB topology (Intra-FE)
FE-TOPO (0x4) ¡ Inter-FE topology
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 22]
Internet Draft Forwarding and Control Element protocol November 2003
Note: By properties we mean both read-only attributes such as
statistics and read-write attributes.
Query Specific Data: This is query specific data, which can be in
encapsulated as TLV or OID or XML.
5.2.6.Query Response
After receiving a Query Request Message, a FE replies with the Query
Response Message. The format of the Query Response Message is as
follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0B) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Info Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Query Response Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
For Info Type=LFB Properties, the Query Response Data is as follows
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> LFB Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The LFB data is encapsulated similar to the query specific data in
the respective format.
For Info Type=Inter-FE Topology, the Query Response Data is as
follows
It consists of the list of FEs (FE Addresses) directly connected to
the communicating FE. This structure might change i.e. some more
information might need to be added depending on the input from FE Model
[5].
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| #of neighbor FEs |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FE1 ADDRESS |
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 23]
Internet Draft Forwarding and Control Element protocol November 2003
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FE2 ADDRESS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
. .
. .
| FEN ADDRESS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
For Info Type=LFB Topology, the Query Response Data is as follows
It consists of a list of LFB connectivity information i.e. {LFB
Handle, LFB Type, Downstream LFB Count, Downstream LFB Handles}.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DownStream LFB Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DownStream LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
. .
| DownStream LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
. .
| DownStream LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
5.2.7.Error TLV
The Error TLV is used to notify the CE or FE of an error associated
with an incoming synchronous Request message. For example, the
message might be unexpected, given the current state, or a parameter
value might be invalid. This Error TLV can be part of Join Response,
Leave Response, Capability Response, Configure Response, Query
Response, etc.
The format of the Error TLV is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 24]
Internet Draft Forwarding and Control Element protocol November 2003
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x16) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Error Code |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The Error Code parameter indicates the reason for the error.
Possible error parameter values include:
INV-PROT (0x01) - Invalid Protocol Version
INV-ASSOC (0x02) ¡ Invalid Association ID
BAD-MSGCLS (0x03) ¡ Unsupported message class
BAD-MSGTYP (0x04) ¡ Unsupported message type
UNEX-MSG (0x05) ¡ Unexpected message
PROT-ERROR (0x06) ¡ Protocol Error
TWOPHASE-ERR (0x07) ¡ Could not complete two-phase command
COMM-FAIL (0x08) ¡ Communication lost between CE and FE
INVALID-TRAF-MODE (0x09) ¡ Unsupported Traffic Mode
5.3. PE State Maintenance Messages
5.3.1.Protocol Element UP (PEUP)
The PE UP message is sent by a FE to indicate to the CE that it is
UP (in-service) and ready to be used. It consists of the FACT header
with the PE UP message type. This message is typically sent after
the capability and topology discovery of the FE has been completed
by the CE (see section 7.1).
5.3.2. Protocol Element Up Acknowledge (PEUP-ACK)
The PEUP Acknowledgement message is used to acknowledge a PE-Up
message received from the FE. It consists of the FACT header with
the PEUP-ACK message type.
5.3.3. Protocol Element Active (PEACT)
The PE ACT message is sent by the CE to ask the FE to go ACTIVE and
start handling traffic. The FE must be UP and must have sent the
PEUP message to the CE. This message is used to trigger the
establishment of the data channel between the FE and CE. Note: The
data channel should be established before the FE starts handling
traffic.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 25]
Internet Draft Forwarding and Control Element protocol November 2003
The PE ACT message contains the following parameters
Traffic Mode Type (Optional)
Data Channel Info (Optional)
The format for the PE ACT message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0C) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| reserved | Traffic Mode Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0D) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Data Channel Info <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The Traffic Mode Type parameter identifies the failover mode of the
FE within an NE. The valid values for Type are shown in the
following table:
Value Description
0x1 Over-ride-strong-consistency
0x2 Over-ride-weak-consistency
0x3 Other
Within a particular association, only one Traffic Mode Type can be
used. The Over-ride value indicates that the CE is over-riding the
current failover mode of the FE. For example, the primary CE can
change the failover mode from Strong-consistency to weak-consistency
in order to update some software on the Standby CE.
The optional Data Channel Info contains information relevant to
establish the data channel such as the CE port number that should be
used to establish the channel.
5.3.4. Protocol Element Active Acknowledgement (PEACT-ACK)
The PE ACT Acknowledgement message is used to acknowledge a PE-
Active message received from the CE. It consists of the FACT header
with the PEACT-ACK message type. This message is sent after the data
channel has been established with the CE.
5.3.5. Protocol Element Inactive (PEINACT)
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 26]
Internet Draft Forwarding and Control Element protocol November 2003
The PE INACT message is sent by the CE to ask its FE to go IN ACTIVE
and stop handling all data traffic. After receiving this message,
the FE shuts down the Data channel with the CE. The FE will remain
IN ACTIVE till it receives a PE ACTIVE message from the CE.
The PE INACT message contains the following parameters
Traffic Mode Type (Optional)
The format for the CE/FE Inactive message parameters is as shown for
FE/CE Active Message(See 5.3.3)
5.3.6. Protocol Element Inactive Acknowledgement (PEINACT-ACK)
The PE INACT Acknowledgement message is used to acknowledge a PE-
Inactive message received from the CE. It consists of the FACT
header with the PEINACT-ACK message type.
5.3.7. Protocol Element Down (PEDOWN)
Due to failure or maintenance operation, FE can send this PE DOWN
message to its primary CE. Upon receiving this request, primary CE
may reassign the responsibility to other FE?s (if possible).
The PE DOWN message contains the following parameters:
Reason - (Mandatory) reason for going down
The format for the PE DOWN Message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0E) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reason |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The reason parameter indicates the reason the PE is leaving the NE.
Valid values are as follows:
Value Description
0x1 Management Inhibit (Manual Removal)
0x2 Device Fault
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 27]
Internet Draft Forwarding and Control Element protocol November 2003
5.3.8. Protocol Element Down Ack (PEDOWN-ACK)
The PEDOWN-ACK message is used by the primary CE to acknowledge the
PEDN message sent by the outgoing PE. It consists of the FACT header
with the PEDOWN-ACK message type.
5.3.9. Heartbeat
CE periodically polls each FE to ensure that it is operational. A CE
starts generating these messages after the PE Active message has
been sent to the FE. The timers for these messages are configurable
during pre-configuration and can be different for the active and
standby CEs. The heartbeat interval for a standby CE can be much
larger than that of the active CE.
An optional Heartbeat Data parameter may be sent in the heart beat
message. Its contents are defined by the sending node and are opaque
to the receiving FE, which simply echoes the contents back to the
sending CE via the HB-ACK message (see below). Examples of values
encoded in the Heartbeat Data field could include a Heartbeat
Sequence Number or Timestamp. The receiver of a Heartbeat message
does not process this field as it is only of significance to the
sender. The receiver MUST respond with a Heartbeat Acknowledgement
message.
The format for the Heartbeat Message is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x0F) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Heartbeat Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
5.3.10. Heartbeat Acknowledgement (HB-ACK)
After verifying the CE-Tag, FE simply echo?s the original heartbeat
message as a Heartbeat ACK message to the CE.
5.4. PE Traffic Maintenance (PETM) Messages
These messages are sent over the data channel. The data channel is
established after the PE Active message is sent from the CE to FE.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 28]
Internet Draft Forwarding and Control Element protocol November 2003
5.4.1. Control Packet Redirect to CE
When a Router receives both control and data packets through a
physical port, any of the following scenarios may occur:
(a) Forwarding blade receives IP packet that are destined for it;
these packets are forwarded to the CE by the FE.
(b) Forwarding blade receives IP packets that may be routing
protocol packets or packets which cannot be processed by the
stack in the line card. Such packets have to be forwarded to
the control plane by the FE.
The format of the Control Packet Redirect is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x10) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle on which packet arrived |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x11) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Control Packet <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
LFB Handle:
This is used to specify the interface on which the packet arrived.
Control Packet:
The control packet (including the IP/Layer 3 header and L2
header) that the network element received through a particular
interface, which needs to be redirected to the CE.
5.4.2. Control Packet Forwarding to FE
CE may generate a packet and want FE to forward that packet through
a particular or multiple egress port(s). Examples of such packets
are routing protocol updates, discoveries, etc.
Before generating such request, CE has to know the FE?s LFBs and the
list of available port and the configuration status.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 29]
Internet Draft Forwarding and Control Element protocol November 2003
| Tag (0x10) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle through which to forward packet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x11) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Packet to be forwarded <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
LFB Handle : Packet to be forwarded is preceded by the LFB handle
for the egress port through which the packet must be forwarded by the
FE.
To forward to multiple egress ports (such as multicast), the format
is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x12) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Start LFB ID1 through which to forward packet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> .. <
| End LFB IDn through which to forward packet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x11) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Packet to be forwarded <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
5.5. Event Notification Messages
Various events in the data path can be monitored for by the FE and
reported to the CE. The CE must first inform the FE which of these
events it is interested in through a registration process.
5.5.1. Event Register
This is sent by the CE to the FE to request that FE notify the CE
when the indicated events occur on the FE. The format of the message
is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 30]
Internet Draft Forwarding and Control Element protocol November 2003
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x13) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Event Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Event Specific Data <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Event Type is the type of the event to report. Valid values are as
follows:
NO-EVENTS (0x0) ¡ No Events reported
PKT-EVENTS (0x1) ¡ Packet events (redirection)
PKT-MEVENTS (0x2) ¡ Packet mirroring events
FE-EVENTS (0x3) ¡ FE events
SS-USAGE (0x4) ¡ System Specific events
LC-EVENTS (0x5) ¡ LFB events
ALL-EVENTS (0x6) ¡ All events reported
Event Specific Data: This is the variable length event specific
data, which can be encapsulated using TLV or OID or XML formats. For
example, this could be used to specify what packets (e.g. RIP/OSPF
packets) should be redirected or mirrored to the CE. This may be a
filter to specify the packets.
5.5.2. Event Register Response
This is used by the FE to acknowledge CE?s event registration
request. The format of this message is as follows.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x18) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Result |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Result Value Meaning
0x0 Success
0x0 Failure
5.5.3. Event De-Register
This is sent by the CE to the FE to indicate that it no longer is
interested in receiving notification for the particular events
indicated in message. The format of this message is same as in the
Event Register Request. After receiving this message the FE SHOULD
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 31]
Internet Draft Forwarding and Control Element protocol November 2003
not deliver those particular events to the CE and MUST send an Event
De-Register Acknowledgement to the CE.
5.5.4. Event De-Register Response
The FE sends this message to the CE to acknowledge the CE?s request
not get any more notification for events indicated in message. The
format for this message is same as in the Event Register Response.
After a FE transmits this message to a CE it MUST not deliver the
particular events until a new event register is received and
acknowledged.
5.5.5. Asynchronous FE Event Notification
This is used to report asynchronous events occurring in the FE.
These could be overall FE errors, or LFB specific events. The
message contains the following:
Event Type ¡ same as that defined for Event Register
Event ID
LFB Handle
Diagnostic Info - this describes the event in more detail.
The format of the Async FE Event is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x15) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Event Type | Event ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LFB Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x14) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Diagnostic Info <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Valid values of the Event ID are as follows:
SYS-SPECIFIC (0x1) ¡ CPU usage more than 75% of capacity
FE-ERROR (0x2) ¡ FE in non-catastrophic error state
FE-RES-CHANGE (0x3) ¡ FE resource change.
LFB-SPECIFIC (0x4) ¡ LFB Specific Events.
PRI-CE-DOWN (0x5) ¡ Primary CE has gone DOWN.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 32]
Internet Draft Forwarding and Control Element protocol November 2003
5.6.Vendor Specific Messages
This allows extensions to the FE functions so that new, currently
unknown FE functionality (outside of those already specified) can be
expressed. These messages will be transported transparently by the
FACT protocol and are provided to meet Requirements Section 6.5,
point#5. Interpretation of the transported messages will be left
solely to the application layers sitting on FACT in the CE and FE.
5.6.1. Vendor Specific Data (VS-DATA Request)
Separated CEs or FEs may use this message to pass any information
that is not to be consumed by FACT to each other. This message is
not destined outside the involved CEs or FEs either. Application
layers sitting on top of the FACT protocol layer can exchange
information with this message between PEs.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Tag (0x17) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> Data to be transported <
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
As this message is opaque to FACT, the content is vendor-specific.
FACT does not parse the content of this message.
5.6.2. Vendor Specific Data Response (VS-Data Response)
This is used by the PE that receives the VS-DATA Request message to
send a response back to the sender.
The format of this message is same as for VS-DATA Request, it
contains vendor specific data which is transparent to FACT protocol.
6. Procedures for FACT Protocol
6.1.CE and FE State Maintenance
FACT layer on the CE needs to maintain the states of the FEs it
communicates with. Likewise, the FACT layer on the FE needs to
maintain the states of the CEs the FE communicates with. The state
of the (logical) NE also needs to be maintained. Since the NE is
comprised of CEs and FEs, the NE state will be determined by the
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 33]
Internet Draft Forwarding and Control Element protocol November 2003
states of the contained FE and CE elements.
Figure 2 below shows a hypothetical NE with its set of CEs and
FEs
|--------------------------------------------|
| NE |
| |------| |-------| |
| | CE1 | | CE2 | |
| |(active)| |(standby)| |
| |------| |-------| |
| ^ ^ ^ ^ |
| | | | | |
| | \-----| |-------/ | |
| | |-------|-/ |---------/ |
| v v v v |
| |------| |-----| |-----| |-----| |
| | FE1 | | FE2 | | FE3 | | FE4 | |
| |(act) |--->|(act)| |(stby)| |(stby)| |
| |------| |-----| |-----| |-----| |
| ^ | |
| | | |
|---|------------|---------------------------|
| v
Figure 2. Showing logical NE and its components
6.1.1.CE and FE States
The state of each configured FE and CE is maintained by the FACT
layer. The state of each CE or FE element can change due to the
following events:
. Reception of management messages by CE.
. Reception of management messages by FE.
. Reception of control messages by FE from CE.
. Loss of communication between CE and FE (e.g. due to faults).
The CE and FE state transition diagram is shown in figure 3.
+-------------+
+---------------------| |
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 34]
Internet Draft Forwarding and Control Element protocol November 2003
| Alternate +-------| PE-ACTIVE |
| PE | +-------------+
| Takeover | ^ |
| | CE/FE | | CE/FE
| | Active | | Inactive
| | | v
| | +-------------+
| | | |
| +------>| PE-INACT |
| +-------------+
| ^ |
CE/FE Down/ | CE/FE | | CE/FE Down /
CE-FE COMM | Up | | CE-FE COMM FAIL
FAIL | | v
| +-------------+
+-------------------->| |
| PE-DOWN |
+-------------+
Figure 3. CE and FE State Transition Diagram
The possible states of a protocol element (CE or FE) are:
PE-DOWN: CE or FE is unavailable for service and/or the related
CE-FE association is down. Initially, all CEs and FEs will be in
this state. A CE or FE in this state should not be sent any traffic
messages.
PE-INACTIVE: The CE or FE is available for service and the related
CE-FE FACT association is up, but application traffic is stopped
(the CE or FE could be in a standby state for example). In this
state, the CE or FE involved can be sent management, control, and
non-traffic related messages.
PE-ACTIVE: The CE or FE is available, and actively carrying
application traffic.
6.2.State Maintenance Procedures
Before the establishment of a CE-FE association, the CE must be in-
service and active but the FE is Down. Local management (CE Manager
or FE Manager) can be used to effect appropriate state transitions
of CEs and FEs.
6.2.1.Protocol Element Up
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 35]
Internet Draft Forwarding and Control Element protocol November 2003
After an FE has successfully established an association with a CE,
the FE sends a PE-UP message to indicate to the CE that it has
finished all its internal configuration and is available.
When the CE gets the PE-UP message, and the FE is not locked out for
local management reasons, FACT at the CE will mark the FE as UP but
?Inactive?. The CE responds with a PE-UP Ack message in
acknowledgement. If for any reason the CE cannot respond with a PE-
UP, it will respond with a PE-DOWN Ack message with an appropriate
reason parameter.
The CE can also generate the PE-UP message. The last ACTIVE CE may
have gone DOWN after establishing an association with a FE. In this
case, the NE would first transition into the PENDING state for a
duration of T(r ), and then to DOWN state. The first CE that
transitions to UP state will send a PE-UP to the FE to notify it of
its status, assuming the link between them is up. The FE will
acknowledge with a PE-UP Ack.
If the source PE does not receive a response from the target PE, or
if a PE-DOWN Ack is received, source PE MAY resend PE-UP message
until it receives a PE-UP Ack from the target. The default behavior
is NO retransmission.
6.2.2.Protocol Element Down
The FE will send a PE-DOWN to the CE when the FE is to be removed
from the list of FEs in an NE that it is a member of, that is
eligible to receive application traffic or management messages.
FACT at the CE marks the FE as ?Down? and returns a PE-DOWN Ack
message to the FE if one of the following events occur:
- a PE-DOWN message is received from the FE
- another state message is received from an FE but the FE is
locked out by management for some reason.
The CE sends a PE-DOWN Ack message in response this message. If the
FE does not receive a response from the CE, the FE MAY send PE-DOWN
messages until it receives a PE-DOWN Ack message from the CE or the
association goes down. The default behavior is NO retransmission.
The CE may also send a PE-DOWN messaged to the FE. This occurs when
the CE is about to be removed from service, and it is the ACTIVE CE.
On getting this notification, the FE will respond with a PE-DOWN
Ack, and stop sending any more messages to the out-going CE.
The whole mechanism allows for a graceful removal of CEs or FEs.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 36]
Internet Draft Forwarding and Control Element protocol November 2003
6.2.3.Protocol Element ACTIVE
Any time after the CE has sent a PE-UP Ack to the FE, the CE can
send a PE-Active (PEACT) to the FE, to activate the FE to start
processing traffic.
When a PEACT message is received, the FE responds with a PEACT Ack
message, after which it starts handling traffic messages. The FE
establishes the Data channel with the CE after this message. The FE
must wait for the PEACT message from the CE and establish the data
channel before handling traffic data. The CE only sends the PEACT
message if it intends to transition the FE to ACTIVE state.
6.2.4. Protocol Element Inactive
Any time after the CE has sent a PE-Active to the FE, the CE can
send a PE-Inactive (PEINACT) to the FE, to command the FE to stop
processing traffic.
When a PEINACT message is received, the FE responds with a PEINACT
Ack message, after which it stops handling traffic messages. The FE
shuts down the Data channel with the CE after it receives this
message. The FE must wait for another PEACT message from the CE
before starting handling traffic again. The CE only sends the
PEINACT message if it intends to transition the FE to INACTIVE
state.
7. Example Scenarios
7.1.Establishment of Association
The associations among CEs and FEs are initiated via Join request
and response messages. If a join request is granted by the CE, a
join response message is replied to the request message. If a join
request is denied, a Leave response message is replied to the join
request message. If CEs and FEs are operating in an insecure
environment then the security association has to be established
between them before any FACT messages can be exchanged (see section
8).
This is followed by capability query, topology query. When the FE is
ready to start forwarding data traffic, it sends a PE-UP message to
the CE. The CE responds with PE-UP Ack. According to the
configuration, the CE sends a PE-ACTIVE to inform the FE to go
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 37]
Internet Draft Forwarding and Control Element protocol November 2003
active and start forwarding data traffic. The FE establishes the
data channel with the CE, acknowledges it with a PE-ACTIVE Ack and
then starts forwarding traffic. At this point the association
establishment is complete. The sequences of messages are illustrated
in the Figure below.
FE CE
| |
| Join REQ |
1 |---------------------->|
| |
| Join RESP |
2 |<----------------------|
| |
| CAPABILITY Request |
3 |<----------------------|
| |
| CAPABILITY Response |
4 |---------------------->|
| |
| T. QUERY Request |
5 |<----------------------|
| |
| T. QUERY Response |
6 |---------------------->|
| |
| PE UP |
7 |---------------------->|
| |
| PE UP ACK |
8 |<----------------------|
| |
| PE ACT |
9|<----------------------|
| |
| Data channel Estb |
10|---------------------->|
| |
| PE ACT ACK |
11|---------------------->|
Figure 4: Association Establishment messages between CE and FE
7.2.Steady State Communication
Once the CE and FEs establish their association and exchange
initial configuration information, they enter a phase of steady
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 38]
Internet Draft Forwarding and Control Element protocol November 2003
state communication, with the following example messages
exchanging.
FE CE
| |
| Heart Beat |
1 |<----------------------|
| |
| Heart Beat ACK |
2 |---------------------->|
| |
| Configure Request |
3 |<----------------------|
| |
| Configure Response |
4 |---------------------->|
| |
| Aysnc Event Notice |
5 |---------------------->|
| |
| Query Request |
6 |<----------------------|
| |
| Query Response |
7 |---------------------->|
| |
|Control Packet Redirect|(over data channel)
8 |---------------------->|
| |
Figure 5: Steady State communication between CE and FE
When transferring forwarding information to FE, the CE uses the
configure request message with a 16-bit length field indicating
the number of bytes of configuration information. If the CE has
a very large amount of database that exceeds 64K bytes in
length, it should send multiple configure request messages to
complete the configuration.
7.3.CE Fail-over Scenarios
As mentioned before, there are two basic modes of high-availability
or Failover support provided by FACT protocol. This is configured
during the pre-association phase [as described in section 9]. For
both cases, the FE must establish association using FACT protocol
[as described in section 7.1] with both the primary and standby CEs
in the CE set. This association establishment includes the security
associations described in section 8, also capability and topology
discovery. This helps with fast failover since the FE avoids re-
establishment of CE-FE association during failover.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 39]
Internet Draft Forwarding and Control Element protocol November 2003
For strong consistency, the FE establishes the control and data
channels with both CEs and forwards all asynchronous events and
protocol control packets such as RIP, OSPF packets to both CEs. But
only the primary CE configures and controls the FE, the standby CE
uses the information provided by the FE to keep its state
synchronized with the primary CE. When FE detects failure of primary
CE it informs the standby CE using the asynchronous Event message.
The standby CE can also obtain that information using some CE-to-CE
protocol. In case of failure of the primary CE, the standby CE takes
over the control of the FE. Note that in case of strong consistency,
CE-to-CE protocol may not be needed (or will be minimally used) to
keep the state in the primary and standby CEs synchronized.
FE CE Primary CE Standby
| | |
| Asso Estb(Caps, topo) | |
1 |<--------------------->| |
| | |
| Asso Estb(Caps, topo exchange) |
2 |<----------------------|------------------->|
| | |
| data + control | |
3 |<--------------------->| |
| | |
| data + control|(HeartBeats only) |
4 |-----------------------|------------------->|
| | |
| FAILURE |
| |
| PRI-CE-DOWN |
5 |------------------------------------------->|
| |
| data + control |
6 |------------------------------------------->|
Figure 6: CE Failover for strong consistency mode
For weak consistency, the FE establishes the control channel with
both CEs but the data channel with only the primary CE. The only
communication with the standby CE is the heartbeat exchange. The
standby CE keeps it state synchronized using some CE-to-CE protocol.
When FE detects failure of primary CE it informs the standby CE
using the asynchronous Event message. In case of failure of the
primary CE, the FE establishes the data channel with the standby CE
which then takes over the control of the FE.
Note that in both failover modes the FE does not need to store any
state in order to synchronize the CEs.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 40]
Internet Draft Forwarding and Control Element protocol November 2003
FE CE Primary CE Standby
| | |
| Asso Estb(Caps, topo)| |
1 |<--------------------->| |
| | |
| Asso Estb(Caps, topo exchange) |
2 |<----------------------|------------------->|
| | |
| data + control | |
3 |<--------------------->| |
| | |
| control|(HeartBeats only) |
4 |-----------------------|------------------->|
| | |
| FAILURE |
| |
| PRI-CE-DOWN |
5 |------------------------------------------->|
| |
| data + control |
6 |------------------------------------------->|
Figure 7: CE Failover for weak consistency mode
8. Security Considerations
If the CE or FE are in a single box and network operator is running
under a secured environment then it is up to the network
administrator to turn off all the security functions. This is
configured during the pre-association phase of the protocol.
Whether FE and CE are in a single box or multiple-hop, rate limiter
mechanism should be in place to defend against the CPU bound and
bandwidth (network) bound DoS attacks. We recommend this for all
security mechanisms.
When the CEs, FEs or FACT endpoints are running over IP networks or
in an insecure environment, FACT MUST use TLS [8] to provide
security. The security association between the CEs and FEs MUST be
established before any FACT association establishment messages are
exchanged between the CEs and FEs.
8.1.TLS Usage with FACT
This section is applicable for CE or FE endpoints that use FACT with
TLS [9][8] to secure communication.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 41]
Internet Draft Forwarding and Control Element protocol November 2003
Since CE is master and FEs are slaves, the FEs are TLS clients and
CEs are TLS server. FACT endpoints that implement TLS MUST perform
mutual authentication during TLS session establishment process. CE
must request certificate from FE and FE needs to pass the requested
information.
We recommend ?TLS-RSA-with-AES-128-CBC-SHA? cipher suite, but CE or
FE may negotiate other TLS cipher suites. TLS must be used for all
control channel messages. TLS is optional for the data channel since
data channel packets are not encrypted externally to the NE.
FACT uses TLS to provide security when the NE is in an insecure
environment. This is because IPsec provides less flexibility when
configuring trust anchors since it is transparent to the application
and use of Port identifiers is prohibited within IKE Phase 1. This
provides restriction for IPsec to configure trust anchors for each
application separately and policy configuration is common for all
applications.
9. Architecture support for FACT protocol
Pre-association phase is used to configure certain key attributes.
FE-Manager and CE-Manager are responsible for providing that
information.
9.1.Configurable parameters
The following are the currently identified configurable parameters
that can be done through FE-Manager and CE-Manager for FE and CE?s
respectively
(1) Fail over configuration (Strong, Weak consistency)
(2) Number of CE to which it has to communicate
(3) Maximum number of CE
(4) Timer for health check
(5) Maximum numbers of FEs that each CE can support in a NE
10. IANA Considerations
FACT protocol needs to have a two well-defined port numbers, which
needs to be assigned by IANA.
11. References
11.1.Normative References
1. S. Bradner, "The Internet Standards Process -Revision 3", RFC 2026,
October 1996.
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 42]
Internet Draft Forwarding and Control Element protocol November 2003
2. S. Bradner, "Keywords for use in RFCs to Indicate Requirement
Levels", RFC2119 (BCP), IETF, March 1997.
3. Khosravi, et. al., ?Requirements for Separation of IP Control and
Forwarding?, work in progress, July 2003, <draft-ietf-forces-
requirements-10.txt>,IETF.
4. L. Yang, et. al, ? ForCES Architectural Framework?,work in
progress?, August 2003,<draft-ietf-forces-framework-08.txt>
5. L. Yang, et. al, ? ForCES Forwarding Element Functional Model?,
work in progress?, August 2003,<draft-ietf-forces-model-00.txt>
11.2.Informative References
6. Morneault, K,. Rengasami, S,. Kalla, M., and G. Sidebottom, "ISDN
Q.921-User Adaptation Layer", RFC 3057, February 2001
7. J. Moy, "OSPF Version 2", RFC 2328, April 1998
8. Dierks, T., Allen, C., Treese, W., Karlton, P., Freier, A. and P.
Kocher, "The TLS Protocol Version 1.0", RFC 2246, January 1999.
9. Jungmaier, A., Rescorla, E. and M. Tuexen, "Transport Layer
Security over Stream Control Transmission Protocol", RFC 3436,
December 2002.
10. R. Stewart, et al, ?Stream Control Transmission Protocol
(SCTP)?, RFC 2960, October 2000.
11. E. Kohler, M. Handley, S. Floyd, J. Padhye, ?Datagram
Congestion Control Protocol (DCCP)?, draft-ietf-dccp-spec-04.txt,
June 2003.
12. Floyd, S., ?Congestion Control Principles?, RFC 2914, September
2000.
13. A. Doria, F. Hellstrand, K. Sundell, T. Worster, ?General
Switch Management Protocol (GSMP) V3?, RFC 3292, June 2002.
14. M. Handley, Floyd, S., et al ?The Reliable Multicast Design
Space for Bulk Transfer?, RFC 2887, August 2000.
15. http://tipc.sourceforge.net/
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 43]
Internet Draft Forwarding and Control Element protocol November 2003
16. R. Presuhn, et al ?Simple Network Management Protocol (SNMP)
Version 2?, RFC 3416, December 2002.
17. H. Balakrishnan, et al ?The Congestion Manager?, RFC 3124, June
2001.
12. Acknowledgments
We would like to thank Man Li, Nokia Research Center for her
suggestions and comments. We would also like to acknowledge Jing
Qian from Alcatel for her help with Appendix 1 and Joel Halpern,
David Putzolu for their comments and suggestions on the protocol.
13. Authors' Addresses
Ram Gopal
Nokia Research Center
5, Wayside Road,
Burlington, MA 01803
Phone: 1-781-993-3685
Email: ram.gopal@nokia.com
Alex Audu
Alcatel R&I
1000 Coit Road
Plano, TX 75075
Phone: 1-972-477-7809
Email: alex.audu@alcatel.com
Chaoping Wu
Azanda Network Devices
250 Santa Ana Court
Sunnyvale, CA 94085
Phone: 1-408-720-3117
Email: chaoping_wu@yahoo.com
Hormuzd Khosravi
Intel
2111 NE 25th Avenue
Hillsboro, OR 97124
Phone: 1-503-264-0334
Email: hormuzd.m.khosravi@intel.com
Appendix-1: Tag (Hex) Values Used in FACT Messages
+-----+--------------------+---------------------------------------+
|Tag | Meaning | Messages |
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 44]
Internet Draft Forwarding and Control Element protocol November 2003
+-----+--------------------+---------------------------------------+
|0001 |IPv4 Join Address | Join Request |
+-----+--------------------+---------------------------------------+
|0002 |IPv6 Join Address | Join Request |
+-----+--------------------+---------------------------------------+
|0003 |Join Configuration | Join Response |
+-----+--------------------+---------------------------------------+
|0004 |Leave Reason | Leave Request/Response |
+-----+--------------------+---------------------------------------+
|0005 |Info String | Leave Req/Resp |
+-----+--------------------+---------------------------------------+
|0006 |FE Capability | Capability Response |
+-----+--------------------+---------------------------------------+
|0007 |LFB Config Command | Configure Request |
+-----+--------------------+---------------------------------------+
|0008 |LFB Config Result | Configure Response |
+-----+--------------------+---------------------------------------+
|0009 |FE Config Command | Configure Request |
+-----+--------------------+---------------------------------------+
|0019 |FE Config Result | Configure Response |
+-----+--------------------+---------------------------------------+
|000A |Query Data | Query Request |
+-----+--------------------+---------------------------------------+
|000B |Query Resp Data | Query Response |
+-----+--------------------+---------------------------------------+
|000C |Traffic Mode | PE (IN)ACT/ACK |
+-----+--------------------+---------------------------------------+
|000D |Data Channel Info | PE ACT |
+-----+--------------------+---------------------------------------+
|000E |Down Reason | PE DOWN/ACK |
+-----+--------------------+---------------------------------------+
|000F |Heartbeat Data | Heartbeat/Heartbeat ACK |
+-----+--------------------+---------------------------------------+
|0010 |LFB ID | CP Redirect, CP Forwarding |
+-----+--------------------+---------------------------------------+
|0011 |Control Packet | CP Redirect, CP Forwarding |
+-----+--------------------+---------------------------------------+
|0012 |Log. Comp ID list | CP Forwarding |
+-----+--------------------+---------------------------------------+
|0013 |Event Data | Event (De-)Register Req |
+-----+--------------------+---------------------------------------+
|0014 | Diagnostic Info | Asynchronous EE Event Notification |
+-----+--------------------+---------------------------------------+
|0015 |Event-Handle ID | Asynchronous EE Event Notification |
+-----+--------------------+---------------------------------------+
|0016 |Error Code | Join, leave, Cap, Response msgs |
+-----+--------------------+---------------------------------------+
|0017 |Vendor Specific Data| VS-DATA Request/Resp |
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 45]
Internet Draft Forwarding and Control Element protocol November 2003
+-----+--------------------+---------------------------------------+
|0018 |Result | Event (De-)Register Resp |
+-----+--------------------+---------------------------------------+
Appendix-2: Interfaces To Local Management(CE/FE Manager)
As part of any normal protocol operation, management interface either CLI or
EMS or NMS or other suitable entity would like to send commands to either FE or CE.
This section identifies minimal command set that may be required for such
operation. This may be implemented by each vendor in various ways and is beyond the
scope of ForCES protocol. This section describes the minimal message and how FE and
CE may use FACT to inform the local management operation to each other.
M-PE-STATUS
The status of CEs and FEs are stored in and tracked by FACT. This primitive
is used to request, confirm, and indicate the status of a PE (FE or CE) to local
management.
M-ASSOCIATION-STATUS
This is used to request and indicate the status of the association between a CE and
an FE.
M-ERROR
This is used to indicate an error with a received FACT message to FE or CE Manager.
M-PE-UP
This primitive can be used by FE or CE Manager to request that a FE or CE be
restored into in-service (UP) state by FACT. This could be triggered by a RESTORE
request from the CLI (Command Line Interface) into FE or CE Manager.
(CLI)-----Restore(COLD)--->( FE or CE Manager)----- M-PE-UP(COLD)--->(FACT)
FACT can also use this primitive to indicate or acknowledge to FE or CE Manager
that a FE or CE is UP (with a M-PE-UP.indicate and M-PE-UP.confirm primitives
respectively). Valid parameters to M-PE-UP.req are:
COLD ¡ initialize all attributes of PE during restore process.
WARM ¡ use previous attributes in memory to restore PE (assumes those
Attributes have not been lost).
CONFIG - Restore PE with new configuration attributes.
M-PE-DOWN
This can be used by FE or CE Manager to request that a PE be taken to the DOWN
State. It could be triggered for example, by CLI sending a Remove request to
the local management layer.
(CLI) ----Remove(BOOT)--->(FE or CE Manager)----- M-PE-DOWN(BOOT) --->(FACT)
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 46]
Internet Draft Forwarding and Control Element protocol November 2003
FACT can in turn indicate a PE-DOWN event or confirm the DOWN request with a
M-PE-DOWN.ind or M-PE-DOWN.con respectively. Valid parameters to M-PE-DOWN.req
are:
NO-BOOT ¡ previous (static) contents in memory are preserved.
BOOT ¡ all previous contents in memory are zero?d out.
CONFIG ¡ previous configuration information between FE and CE are lost
and new ones are being established.
M-PE-ACTIVE
This is used by FACT to inform FE or CE Manager that FE or CE has gone ACTIVE.
M-PE-INACTIVE
This is used by FACT to inform FE or CE Manager that FE or CE has gone INACTIVE.
Appendix-3: NE States
It may be necessary to track the state of the NE itself. Since the
NE consists of distributed CEs and FEs, the state of the NE will be
dependent on the states of its CEs and FEs. The state of the NE is
maintained by FACT in both FE and CE.
The state of an NE can be changed due to events including:
. CE or FE state transitions
. Recovery timer triggers
The possible states of a NE are as follows:
NE-DOWN: The network element is not available for service. This
implies all related CEs and FEs are in the PE-DOWN state. Initially,
the NE will be in this state.
NE-INACTIVE: The network element is available but no application
traffic is active. Here, one or more protocol elements (CE or FE)
are in the PE-INACTIVE state, but none in the PE-ACTIVE state. Also,
the recovery timer is not running, or has expired. This may be the
state of standby NE if redundancy is provided at logical NE level.
NE-ACTIVE: The network element is available and it is carrying
application traffic. This implies that at least, one CE-FE
communicating pair is in PE-ACTIVE state.
NE-PENDING: An active CE or FE has transitioned into inactive or
down state, and it was the last remaining active CE or FE in the NE.
A recovery timer T(r), will be started, and the source FE or CE will
queue up messages meant for the inactive target. If another target
CE or FE becomes active (depending on which went inactive), before
T(r) expires, the queued up messages are directed to the newly
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 47]
Internet Draft Forwarding and Control Element protocol November 2003
active CE or FE, and T(r) timer is cancelled. In this case, NE will
move back to the NE-ACTIVE state. However, if T(r) expires before an
alternate CE or FE becomes active, the queued up messages are
discarded, and the NE will move to NE-DOWN state.
+----------+ one PE goes ACTIVE +-------------+
| |------------------------>| |
| NE-INACT | | NE-ACTIVE |
| | | |
| |< | |
+----------+ \ +-------------+
^ | \ Tr fires; ^ |
| | \ at least one | |
| | \ PE is UP | |
| | \ | |
| | \ | |
| | \ | |
one PE | | \ one PE | | Last ACTIVE PE
goes | | all PEs \------\ goes to | | goes INACT
to | | go DOWN \ ACTIVE | | or DOWN
INACT | | \ | | (start Tr timer)
| | \ | |
| | \ | |
| | \ | |
| v \ | v
+----------+ \ +-------------+
| | -| |
| NE-DOWN | | NE-PENDING |
| | | (queueing) |
| |<------------------------| |
+----------+ Tr Expiry and all +-------------+
PEs in DOWN state
Tr = Recovery Timer
Figure 8: NE State Transition Diagram
Gopal,Audu,Wu,Khosravi Expires May 2004 [Page 48]