Internet Working Group J.-L. Le Roux
Internet Draft J.-Y. Mazeas
Proposed Category: Standard Track France Telecom
Expires: January 2006 J.-P. Vasseur
S. Boutros
Cisco Systems
D. Papadimitriou
Alcatel
July 2005
Procedure to handle (G)MPLS-TE control plane saturation
draft-leroux-ccamp-ctrl-saturation-01.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Le Roux, et al. Standard Track - Expires January 2006 [Page 1]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
Abstract
This document defines extensions to RSVP-TE (Resource Reservation
Protocol-Traffic Engineering) and IGP (IS-IS and OSPF), in order to
notify about control plane resources saturation, when an LSR runs out
of control plane resources available to support any additional LSP.
Such notification may serve as trigger for the impacted Head-end LSR
to take appropriate actions.
Table of Contents
1. Terminology.................................................2
2. Introduction................................................3
3. RSVP-TE Saturation..........................................4
4. Signalling extensions.......................................4
4.1. New RSVP Error Code.........................................4
4.2. Mode of operations..........................................5
4.2.1. Procedures for a saturated LSR..............................5
4.2.2. Procedures for a Head-End LSR...............................5
5. Routing extensions..........................................5
5.1. Encoding of the Saturation TLV..............................5
5.2. Elements of procedure.......................................6
5.2.1. Procedures for a Saturated LSR..............................7
5.2.2. Procedures for a Head-End LSR...............................7
6. Inter-Provider considerations...............................7
7. Security Considerations.....................................8
8. Acknowledgements............................................8
9. References..................................................8
9.1. Normative references........................................8
9.2. Informative references......................................8
10. Authors' Address:...........................................9
11. Intellectual Property Statement.............................9
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119.
1. Terminology
LSR: Label Switching Router
TE-LSP: MPLS Traffic Engineering Label Switched Path
Head-End LSR: Head of a TE-LSP
Tail-End LSR: Tail of a TE-LSP
PSB: RSVP Path State Block
Le Roux, et al. Standard Track - Expires July 2005 [Page 2]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
RSB: RSVP Reservation State Block
RSVP-TE Saturation: State of an LSR that cannot accept any new TE-LSP
due to configured thresholds, or control plane limitations
ASBR: Autonomous System Border Router
2. Introduction
Many service providers have deployed RSVP-TE [RFC3209] to setup TE-
LSPs and achieve Traffic Engineering objectives.
In some circumstances, MPLS-TE deployments in large networks may
require maintaining a high number of TE-LSPs on transit LSRs, which
may lead to the instantiation of a high number of RSVP states and
hence consume a large amount of LSR control plane resources (e.g.
memory, CPU).
Control plane capacities of an LSR are of course not infinite and
there may be some circumstances whereby an LSR may run out of a
specific control plane resource such as memory or CPU available for
the RSVP process; such resource shortage may lead to the inability
for the LSR to handle additional TE-LSPs. For instance, the LSR has
no longer enough memory to instantiate a new RSVP state block (PSB,
RSB [RFC2209]).
Furthermore, there may be circumstances whereby the operator may want
to limit, by configuration, the maximum number of RSVP-TE sessions
that can be accepted and maintained by an LSR at the Ingress, Transit
or Egress position.
This document defines a particular RSVP-TE state (referred to as the
"Saturated" state) whereby the LSR cannot handle any additional TE-
LSP, either because the maximum number of TE-LSPs allowed is reached
or because there is no longer enough control plane resources (CPU,
memory,à) allocated to the RSVP process to instantiate and maintain a
new session state block.
There may be cases, particularly in large scale RSVP-TE deployments,
where an LSR receives a Path message for a new LSP, while it is under
the Saturated state. In such situation, the LSR should simply reject
the LSP setup, and notify the head-end LSR of its control plane
saturation state.
For that purpose, this document specifies:
- A RSVP-TE extension allowing a saturated LSR to reject any new LSP
and notify (using a new Error code and a set of sub-codes carried
in a Path Error message) the head-end LSR about its saturation;
- IGP extensions (that complement the RSVP-TE extension) in order
for a LSR to advertise its current saturation status (saturated or
Le Roux, et al. Standard Track - Expires July 2005 [Page 3]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
not). This allows other head-end LSRs avoiding the set up of new
TE LSP through saturated nodes and also discovering that a given
node is no longer saturated.
These routing and signalling notifications are complementary. They
may serve as trigger for the Head-End LSRs to take appropriate
actions.
3. RSVP-TE Saturation
A RSVP-TE engine is usually designed such that the maximum number of
LSPs it supports equals the total number of LSP with a resource
allocation corresponding to the min LSP bandwidth (set e.g. to 2
Mbps) configured in the context of its admission control mechanism.
However, soft-provisioning techniques (e.g. LSP established with no
bandwidth reservation) are challenging this rule of thumb since the
number of supported LSPs needs now to take not only data plane
resources allocation into account but also control plane resources.
An RSVP node enters the Saturated state when it runs out of specific
control plane resource such as the memory or CPU (globally or for the
RSVP process) or above a pre-configured threshold or when a pre-
configured maximum number of LSPs allowed is reached. Criteria to
trigger such saturated state are local to the LSR and outside of the
scope of this document.
Note that it is recommended to introduce some hysteresis for
saturation state transition, so as to avoid oscillations.
For instance if the number of TE-LSPs is bounded (by configuration),
two thresholds should be configured: An upper-threshold and a lower-
threshold with upper-threshold > lower-threshold. An LSR enters the
saturated state when the number of TE-LSPs reaches the upper-
threshold and leaves the saturated state when it goes down the lower-
threshold.
4. Signalling extensions
4.1. New RSVP Error Code
This document specifies a new Error Code (suggested value
to be confirmed by IANA) for the RSVP ERROR_SPEC object:
26 Control Plane Saturation
The following defines error values sub-codes for the new Control
Plane Saturation Error Code:
1 Saturation unspecified
2 Memory saturation
3 CPU saturation
4 Max number of LSPs reached
100-255 Reserved
Le Roux, et al. Standard Track - Expires July 2005 [Page 4]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
Procedures are detailed in section 4.2 below.
4.2. Mode of operations
4.2.1. Procedures for a saturated LSR
On receipt of a Path message for a new LSP, a saturated LSR compliant
with this document SHOULD reject the LSP setup and send back a
PathErr Message with the Path_State_Removed flag set in the
ERROR_SPEC object, with the Error Code "Saturation" and optionally an
Error sub-code value mentioning the reasons for the saturation.
The address carried within the ERROR_SPEC object SHOULD be set to the
TE router id of the saturated node. The IF_ID ERROR_SPEC object MAY
be used in case saturation can be determined as coming from a given
adjacency.
To avoid inability of the saturated node to generate PathErr
messages, it is expected that the locally pre-configured threshold on
control plane resources is such that it allows generation of such
messages.
4.2.2. Procedures for a Head-End LSR
On receipt of a PathErr message with the Error Code "Saturation" and
with the Path_State_Removed flag not set, the Head-End LSR SHOULD
send a PathTear message for the rejected LSP.
5. Routing extensions
It is desirable to augment the signaling extensions by specific link
state routing ([ISIS] and [OSPF]) extensions that would allow an LSR
to advertise the state of its RSVP process (saturated or not). This
information could then be taken into account by all LSRs (and not
only LSRs on the path of a rejected LSPs), in order to avoid a
saturated node when computing the path of a new TE-LSP, and also to
automatically discover when a node is no longer saturated.
A new TLV is defined for OSPF and ISIS, named the ôSaturation TLVö,
to be included in the Router Information LSA for OSPF [OSPF-CAP] and
in the CAPABILITY TLV for ISIS [ISIS-CAP].
The TLV structure consists of a fixed 8 bit flags field. Currently 4
flags are defined, to indicate if the LSR is saturated or not as well
as the reasons for the saturation.
5.1. Encoding of the Saturation TLV
This document defines a new TLV (named the Saturation TLV) allowing
an LSR advertising if its control plane is saturated or not, where,
- With OSPF, the Saturation TLV is a TLV of the Router Information
LSA defined in [OSPF-CAP]. The TLV type is TBD (To be assigned
Le Roux, et al. Standard Track - Expires July 2005 [Page 5]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
by IANA).
- With ISIS, the Saturation TLV is a sub-TLV of the ISIS
CAPABILITY TLV defined in [ISIS-CAP]. The sub-TLV type is TBD
(To be assigned by IANA).
All relevant generic TLV encoding rules (including TLV format,
padding and alignment, as well as IEEE floating point format
encoding) defined in [OSPF] and [ISIS] are applicable to this new
sub-TLV.
The Saturation TLV is a series of 8 bit flags. Its format is
illustrated below:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S|C|M|N| Res |
+-+-+-+-+-+-+-+-+
Four bits are currently defined:
-S bit: When set this indicates that the node is saturated and
cannot support any new TE-LSP. When not set this
indicates that the node is not saturated and can support
new TE-LSPs.
-C bit: When set this indicates that the saturation is due to
CPU overload.
-M bit: When set this indicates that the saturation is due to
memory shortage .
-N bit: When set this indicates that the maximum number of LSPs
allowed is reached.
C, M or N are not exclusive. They can be set only if S is set.
If S is set and other flags are cleared this means that the reason
for the saturation is unspecified.
Note that new flags may be defined in the future.
The absence of the saturation TLV means that the saturation status is
unspecified.
5.2. Elements of procedure
A router SHOULD originate a new IS-IS LSP/OSPF LSA whenever a
saturation state change occurs or whenever required by the
regular IS-IS/OSPF procedure (LSP/LSA refresh).
It is expected that a proper implementation will support dampening
algorithms so as to dampen IGP flooding, thus preserving the IGP
scalability.
Le Roux, et al. Standard Track - Expires July 2005 [Page 6]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
The flooding scope of this saturation state advertisement SHOULD be
limited to a single IGP area. This implies that this TLV SHOULD be
carried within an OSPF type 10 Router Information LSAs or within an
ISIS CAPIBILITY TLV with the S flag cleared.
Note that a saturation satus change MAY trigger CSPF computation, but
MUST not trigger normal SPF computation.
5.2.1. Procedures for a Saturated LSR
Once an LSR enters the saturation state, the conditions of which are
implementation specific, it SHOULD originate LSPs/LSAs with the S bit
of the Saturation TLV set and potentially with one or more "reason"
bits set.
Once a LSR leaves the saturation state, the conditions of which are
implementation specific, it SHOULD originate LSPs/LSAs with all bits
of the Saturation TLV cleared.
5.2.2. Procedures for a Head-End LSR
A head-end LSR computing a path of a TE-LSP should avoid saturated
nodes.
Note also that when a head-end LSR detects that a node on the path of
an already established TE-LSP, becomes saturated, it should not
reroute this TE-LSP.
Note that when a head-End LSR detects that an LSR is no longer
saturated it may re-use this LSR for establishing new TE LSPs, and
may try to reoptimize some TE-LSPs, should the path along the no-
longer saturated LSR be desirable.
This procedure may be delayed, potentially using some LSP setup
jittering between head-end LSRs, in order to avoid a signalling storm
that may again saturate the node, that is, to avoid TE-LSP routing
oscillations.
Note that the procedures discussed above are local implementation
issues.
6. Inter-Provider considerations
For confidentially purpose in an inter-provider MPLS-TE context (see
[INTER-AS-REQ]), a provider may desire to hide to other providers, a
saturation that could occur in its own network. For that purpose an
ASBR may modify the RSVP error code/sub-code before forwarding the
PathErr message to an upstream provider. For instance, if a provider
wants to hide the reason for the saturation, it should update the
error sub-code to 1. If the provider wants to entirely hide the
saturation, it may change the error code. Note that this is a local
policy-based decision.
Le Roux, et al. Standard Track - Expires July 2005 [Page 7]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
7. Security Considerations
This document does not raise any new security issue.
8. Acknowledgements
We would like to thank Thomas Morin, Bruno Decraene, Adrian Farrel,
Arthi Ayyangar, Muthurajah Sivabalan, Rudy Figaro, David Ward, Reshad
Rahman, and Stefano Previdi for the really useful comments and
suggestions.
9. References
9.1. Normative references
[RFC] Bradner, S., 'Key words for use in RFCs to indicate
requirements levels', RFC 2119, March 1997.
[BCP79] Bradner, S., Ed., 'Intellectual Property Rights in IETF
Technology', BCP 79, RFC 3979, March 2005.
[RSVP] Braden, R., Zhang, L., Berson, S., Herzog, S. and S. Jamin,
'Resource ReSerVation Protocol (RSVP) -- Version 1, Functional
Specification', RFC 2205, September 1997.
[RFC2209] Braden, R., Zhang, L., 'Resource ReSerVation Protocol
(RSVP) -- Version 1 Message Processing Rules', RFC2209, September
1997.
[RSVP-TE] Awduche, D., Berger, L., Gan, D., Li, T.,
Srinivasan, V. and G. Swallow, 'RSVP-TE: Extensions to RSVP for LSP
Tunnels', RFC 3209, December 2001.
[ISIS] "Intermediate System to Intermediate System Intra-Domain
Routing Exchange Protocol " ISO 10589.
[OSPF] Moy, J., "OSPF Version 2", RFC 2328, April 1998.
[ISIS-CAP] Vasseur, J.P., Aggarwal, R., Shen, N. et al. "IS-IS
extensions for advertising router information", draft-ietf-isis-
caps-03.txt, work in progress.
[OSPF-CAP] Lindem, A., Shen, N., Aggarwal, R., Shaffer, S., Vasseur,
J.P., "Extensions to OSPF for advertising Optional Router
Capabilities", draft-ietf-ospf-cap-07.txt, work in progress.
9.2. Informative references
[INTER-AS-REQ] Zhang, R., Vasseur, J.P., "MPLS Inter-AS Traffic
Engineering requirements", draft-ietf-tewg-interas-mpls-te-req-
09.txt, work in progress
Le Roux, et al. Standard Track - Expires July 2005 [Page 8]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
10. Authors' Address:
Jean-Louis Le Roux
France Telecom
2, avenue Pierre-Marzin
22307 Lannion Cedex
France
jeanlouis.leroux@francetelecom.com
Jean-Yves Mazeas
France Telecom
2, avenue Pierre-Marzin
22307 Lannion Cedex
France
jeanyves.mazeas@francetelecom.com
Jean-Philippe Vasseur
CISCO Systems, Inc.
300 Beaver Brook
Boxborough, MA 01719
USA
Email: jpv@cisco.com
Sami Boutros
Cisco Systems
2000 Innovation Drive, Kanata,
Ontario, Canada K2K 3E8
sboutros@cisco.com
Dimitri Papadimitriou
Alcatel
Francis Wellensplein 1,
B-2018 Antwerpen, Belgium
Email: dimitri.papdimitriou@alcatel.be
11. Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
Le Roux, et al. Standard Track - Expires July 2005 [Page 9]
Internet Draft draft-leroux-ccamp-ctrl-saturation-01.txt July 2005
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2005). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Le Roux, et al. Standard Track - Expires July 2005 [Page 10]