Transport Area Working Group                                  B. Briscoe
Internet-Draft                                                        BT
Intended status: Standards Track                          March 24, 2009
Expires: September 25, 2009

             Tunnelling of Explicit Congestion Notification

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at

   This Internet-Draft will expire on September 25, 2009.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.


   This document redefines how the explicit congestion notification
   (ECN) field of the IP header should be constructed on entry to and
   exit from any IP in IP tunnel.  On encapsulation it brings all IP in

Briscoe                Expires September 25, 2009               [Page 1]

Internet-Draft               ECN Tunnelling                   March 2009

   IP tunnels (v4 or v6) into line with the way RFC4301 IPsec tunnels
   now construct the ECN field.  On decapsulation it redefines how the
   ECN field in the forwarded IP header should be calculated for two
   previously invalid combinations of incoming inner and outer headers,
   in order that these combinations may be usefully employed in future
   standards actions.  It includes a thorough analysis of the reasoning
   for these changes and the implications.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  6
     1.1.  Scope  . . . . . . . . . . . . . . . . . . . . . . . . . .  8
     1.2.  Document Roadmap . . . . . . . . . . . . . . . . . . . . .  9
   2.  Requirements Language  . . . . . . . . . . . . . . . . . . . .  9
   3.  Summary of Pre-Existing RFCs . . . . . . . . . . . . . . . . . 10
     3.1.  Encapsulation at Tunnel Ingress  . . . . . . . . . . . . . 10
     3.2.  Decapsulation at Tunnel Egress . . . . . . . . . . . . . . 12
   4.  New ECN Tunnelling Rules . . . . . . . . . . . . . . . . . . . 13
     4.1.  Default Tunnel Ingress Behaviour . . . . . . . . . . . . . 14
     4.2.  Default Tunnel Egress Behaviour  . . . . . . . . . . . . . 14
     4.3.  Design Principles for Future Non-Default Schemes . . . . . 16
   5.  Backward Compatibility . . . . . . . . . . . . . . . . . . . . 17
     5.1.  Non-Issues Upgrading Any Tunnel Decapsulation  . . . . . . 18
     5.2.  Non-Issues for RFC4301 IPsec Encapsulation . . . . . . . . 18
     5.3.  Upgrading Other IP in IP Tunnel Encapsulators  . . . . . . 19
   6.  Changes from Earlier RFCs  . . . . . . . . . . . . . . . . . . 20
   7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 21
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 21
   9.  Conclusions  . . . . . . . . . . . . . . . . . . . . . . . . . 23
   10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24
   11. Comments Solicited . . . . . . . . . . . . . . . . . . . . . . 25
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 25
     12.2. Informative References . . . . . . . . . . . . . . . . . . 25
   Appendix A.  Design Constraints  . . . . . . . . . . . . . . . . . 28
     A.1.  Security Constraints . . . . . . . . . . . . . . . . . . . 28
     A.2.  Control Constraints  . . . . . . . . . . . . . . . . . . . 30
     A.3.  Management Constraints . . . . . . . . . . . . . . . . . . 31
   Appendix B.  Relative Placement of Tunnelling and In-Path Load
                Regulation  . . . . . . . . . . . . . . . . . . . . . 32
     B.1.  Identifiers and In-Path Load Regulators  . . . . . . . . . 32
     B.2.  Non-Dependence of Tunnelling on In-path Load Regulation  . 33
     B.3.  Dependence of In-Path Load Regulation on Tunnelling  . . . 34
   Appendix C.  Contribution to Congestion across a Tunnel  . . . . . 37
   Appendix D.  Why Not Propagating ECT(1) on Decapsulation
                Impedes PCN . . . . . . . . . . . . . . . . . . . . . 38
     D.1.  Alternative Ways to Introduce the New Decapsulation

Briscoe                Expires September 25, 2009               [Page 2]

Internet-Draft               ECN Tunnelling                   March 2009

           Rules  . . . . . . . . . . . . . . . . . . . . . . . . . . 39
   Appendix E.  Why Resetting CE on Encapsulation Impedes PCN . . . . 40
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 40

Briscoe                Expires September 25, 2009               [Page 3]

Internet-Draft               ECN Tunnelling                   March 2009

Changes from previous drafts (to be removed by the RFC Editor)

   Full text differences between IETF draft versions are available at
   <>, and
   between earlier individual draft versions at

   From ietf-01 to ietf-02 (current):

      *  Scope reduced from any encapsulation of an IP packet to solely
         IP in IP tunnelled encapsulation.  Consequently changed title
         and removed whole section 'Design Guidelines for New
         Encapsulations of Congestion Notification' (to be included in a
         future companion informational document).

      *  Included a new normative decapsulation rule for ECT(0) inner
         and ECT(1) outer that had previously only been outlined in the
         non-normative appendix 'Comprehensive Decapsulation Rules'.

         +  The Introduction has been completely re-written to motivate
            this change to decapsulation along with the existing change
            to encapsulation.

         +  The tentative text in the appendix that first proposed this
            change has been split between normative standards text in
            Section 4 and Appendix D, which explains specifically why
            this change would streamline PCN.  New text on the logic of
            the resulting decap rules added.

      *  If inner/outer is Not-ECT/ECT(0), changed decapsulation to
         propagate Not-ECT rather than drop the packet; and added

      *  Considerably restructured:

         +  "Design Constraints" analysis moved to an appendix
            (Appendix A);

         +  Added Section 3 to summarise relevant existing RFCs;

         +  Structured Section 4 and Section 5 into subsections.

         +  Added tables to sections on old and new rules, for precision
            and comparison.

         +  Moved Section 4.3 on Design Principles to the end of the
            section specifying the new default normative tunnelling

Briscoe                Expires September 25, 2009               [Page 4]

Internet-Draft               ECN Tunnelling                   March 2009

            behaviour.  Rewritten and shifted text on identifiers and
            in-path load regulators to Appendix B.1.

   From ietf-00 to ietf-01:

      *  Identified two additional alarm states in the decapsulation
         rules (Figure 4) if ECT(X) in outer and inner contradict each

      *  Altered Comprehensive Decapsulation Rules (Appendix D) so that
         ECT(0) in the outer no longer overrides ECT(1) in the inner.
         Used the term 'Comprehensive' instead of 'Ideal'.  And
         considerably updated the text in this appendix.

      *  Added Appendix D.1 to weigh up the various ways the
         Comprehensive Decapsulation Rules might be introduced.  This
         replaces the previous contradictory statements saying complex
         backwards compatibility interactions would be introduced while
         also saying there would be no backwards compatibility issues.

      *  Updated references.

   From briscoe-01 to ietf-00:

      *  Re-wrote Appendix C giving much simpler technique to measure
         contribution to congestion across a tunnel.

      *  Added discussion of backward compatibility of the ideal
         decapsulation scheme in Appendix D

      *  Updated references.  Minor corrections & clarifications

   From -00 to -01:

      *  Related everything conceptually to the uniform and pipe models
         of RFC2983 on Diffserv Tunnels, and completely removed the
         dependence of tunnelling behaviour on the presence of any in-
         path load regulation by using the [1 - Before] [2 - Outer]
         function placement concepts from RFC2983;

      *  Added specific cases where the existing standards limit new
         proposals, particularly Appendix E;

      *  Added sub-structure to Introduction (Need for Rationalisation,
         Roadmap), added new Introductory subsection on "Scope" and
         improved clarity;

Briscoe                Expires September 25, 2009               [Page 5]

Internet-Draft               ECN Tunnelling                   March 2009

      *  Added Design Guidelines for New Encapsulations of Congestion

      *  Considerably clarified the Backward Compatibility section
         (Section 5);

      *  Considerably extended the Security Considerations section
         (Section 8);

      *  Summarised the primary rationale much better in the

      *  Added numerous extra acknowledgements;

      *  Added Appendix E.  "Why resetting CE on encapsulation harms
         PCN", Appendix C.  "Contribution to Congestion across a Tunnel"
         and Appendix D.  "Ideal Decapsulation Rules";

      *  Re-wrote Appendix B.2, explaining how tunnel encapsulation no
         longer depends on in-path load-regulation (changed title from
         "In-path Load Regulation" to "Non-Dependence of Tunnelling on
         In-path Load Regulation"), but explained how an in-path load
         regulation function must be carefully placed with respect to
         tunnel encapsulation (in a new sub-section entitled "Dependence
         of In-Path Load Regulation on Tunnelling").

1.  Introduction

   This document redefines how the explicit congestion notification
   (ECN) field [RFC3168] in the IP header should be constructed for all
   IP in IP tunnelling.  Previously, tunnel endpoints blocked visibility
   of transitions of the ECN field except the minimum necessary to allow
   the basic ECN mechanism to work.  Three main change are defined, one
   on entry to and two on exit from any IP in IP tunnel.  The newly
   specified behaviours make all transitions to the ECN field visible
   across tunnel end-points, so tunnels no longer restrict new uses of
   the ECN field that were not envisaged when ECN was first designed.

   The immediate motivation for opening up the ECN behaviour of tunnels
   is because otherwise they impede the introduction of pre-congestion
   notification (PCN [I-D.ietf-pcn-marking-behaviour]) in networks with
   tunnels (Appendix E explains why).  But these changes are not just
   intended to ease the introduction of PCN; care has been taken to
   ensure the resulting ECN tunnelling behaviour is simple and generic
   for other potential future uses.

   Given this is a change to behaviour at 'the neck of the hourglass',

Briscoe                Expires September 25, 2009               [Page 6]

Internet-Draft               ECN Tunnelling                   March 2009

   an extensive analysis of the trade-offs between control, management
   and security constraints has been conducted in order to minimise
   unexpected side-effects both now and in the future.  Care has also
   been taken to ensure the changes are fully backwards compatible with
   all previous tunnelling behaviours.

   The ECN protocol allows a forwarding element to notify the onset of
   congestion of its resources without having to drop packets.  Instead
   it can explicitly mark a proportion of packets by setting the
   congestion experienced (CE) codepoint in the 2-bit ECN field in the
   IP header (see Table 1 for a recap of the ECN codepoints).

     | Binary codepoint | Codepoint name | Meaning                   |
     |        00        | Not-ECT        | Not ECN-capable transport |
     |        01        | ECT(1)         | ECN-capable transport     |
     |        10        | ECT(0)         | ECN-capable transport     |
     |        11        | CE             | Congestion experienced    |

     Table 1: Recap of Codepoints of the ECN Field [RFC3168] in the IP

   The outer header of an IP packet can encapsulate one (or more)
   additional IP headers tunnelled within it.  A forwarding element that
   is using ECN to signify congestion will only mark the outer IP header
   that is immediately visible to it.  When a tunnel decapsulator later
   removes this outer header, it must follow rules to ensure the marking
   is propagated into the IP header being forwarded onwards, otherwise
   congestion notifications will disappear into a black hole leading to
   potential congestion collapse.

   The rules for constructing the ECN field to be forwarded after tunnel
   decapsulation ensure this happens, but they are not wholly
   straightforward, and neither are the rules for encapsulating one IP
   header in another on entry to a tunnel.  The factor that has
   introduced most complication at both ends of a tunnel has been the
   possibility that the ECN field might be used as a covert channel to
   compromise the integrity of an IPsec tunnel.

   A common use for IPsec is to create a secure tunnel between two
   secure sites across the public Internet.  A field like ECN that can
   change as it traverses the Internet cannot be covered by IPsec's
   integrity mechanisms.  Therefore, the ECN field might be toggled
   (with two bits per packet) to communicate between a secure site and
   someone on the public Internet--a covert channel.

Briscoe                Expires September 25, 2009               [Page 7]

Internet-Draft               ECN Tunnelling                   March 2009

   Over the years covert channel restrictions have been added to the
   design of ECN (with consequent backward compatibility complications).
   However the latest IPsec architecture [RFC4301] takes the view that
   simplicity is more important than closing off the covert channel
   threat, which it deems manageable given its bandwidth is limited to
   two bits per packet.

   As a result, an unfortunate sequence of standards actions has left us
   with nearly the worst of all possible combinations of outcomes,
   despite the best endeavours of everyone concerned.  The new IPsec
   architecture [RFC4301] only updates the earlier specification of ECN
   tunnelling behaviour [RFC3168] for the case of IPsec tunnels.  For
   the case of non-IPsec tunnels the earlier RFC3168 specification still
   applies.  At the time RFC3168 was standardised, covert channels
   through the ECN field were restricted, whether or not IPsec was being
   used.  The perverse position now is that non-IPsec tunnels restrict
   covert channels, while IPsec tunnels don't.

   Actually, this statement needs some qualification.  IPsec tunnels
   only don't restrict the ECN covert channel at the ingress.  At the
   tunnel egress, the presumption that the ECN covert channel should be
   restricted has not been removed from any tunnelling specifications,
   whether IPsec or not.

   Now that these historic 2-bit covert channel constraints are impeding
   the introduction of PCN, this specification is designed to remove
   them and at the same time streamline the whole ECN behaviour for the

1.1.  Scope

   This document only concerns wire protocol processing at tunnel
   endpoints and makes no changes or recommendations concerning
   algorithms for congestion marking or congestion response.

   This document specifies common, default ECN field processing at
   encapsulation and decapsulation for any IP in IP tunnelling.  It
   applies irrespective of whether IPv4 or IPv6 is used for either of
   the inner and outer headers.  It applies to all Diffserv per-hop
   behaviours (PHBs), unless stated otherwise in the specification of a
   PHB.  It is intended to be a good trade off between somewhat
   conflicting security, control and management requirements.

   Nonetheless, if necessary, an alternate congestion encapsulation
   behaviour can be introduced as part of the definition of an alternate
   congestion marking scheme used by a specific Diffserv PHB (see S.5 of
   [RFC3168] and [RFC4774]).  When designing such new encapsulation
   schemes, the principles in Section 4.3 should be followed as closely

Briscoe                Expires September 25, 2009               [Page 8]

Internet-Draft               ECN Tunnelling                   March 2009

   as possible.  There is no requirement for a PHB to state anything
   about ECN tunnelling behaviour if the new default behaviour is

   [RFC2983] is a comprehensive primer on differentiated services and
   tunnels.  Given ECN raises similar issues to differentiated services
   when interacting with tunnels, useful concepts introduced in RFC2983
   are used throughout, with brief recaps of the explanations where

1.2.  Document Roadmap

   The body of the document focuses solely on standards actions
   impacting implementation.  Appendices record the analysis that
   motivates and justifies these actions.  The whole document is
   organised as follows:

   o  Section 3 recaps relevant existing RFCs and explains exactly why
      changes are needed, referring to Appendix D and Appendix E in
      order to explain in detail why current tunnelling behaviours
      impede PCN deployment, at egress and ingress respectively.

   o  Section 4 uses precise standards terminology to specify the new
      ECN tunnelling behaviours.  It refers to Appendix A for analysis
      of the trade-offs between security, control and management design
      constraints that led to these particular standards actions.

   o  Extending the new IPsec tunnel ingress behaviour to all IP in IP
      tunnels requires consideration of backwards compatibility, which
      is covered in Section 5 and detailed changes from earlier RFCs are
      brought together in Section 6.

   o  Finally, a number of security considerations are discussed and
      conclusions are drawn.

   o  Additional specialist issues are deferred to appendices in
      addition to those already referred to above, in particular
      Appendix B discusses specialist tunnelling issues that could arise
      when ECN is fed back to a load regulation function on a middlebox,
      rather than at the source of the path.

2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in RFC 2119 [RFC2119].

Briscoe                Expires September 25, 2009               [Page 9]

Internet-Draft               ECN Tunnelling                   March 2009

3.  Summary of Pre-Existing RFCs

   This section is informative not normative.  It merely recaps pre-
   existing RFCs to help motivate changing these behaviours.  Earlier
   relevant RFCs that were either experimental or incomplete with
   respect to ECN tunnelling (RFC2481, RFC2401 and RFC2003) are not
   discussed, although the backwards compatibility considerations in
   Section 5 take them into account.  The question of whether tunnel
   implementations used in the Internet comply with any of these RFCs is
   also not discussed.

3.1.  Encapsulation at Tunnel Ingress

   The controversy at tunnel ingress has been over whether to propagate
   information about congestion experienced on the path upstream of the
   tunnel ingress into the outer header of the tunnel.

   Specifically, RFC3168 says that, if a tunnel fully supports ECN
   (termed a 'full-functionality' ECN tunnel in [RFC3168]), the tunnel
   ingress must not copy a CE marking from the inner header into the
   outer header that it creates.  Instead the tunnel ingress must set
   the outer header to ECT(0) (i.e. codepoint 10) if the ECN field is
   marked CE (codepoint 11) in the arriving IP header.  We term this
   'resetting' a CE codepoint.

   However, the new IPsec architecture in [RFC4301] reverses this rule,
   stating that the tunnel ingress must simply copy the ECN field from
   the arriving to the outer header.  The main purpose of the present
   specification is to carry the new behaviour of IPsec over to all IP
   in IP tunnels, so all tunnel ingress nodes consistently copy the ECN

   RFC3168 also provided a Limited Functionality mode that turns off ECN
   processing over the scope of the tunnel.  This is necessary if the
   ingress does not know whether the tunnel egress supports propagation
   of ECN markings.  Neither Limited Functionality mode nor Full
   Functionality mode are used in RFC4301 IPsec.

   These pre-existing behaviours are summarised in Figure 1.

Briscoe                Expires September 25, 2009              [Page 10]

Internet-Draft               ECN Tunnelling                   March 2009

    | Incoming Header |             Outgoing Outer Header             |
    | (also equal to  +---------------+---------------+---------------+
    | Outgoing Inner  |  RFC3168 ECN  |  RFC3168 ECN  | RFC4301 IPsec |
    |     Header)     |    Limited    |     Full      |               |
    |                 | Functionality | Functionality |               |
    |    Not-ECT      |   Not-ECT     |   Not-ECT     |   Not-ECT     |
    |     ECT(0)      |   Not-ECT     |    ECT(0)     |    ECT(0)     |
    |     ECT(1)      |   Not-ECT     |    ECT(1)     |    ECT(1)     |
    |       CE        |   Not-ECT     |    ECT(0)     |      CE       e|

    Figure 1: IP in IP Encapsulation: Recap of Pre-existing Behaviours

   For encapsulation, the specification in Section 4 below brings all IP
   in IP tunnels (v4 or v6) into line with the way IPsec tunnels
   [RFC4301] now construct the ECN field, except where a legacy tunnel
   egress might not understand ECN at all.  This removes the now
   redundant full functionality mode in the middle column of Figure 1.
   Wherever possible it ensures that the outer header reveals any
   congestion experienced so far on the whole path, not just since the
   last tunnel ingress.

   Why does it matter if we have different ECN encapsulation behaviours
   for IPsec and non-IPsec tunnels?  A general answer is that gratuitous
   inconsistency constrains the available design space and makes it
   harder to design networks and new protocols that work predictably.

   But there is also a specific need not to reset the CE codepoint.  The
   standards track proposal for excess rate pre-congestion notification
   (PCN [I-D.ietf-pcn-marking-behaviour]) only works correctly in the
   presence of RFC4301 IPsec encapsulation or [RFC5129] MPLS
   encapsulation, but not with RFC3168 IP in IP encapsulation
   (Appendix E explains why).  The PCN architecture
   [I-D.ietf-pcn-architecture] states that the regular RFC3168 rules for
   IP in IP tunnelling of the ECN field should not be used for PCN.  But
   if non-IPsec tunnels are already present within a network to which
   PCN is being added, that is not particularly helpful advice.

   The present specification provides a clean solution to this problem,
   so that network operators who want to use PCN and tunnels can specify
   that all tunnel endpoints in a PCN region need to be upgraded to
   comply with this specification.  Also, whether using PCN or not, as
   more tunnel endpoints comply with this specification, it should make
   ECN behaviour simpler, faster and more predictable.

Briscoe                Expires September 25, 2009              [Page 11]

Internet-Draft               ECN Tunnelling                   March 2009

   To ensure copying rather than resetting CE on ingress will not cause
   unintended side-effects, Appendix A assesses whether either harm any
   security, control or management functions.  It finds that resetting
   CE makes life difficult in a number of directions, while copying CE
   harms nothing (other than opening a low bit-rate covert channel
   vulnerability which the IETF Security Area now deems is manageable).

3.2.  Decapsulation at Tunnel Egress

   Both RFC3168 and RFC4301 specify the decapsulation behaviour
   summarised in Figure 2.  The ECN field in the outgoing header is set
   to the codepoint at the intersection of the appropriate incoming
   inner header (row) and incoming outer header (column).
    |  Incoming Inner  |             Incoming Outer Header            |
    |      Header      +---------+------------+------------+----------+
    |                  | Not-ECT |   ECT(0)   |   ECT(1)   |    CE    |
    |     Not-ECT      | Not-ECT |   drop(!!!)|   drop(!!!)| drop(!!!)|
    |      ECT(0)      |  ECT(0) | ECT(0)     | ECT(0)     |   CE     |
    |      ECT(1)      |  ECT(1) | ECT(1)     | ECT(1)     |   CE     |
    |        CE        |      CE |     CE     |     CE     |   CE     |
                       |                Outgoing Header               |

     Figure 2: IP in IP Decapsulation; Recap of Pre-existing Behaviour

   The behaviour in the table derives from the logic given in RFC3168,
   briefly recapped as follows:

   o  On decapsulation, if the inner ECN field is Not-ECT but the outer
      ECN field is anything except Not-ECT the decapsulator must drop
      the packet.  Drop is mandated because known legal protocol
      transitions should not be able to lead to these cases (indicated
      in the table by '(!!!)'), therefore the decapsulator may also
      raise an alarm;

   o  In all other cases, the outgoing ECN field is set to the more
      severe marking of the outer and inner ECN fields, where the
      ranking of severity from highest to lowest is CE, ECT, Not-ECT;

   o  ECT(0) and ECT(1) are considered of equal severity (indicated by
      just 'ECT' in the rank order above).  Where the inner and outer
      ECN fields are both ECT but they differ, the packet is forwarded
      with the codepoint of the inner ECN field, which prevents ECT
      codepoints being used for a covert channel.

Briscoe                Expires September 25, 2009              [Page 12]

Internet-Draft               ECN Tunnelling                   March 2009

   The specification for decapsulation in Section 4 fixes two problems
   with this pre-existing behaviour:

   o  Firstly, forwarding the codepoint of the inner header in the cases
      where both inner and outer are different values of ECT effectively
      implies that any distinction between ECT(0) and ECT(1) cannot be
      introduced in the future wherever a tunnel might be deployed.
      Therefore, the currently specified tunnel decapsulation behaviour
      unnecessarily wastes one of four codepoints (effectively wasting
      half a bit) in the IP (v4 & v6) header.  As explained in
      Appendix A.1, the original reason for not using the outer ECT
      codepoints for onward forwarding was to limit the covert channel
      across a decapsulator to 1 bit per packet.  However, now that the
      IETF Security Area has deemed that a 2-bit covert channel through
      an encapsulator is a manageable risk, the same should be true for
      a decapsulator.

      As well as being a general future-proofing issue, this problem is
      immediately pressing for standardisation of pre-congestion
      notification (PCN).  PCN solutions generally require three
      encoding states in addition to Not-ECT: one for 'not marked' and
      two increasingly severe levels of marking.  Although the ECN field
      gives sufficient codepoints for these three states, they cannot
      all be used for PCN because a change between ECT(0) and ECT(1) in
      any tunnelled packet would be lost when the outer header was
      decapsulated, dangerously discarding congestion signalling.  A
      number of wasteful or convoluted work-rounds to this problem are
      being considered for standardisation by the PCN working group (see
      Appendix D), but by far the simplest approach is just to remove
      the covert channel blockages from tunnelling behaviour, that are
      now deemed unnecessary anyway.  Not only will this streamline PCN
      standardisation, but it could also streamline other future uses of
      these codepoints.

   o  Secondly, mandating drop is not always a good idea just because a
      combination of headers seems invalid.  There are many cases where
      it has become nearly impossible to deploy new standards because
      legacy middleboxes drop packets carrying header values they don't
      expect.  Where possible, the new decapsulation behaviour specified
      in Section 4 below is more liberal in its response to unexpected
      combinations of headers.

4.  New ECN Tunnelling Rules

   The ECN tunnel processing rules below in Section 4.1 (ingress
   encapsulation) and Section 4.2 (egress decapsulation) are the default
   for a packet with any DSCP.  If required, different ECN encapsulation

Briscoe                Expires September 25, 2009              [Page 13]

Internet-Draft               ECN Tunnelling                   March 2009

   rules MAY be defined as part of the definition of an appropriate
   Diffserv PHB using the guidelines that follow in Section 4.3.
   However, the deployment burden of handling exceptional PHBs in
   implementations of all affected tunnels and lower layer link
   protocols should not be underestimated.

4.1.  Default Tunnel Ingress Behaviour

   A tunnel ingress compliant with this specification MUST implement a
   `normal mode'.  It might also need to implement a `compatibility
   mode' for backward compatibility with legacy tunnel egresses that do
   not understand ECN (see Section 5 for when compatibility mode is
   required).  Note that these are modes of the ingress tunnel endpoint
   only, not the tunnel as a whole.

   Whatever the mode, the tunnel ingress forwards the inner header
   without changing the ECN field.  In normal mode a tunnel ingress
   compliant with this specification MUST construct the outer
   encapsulating IP header by copying the 2-bit ECN field of the
   arriving IP header.  In compatibility mode it clears the ECN field in
   the outer header to the Not-ECT codepoint.  These rules are tabulated
   for convenience in Figure 3.
            | Incoming Header |     Outgoing Outer Header     |
            | (also equal to  +---------------+---------------+
            | Outgoing Inner  | Compatibility |    Normal     |
            |     Header)     |     Mode      |     Mode      |
            |    Not-ECT      |   Not-ECT     |   Not-ECT     |
            |     ECT(0)      |   Not-ECT     |    ECT(0)     |
            |     ECT(1)      |   Not-ECT     |    ECT(1)     |
            |       CE        |   Not-ECT     |      CE       |

              Figure 3: New IP in IP Encapsulation Behaviours

   Compatibility mode is the same per packet behaviour as the ingress
   end of RFC3168's limited functionality mode.  Normal mode is the same
   per packet behaviour as the ingress end of RFC4301 IPsec.

4.2.  Default Tunnel Egress Behaviour

   To decapsulate the inner header at the tunnel egress, a compliant
   tunnel egress MUST set the outgoing ECN field to the codepoint at the
   intersection of the appropriate incoming inner header (row) and outer
   header (column) in Figure 4.

Briscoe                Expires September 25, 2009              [Page 14]

Internet-Draft               ECN Tunnelling                   March 2009

    |  Incoming Inner  |             Incoming Outer Header            |
    |      Header      +---------+------------+------------+----------+
    |                  | Not-ECT |   ECT(0)   |   ECT(1)   |    CE    |
    |     Not-ECT      | Not-ECT |Not-ECT(!!!)|   drop(!!!)| drop(!!!)|
    |      ECT(0)      |  ECT(0) | ECT(0)     | ECT(1)     |   CE     |
    |      ECT(1)      |  ECT(1) | ECT(1)(!!!)| ECT(1)     |   CE     |
    |        CE        |      CE |     CE     |     CE(!!!)|   CE     |
                       |                Outgoing Header               |

              Figure 4: New IP in IP Decapsulation Behaviour

   This table for decapsulation behaviour is derived from the following

   o  If the inner ECN field is Not-ECT the decapsulator MUST NOT
      propagate any other ECN codepoint in the outer header onwards.
      This is because the inner Not-ECT marking is set by transports
      that would not understand the ECN protocol.  Instead:

      *  If the inner ECN field is Not-ECT and the outer ECN field is
         ECT(1) or CE the decapsulator MUST drop the packet.
         Reasoning: these combinations of codepoints either imply some
         illegal protocol transition has occurred within the tunnel, or
         that some locally defined mechanism is being used within the
         tunnel that might be signalling congestion.  In either case,
         the only appropriate signal to the transport is a packet drop.
         It would have been nice to allow packets with ECT(1) in the
         outer to be forwarded, but drop has had to be mandated in case
         future multi-level ECN schemes are defined.  Then ECT(1) and CE
         can be used in the future to signify two levels of congestion

      *  If the inner ECN field is Not-ECT and the outer ECN field is
         ECT(0) or Not-ECT the decapsulator MUST forward the packet with
         the ECN field cleared to Not-ECT.
         Reasoning: Although no known legal protocol transition would
         lead to ECT(0) in the outer and Not-ECT in the inner, no known
         or proposed protocol uses ECT(0) as a congestion signal either.
         Therefore in this case the packet can be forwarded rather than
         dropped, which will allow future standards actions to use this

Briscoe                Expires September 25, 2009              [Page 15]

Internet-Draft               ECN Tunnelling                   March 2009

   o  In all other cases, the outgoing ECN field is set to the more
      severe marking of the outer and inner ECN fields, where the
      ranking of severity from highest to lowest is CE, ECT(1), ECT(0),

   o  There are cases where no currently legal transition in any current
      or previous ECN tunneling specification would result in certain
      combinations of inner and outer ECN fields.  These cases are
      indicated in Figure 4 by '(!!!)').  In these cases, the
      decapsulator SHOULD log the event and MAY also raise an alarm, but
      not so often that the illegal combinations would amplify into a
      flood of alarm messages.

   The above logic allows for ECT(0) and ECT(1) to both represent the
   same severity of congestion marking (e.g. "not congestion marked").
   But it also allows future schemes to be defined where ECT(1) is a
   more severe marking than ECT(0).  This approach is discussed in
   Appendix D and in the discussion of the ECN nonce [RFC3540] in
   Section 8.

4.3.  Design Principles for Future Non-Default Schemes

   This section is informative not normative.

   S.5 of RFC3168 permits the Diffserv codepoint (DSCP)[RFC2474] to
   'switch in' different behaviours for marking the ECN field, just as
   it switches in different per-hop behaviours (PHBs) for scheduling.
   Therefore here we give guidance for designing possibly different
   marking schemes.

   In one word the guidance is "Don't".  If a scheme requires tunnels to
   implement special processing of the ECN field for certain DSCPs, it
   is highly unlikely that every implementer of every tunnel will want
   to add the required exception and that operators will want to deploy
   the required configuration options.  Therefore it is highly likely
   that some tunnels within a network will not implement this special
   case.  Therefore, designers should avoid non-default tunnelling
   schemes if at all possible.

   That said, if a non-default scheme for processing the ECN field is
   really required, the following guidelines may prove useful in its

   o  For any new scheme, a tunnel ingress should not set the ECN field
      of the outer header if it cannot guarantee that any corresponding
      tunnel egress will understand how to handle such an ECN field.

Briscoe                Expires September 25, 2009              [Page 16]

Internet-Draft               ECN Tunnelling                   March 2009

   o  On encapsulation in any new scheme, an outer header capable of
      carrying congestion markings should reflect accumulated congestion
      since the last interface designed to regulate load (see
      Appendix A.2 for the definition of a Load Regulator, which is
      usually but not always the data source).  This implies that new
      schemes for tunnelling congestion notification should copy
      congestion notification into the outer header of each new
      encapsulating header that supports it.

      Reasoning: The constraints from the three perspectives of
      security, control and management in Appendix A are somewhat in
      tension as to whether a tunnel ingress should copy congestion
      markings into the outer header it creates or reset them.  From the
      control perspective either copying or resetting works for existing
      arrangements, but copying has more potential for simplifying
      control.  From the management perspective copying is preferable.
      From the security perspective resetting is preferable but copying
      is now considered acceptable given the bandwidth of a 2-bit covert
      channel can be managed.  Therefore, on balance, copying is simpler
      and more useful than resetting and does minimal harm.

   o  For any new scheme, a tunnel egress should not forward any ECN
      codepoint if the arriving inner header implies the transport will
      not understand how to process it.

   o  On decapsulation in any new scheme, if a combination of inner and
      outer headers is encountered that should not have been possible,
      this event should be logged and an alarm raised.  But the packet
      should still be forwarded with a safe codepoint setting if at all
      possible.  This increases the chances of 'forward compatibility'
      with possible future protocol extensions.

   o  On decapsulation in any new scheme, the ECN field that the tunnel
      egress forwards should reflect the more severe congestion marking
      of the arriving inner and outer headers.

5.  Backward Compatibility

   Note: in RFC3168, a whole tunnel was considered in one of two modes:
   limited functionality or full functionality.  The new modes defined
   in this specification are only modes of the tunnel ingress.  The new
   tunnel egress behaviour has only one mode and doesn't need to know
   what mode the ingress is in.

Briscoe                Expires September 25, 2009              [Page 17]

Internet-Draft               ECN Tunnelling                   March 2009

5.1.  Non-Issues Upgrading Any Tunnel Decapsulation

   This specification only changes the egress per-packet calculation of
   the ECN field for combinations of inner and outer headers that have
   so far not been used in any IETF protocols.  Therefore, a tunnel
   egress complying with any previous specification (RFC4301, both modes
   of RFC3168, both modes of RFC2481, RFC2401 and RFC2003) can be
   upgraded to comply with this new decapsulation specification without
   any backwards compatibility issues.

   The proposed tunnel egress behaviour also requires no additional mode
   or option configuration at the ingress or egress nor any additional
   negotiation with the ingress.  A compliant tunnel egress merely needs
   to implement the one behaviour in Section 4.  The reduction to one
   mode at the egress has no backwards compatibility issues, because
   previously the egress produced the same output whichever mode the
   tunnel was in.

   These new decapsulation rules have been defined in such a way that
   congestion control will still work safely if any of the earlier
   versions of ECN processing are used unilaterally at the encapsulating
   ingress of the tunnel (any of RFC2003, RFC2401, either mode of
   RFC2481, either mode of RFC3168, RFC4301 and this present
   specification).  If a tunnel ingress tries to negotiate to use
   limited functionality mode or full functionality mode [RFC3168], a
   decapsulating tunnel egress compliant with this specification MUST
   agree to either request, as its behaviour will be the same in both

   For 'forward compatibility', a compliant tunnel egress SHOULD raise a
   warning about any requests to enter modes it doesn't recognise, but
   it can continue operating.  If no ECN-related mode is requested, a
   compliant tunnel egress can continue without raising any error or
   warning as its egress behaviour is compatible with all the legacy
   ingress behaviours that don't negotiate capabilities.

5.2.  Non-Issues for RFC4301 IPsec Encapsulation

   The new normal mode of ingress behaviour defined above (Section 4.1)
   brings all IP in IP tunnels into line with [RFC4301].  If one end of
   an IPsec tunnel is compliant with [RFC4301], the other end is
   guaranteed to also be RFC4301-compliant (there could be corner cases
   where manual keying is used, but they will be set aside here).
   Therefore the new normal ingress behaviour introduces no backward
   compatibility isses with IKEv2 [RFC4306] IPsec [RFC4301] tunnels, and
   no need for any new modes, options or configuration.

Briscoe                Expires September 25, 2009              [Page 18]

Internet-Draft               ECN Tunnelling                   March 2009

5.3.  Upgrading Other IP in IP Tunnel Encapsulators

   At the tunnel ingress, this specification effectively extends the
   scope of RFC4301's ingress behaviour to any IP in IP tunnel.  If any
   other IP in IP tunnel ingress (i.e. not RFC4301 IPsec) is upgraded to
   be compliant with this specification, it has to cater for the
   possibility that it is talking to a legacy tunnel egress that may not
   know how to process the ECN field.  If ECN capable outer headers were
   sent towards a legacy (e.g.  [RFC2003]) egress, it would most likely
   simply disregard the outer headers, dangerously discarding
   information about congestion experienced within the tunnel.  ECN-
   capable traffic sources would not see any congestion feedback and
   instead continually ratchet up their share of the bandwidth without
   realising that cross-flows from other ECN sources were continually
   having to ratchet down.

   This specification introduces no new backward compatibility issues
   when a compliant ingress talks with a legacy egress, but it has to
   provide similar sfaeguards to those already defined in RFC3168.
   Therefore, to comply with this specification, a tunnel ingress that
   does not always know the ECN capability of its tunnel egress MUST
   implement a 'normal' mode and a 'compatibility' mode, and for safety
   it MUST initiate each negotiated tunnel in compatibility mode.

   However, a tunnel ingress can be compliant even if it only implements
   the 'normal mode' of encapsulation behaviour, but only as long as it
   is designed or configured so that all possible tunnel egress nodes it
   will ever talk to will have at least full ECN functionality
   (complying with either RFC3168 full functionality mode, RFC4301 or
   this present specification).

   Before switching to normal mode, a compliant tunnel ingress that does
   not know the egress ECN capability MUST negotiate with the tunnel
   egress.  If the egress says it is compliant with this specification
   or with RFC3168 full functionality mode, the ingress puts itself into
   normal mode.  If the egress denies compliance with all of these or
   doesn't understand the question, the tunnel ingress MUST remain in
   compatibility mode.

   The encapsulation rules for normal mode and compatibility mode are
   defined in Section 4 (i.e. header copying or zeroing respectively).

   An ingress cannot claim compliance with this specification simply by
   disabling ECN processing across the tunnel (only implementing
   compatibility mode).  Although such a tunnel ingress is at least safe
   with the ECN behaviour of any egress it may encounter (any of
   RFC2003, RFC2401, either mode of RFC2481 and RFC3168's limited
   functionality mode), it doesn't meet the aim of introducing ECN.

Briscoe                Expires September 25, 2009              [Page 19]

Internet-Draft               ECN Tunnelling                   March 2009

   Therefore, a compliant tunnel ingress MUST at least implement `normal
   mode' and, if it might be used with arbitrary tunnel egress nodes, it
   MUST also implement `compatibility mode'.

   Implementation note: if a compliant node is the ingress for multiple
   tunnels, a mode setting will need to be stored for each tunnel
   ingress.  However, if a node is the egress for multiple tunnels, none
   of the tunnels will need to store a mode setting, because a compliant
   egress can only be in one mode.

6.  Changes from Earlier RFCs

   On encapsulation, the rule that a normal mode tunnel ingress MUST
   copy any ECN field into the outer header is a change to the ingress
   behaviour of RFC3168, but it is the same as the rules for IPsec
   tunnels in RFC4301.

   On decapsulation, the rules for calculating the outgoing ECN field at
   a tunnel egress are similar to the full functionality mode of ECN in
   RFC3168 and to RFC4301, with the following exceptions:

   o  The outer, not the inner, is propagated when the outer is ECT(1)
      and the inner is ECT(0);

   o  A packet with Not-ECT in the inner may be forwarded as Not-ECT
      rather than dropped, if the outer is ECT(0);

   o  The following extra illegal combinations have been identified,
      which may require logging and/or an alarm: outer ECT(1) with inner
      CE; outer ECT(0) with inner ECT(1)

   The rules for how a tunnel establishes whether the egress has full
   functionality ECN capabilities are an update to RFC3168.  For all the
   typical cases, RFC4301 is not updated by the ECN capability check in
   this specification, because a typical RFC4301 tunnel ingress will
   have already established that it is talking to an RFC4301 tunnel
   egress (e.g. if it uses IKEv2).  However, there may be some corner
   cases (e.g. manual keying) where an RFC4301 tunnel ingress talks with
   an egress with limited functionality ECN handling.  Strictly, for
   such corner cases, the requirement to use compatibility mode in this
   specification updates RFC4301, but this is unlikely to be necessary
   to implement for this corner case in practice.

   The optional ECN Tunnel field in the IPsec security association
   database (SAD) and the optional ECN Tunnel Security Association
   Attribute defined in RFC3168 are no longer needed.  The security
   association (SA) has no policy on ECN usage, because all RFC4301

Briscoe                Expires September 25, 2009              [Page 20]

Internet-Draft               ECN Tunnelling                   March 2009

   tunnels now support ECN without any policy choice.

   RFC3168 defines a (required) limited functionality mode and an
   (optional) full functionality mode for a tunnel, but RFC4301 doesn't
   need modes.  In this specification only the ingress might need two
   modes: a normal mode (required) and a compatibility mode (required in
   some scenarios, optional in others).  The egress needs only one mode
   which correctly handles any ingress ECN behaviour.

Additional changes to the RFC Index (to be removed by the RFC Editor):

   In the RFC index, RFC3168 should be identified as an update to
   RFC2003.  RFC4301 should be identified as an update to RFC3168.

   This specification updates RFC3168 and RFC4301.

7.  IANA Considerations

   This memo includes no request to IANA.

8.  Security Considerations

   Appendix A.1 discusses the security constraints imposed on ECN tunnel
   processing.  The new rules for ECN tunnel processing (Section 4)
   trade-off between security (covert channels) and congestion
   monitoring & control.  In fact, ensuring congestion markings are not
   lost is itself another aspect of security, because if we allowed
   congestion notification to be lost, any attempt to enforce a response
   to congestion would be much harder.

   If alternate congestion notification semantics are defined for a
   certain PHB (e.g. the pre-congestion notification architecture
   [I-D.ietf-pcn-architecture]), the scope of the alternate semantics
   might typically be bounded by the limits of a Diffserv region or
   regions, as envisaged in [RFC4774].  The inner headers in tunnels
   crossing the boundary of such a Diffserv region but ending within the
   region can potentially leak the external congestion notification
   semantics into the region, or leak the internal semantics out of the
   region.  [RFC2983] discusses the need for Diffserv traffic
   conditioning to be applied at these tunnel endpoints as if they are
   at the edge of the Diffserv region.  Similar concerns apply to any
   processing or propagation of the ECN field at the edges of a Diffserv
   region with alternate ECN semantics.  Such edge processing must also
   be applied at the endpoints of tunnels with one end inside and the
   other outside the domain.  [I-D.ietf-pcn-architecture] gives specific
   advice on this for the PCN case, but other definitions of alternate

Briscoe                Expires September 25, 2009              [Page 21]

Internet-Draft               ECN Tunnelling                   March 2009

   semantics will need to discuss the specific security implications in
   each case.

   With the decapsulation rules as they stood in RFC3168 and RFC4301, a
   small part of the protection of the ECN nonce [RFC3540] was
   compromised.  The new decapsulation rules do not solve this problem.

   The minor problem is as follows: The ECN nonce was defined to enable
   the data source to detect if a CE marking had been applied then
   subsequently removed.  The source could detect this by weaving a
   pseudo-random sequence of ECT(0) and ECT(1) values into a stream of
   packets, which is termed an ECN nonce.  By the decapsulation rules in
   RFC3168 and RFC4301, if the inner and outer headers carry
   contradictory ECT values only the inner header is preserved for
   onward forwarding.  So if a CE marking added to the outer ECN field
   in a tunnel has been illegally (or accidentally) suppressed by a
   subsequent node in the tunnel, the decapsulator will revert the ECN
   field to its value before tampering, hiding all evidence of the crime
   from the onward feedback loop.  We chose not to close this minor
   loophole for all the following reasons:

   1.  This loophole is only applicable in the corner case where the
       attacker controls a network node downstream of a congested node
       in the same tunnel;

   2.  In tunnelling scenarios, the ECN nonce is already vulnerable to
       suppression by nodes downstream of a congested node in the same
       tunnel, if they can copy the ECT value in the inner header to the
       outer header (any node in the tunnel can do this if the inner
       header is not encrypted, and an IPsec tunnel egress can do it
       whether or not the tunnel is encrypted);

   3.  Although the new decapsulation behaviour removes evidence of
       congestion suppression from the onward feedback loop, the
       decapsulator itself can at least detect that congestion within
       the tunnel has been suppressed;

   4.  The ECN nonce [RFC3540] currently has experimental status and
       there has been no evidence that anyone has implemented it beyond
       the author's prototype.

   We could have fixed this loophole by specifying that the outer header
   should always be propagated onwards if inner and outer are both ECT.
   Although this would close the minor loophole in the nonce, it would
   raise a minor safety issue if multilevel ECN or PCN were used.  A
   less severe marking in the inner header would override a more severe
   one in the outer.  Both are corner cases so it is difficult to decide
   which is more important:

Briscoe                Expires September 25, 2009              [Page 22]

Internet-Draft               ECN Tunnelling                   March 2009

   1.  The loophole in the nonce is only for a minor case of one tunnel
       node attacking another in the same tunnel;

   2.  The severity inversion for multilevel congestion notification
       would not result from any legal codepoint transition.

   We decided safety against misconfiguration was slightly more
   important than securing against an attack that has little, if any,
   clear motivation.

   If a legacy security policy configures a legacy tunnel ingress to
   negotiate to turn off ECN processing, a compliant tunnel egress will
   agree to a request to turn off ECN processing but it will actually
   still copy CE markings from the outer to the forwarded header.
   Although the tunnel ingress 'I' in Figure 5 (Appendix A.1) will set
   all ECN fields in outer headers to Not-ECT, 'M' could still toggle CE
   on and off to communicate covertly with 'B', because we have
   specified that 'E' only has one mode regardless of what mode it says
   it has negotiated.  We could have specified that 'E' should have a
   limited functionality mode and check for such behaviour.  But we
   decided not to add the extra complexity of two modes on a compliant
   tunnel egress merely to cater for a legacy security concern that is
   now considered manageable.

9.  Conclusions

   This document updates the ingress tunnelling encapsulation of RFC3168
   ECN for all IP in IP tunnels to bring it into line with the new
   behaviour in the IPsec architecture of RFC4301.  It copies rather
   than resets a congestion experienced (CE) marking when creating outer

   It also specifies new rules that update both RFC3168 and RFC4301 for
   calculating the outgoing ECN field on tunnel decapsulation.  The new
   rules update egress behaviour for two specific combinations of inner
   and outer header that have no current legal usage, but will now be
   possible to use in future standards actions, rather than being wasted
   by current tunnelling behaviour.

   The new rules propagate changes to the ECN field across tunnel end-
   points that were previously blocked due to a perceived covert channel
   vulnerability.  The new IPsec architecture deems the two-bit covert
   channel that the ECN field opens up is a manageable threat, so these
   new rules bring all IP in IP tunnelling into line with this new more
   permissive attitude.  The result is a single specification for all
   future tunnelling of ECN, whether IPsec or not.  Then equipment can
   be specified against a single ECN behaviour and ECN markings can have

Briscoe                Expires September 25, 2009              [Page 23]

Internet-Draft               ECN Tunnelling                   March 2009

   a well-defined meaning wherever they are measured in a network.  This
   new certainty will enable new uses of the ECN field that would
   otherwise be confounded by ambiguity.

   The immediate motivation for making these changes is to allow the
   introduction of multi-level pre-congestion notification (PCN).  But
   great care has been taken to ensure the resulting ECN tunnelling
   behaviour is simple and generic for other potential future uses.

   The change to encapsulation has been analysed from the three
   perspectives of security, control and management.  They are somewhat
   in tension as to whether a tunnel ingress should copy congestion
   markings into the outer header it creates or reset them.  From the
   control perspective either copying or resetting works for existing
   arrangements, but copying has more potential for simplifying control
   and resetting breaks at least one proposal already on the standards
   track.  From the management and monitoring perspective copying is
   preferable.  From the network security perspective (theft of service
   etc) copying is preferable.  From the information security
   perspective resetting is preferable, but the IETF Security Area now
   considers copying acceptable given the bandwidth of a 2-bit covert
   channel can be managed.  Therefore there are no points against
   copying and a number against resetting CE on ingress.

   The only downside of the changes to decapsulation is that the same
   2-bit covert channel is opened up as at the ingress, but this is now
   deemed to be a manageable threat.  The changes at decapsulation have
   been found to be free of any backwards compatibility issues.

10.  Acknowledgements

   Thanks to Anil Agawaal for pointing out a case where it's safe for a
   tunnel decapsulator to forward a combination of headers it doesn't
   understand.  Thanks to David Black for explaining a better way to
   think about function placement and to Louise Burness for a better way
   to think about multilayer transports and networks, having read
   [Patterns_Arch].  Also thanks to Arnaud Jacquet for the idea for
   Appendix C.  Thanks to Michael Menth, Bruce Davie, Toby Moncaster,
   Gorry Fairhurst, Sally Floyd, Alfred Hoenes and Gabriele Corliano for
   their thoughts and careful review comments.

   Bob Briscoe is partly funded by Trilogy, a research project (ICT-
   216372) supported by the European Community under its Seventh
   Framework Programme.  The views expressed here are those of the
   author only.

Briscoe                Expires September 25, 2009              [Page 24]

Internet-Draft               ECN Tunnelling                   March 2009

11.  Comments Solicited

   Comments and questions are encouraged and very welcome.  They can be
   addressed to the IETF Transport Area working group mailing list
   <>, and/or to the authors.

12.  References

12.1.  Normative References

   [RFC2003]  Perkins, C., "IP Encapsulation within IP", RFC 2003,
              October 1996.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
              "Definition of the Differentiated Services Field (DS
              Field) in the IPv4 and IPv6 Headers", RFC 2474,
              December 1998.

   [RFC3168]  Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
              of Explicit Congestion Notification (ECN) to IP",
              RFC 3168, September 2001.

   [RFC4301]  Kent, S. and K. Seo, "Security Architecture for the
              Internet Protocol", RFC 4301, December 2005.

12.2.  Informative References

              Briscoe, B., "PCN 3-State Encoding Extension in a single
              DSCP", draft-briscoe-pcn-3-in-1-encoding-00 (work in
              progress), October 2008.

              Charny, A., Zhang, X., Faucheur, F., and V. Liatsos, "Pre-
              Congestion Notification Using Single Marking for Admission
              and  Termination", draft-charny-pcn-single-marking-03
              (work in progress), November 2007.

              Eardley, P., "Pre-Congestion Notification (PCN)
              Architecture", draft-ietf-pcn-architecture-10 (work in
              progress), March 2009.


Briscoe                Expires September 25, 2009              [Page 25]

Internet-Draft               ECN Tunnelling                   March 2009

              Moncaster, T., Briscoe, B., and M. Menth, "Baseline
              Encoding and Transport of Pre-Congestion Information",
              draft-ietf-pcn-baseline-encoding-02 (work in progress),
              February 2009.

              Eardley, P., "Marking behaviour of PCN-nodes",
              draft-ietf-pcn-marking-behaviour-02 (work in progress),
              March 2009.

              Bryant, S., Davie, B., Martini, L., and E. Rosen,
              "Pseudowire Congestion Control Framework",
              draft-ietf-pwe3-congestion-frmwk-01 (work in progress),
              May 2008.

              Menth, M., Babiarz, J., Moncaster, T., and B. Briscoe,
              "PCN Encoding for Packet-Specific Dual Marking (PSDM)",
              draft-menth-pcn-psdm-encoding-00 (work in progress),
              July 2008.

              Moncaster, T., Briscoe, B., and M. Menth, "A three state
              extended PCN encoding scheme",
              draft-moncaster-pcn-3-state-encoding-01 (work in
              progress), March 2009.

              Satoh, D., Maeda, Y., Phanachet, O., and H. Ueno, "Single
              PCN Threshold Marking by using PCN baseline encoding for
              both  admission and termination controls",
              draft-satoh-pcn-st-marking-01 (work in progress),
              March 2009.

              IEEE, "IEEE Standard for Local and Metropolitan Area
              Networks--Virtual Bridged Local Area Networks - Amendment
              10: Congestion Notification", 2008,

              (Work in Progress; Access Controlled link within page)

              ITU-T, "Traffic Control and Congestion Control in B-ISDN",
              ITU-T Rec. I.371 (03/04), March 2004.


Briscoe                Expires September 25, 2009              [Page 26]

Internet-Draft               ECN Tunnelling                   March 2009

              IETF, "Congestion and Pre-Congestion Notification (pcn)",
              IETF w-g charter , Feb 2007,

              Day, J., "Patterns in Network Architecture: A Return to
              Fundamentals", Pub: Prentice Hall ISBN-13: 9780132252423,
              Jan 2008.

   [RFC1254]  Mankin, A. and K. Ramakrishnan, "Gateway Congestion
              Control Survey", RFC 1254, August 1991.

   [RFC2205]  Braden, B., Zhang, L., Berson, S., Herzog, S., and S.
              Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
              Functional Specification", RFC 2205, September 1997.

   [RFC2983]  Black, D., "Differentiated Services and Tunnels",
              RFC 2983, October 2000.

   [RFC3426]  Floyd, S., "General Architectural and Policy
              Considerations", RFC 3426, November 2002.

   [RFC3540]  Spring, N., Wetherall, D., and D. Ely, "Robust Explicit
              Congestion Notification (ECN) Signaling with Nonces",
              RFC 3540, June 2003.

   [RFC4306]  Kaufman, C., "Internet Key Exchange (IKEv2) Protocol",
              RFC 4306, December 2005.

   [RFC4423]  Moskowitz, R. and P. Nikander, "Host Identity Protocol
              (HIP) Architecture", RFC 4423, May 2006.

   [RFC4774]  Floyd, S., "Specifying Alternate Semantics for the
              Explicit Congestion Notification (ECN) Field", BCP 124,
              RFC 4774, November 2006.

   [RFC5129]  Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion
              Marking in MPLS", RFC 5129, January 2008.

   [Shayman]  "Using ECN to Signal Congestion Within an MPLS Domain",
              2000, <


Briscoe                Expires September 25, 2009              [Page 27]

Internet-Draft               ECN Tunnelling                   March 2009

Appendix A.  Design Constraints

   Tunnel processing of a congestion notification field has to meet
   congestion control and management needs without creating new
   information security vulnerabilities (if information security is
   required).  This appendix documents the analysis of the tradeoffs
   between these factors that led to the new encapsulation rules in
   Section 4.1.

A.1.  Security Constraints

   Information security can be assured by using various end to end
   security solutions (including IPsec in transport mode [RFC4301]), but
   a commonly used scenario involves the need to communicate between two
   physically protected domains across the public Internet.  In this
   case there are certain management advantages to using IPsec in tunnel
   mode solely across the publicly accessible part of the path.  The
   path followed by a packet then crosses security 'domains'; the ones
   protected by physical or other means before and after the tunnel and
   the one protected by an IPsec tunnel across the otherwise unprotected
   domain.  We will use the scenario in Figure 5 where endpoints 'A' and
   'B' communicate through a tunnel.  The tunnel ingress 'I' and egress
   'E' are within physically protected edge domains, while the tunnel
   spans an unprotected internetwork where there may be 'men in the
   middle', M.

                physically       unprotected     physically
            <-protected domain-><--domain--><-protected domain->
            +------------------+            +------------------+
            |                  |      M     |                  |
            |    A-------->I=========>==========>E-------->B   |
            |                  |            |                  |
            +------------------+            +------------------+
                                <----IPsec secured---->

                      Figure 5: IPsec Tunnel Scenario

   IPsec encryption is typically used to prevent 'M' seeing messages
   from 'A' to 'B'.  IPsec authentication is used to prevent 'M'
   masquerading as the sender of messages from 'A' to 'B' or altering
   their contents.  But 'I' can also use IPsec tunnel mode to allow 'A'
   to communicate with 'B', but impose encryption to prevent 'A' leaking
   information to 'M'.  Or 'E' can insist that 'I' uses tunnel mode
   authentication to prevent 'M' communicating information to 'B'.
   Mutable IP header fields such as the ECN field (as well as the TTL/
   Hop Limit and DS fields) cannot be included in the cryptographic
   calculations of IPsec.  Therefore, if 'I' copies these mutable fields

Briscoe                Expires September 25, 2009              [Page 28]

Internet-Draft               ECN Tunnelling                   March 2009

   into the outer header that is exposed across the tunnel it will have
   allowed a covert channel from 'A' to M that bypasses its encryption
   of the inner header.  And if 'E' copies these fields from the outer
   header to the inner, even if it validates authentication from 'I', it
   will have allowed a covert channel from 'M' to 'B'.

   ECN at the IP layer is designed to carry information about congestion
   from a congested resource towards downstream nodes.  Typically a
   downstream transport might feed the information back somehow to the
   point upstream of the congestion that can regulate the load on the
   congested resource, but other actions are possible (see [RFC3168]
   S.6).  In terms of the above unicast scenario, ECN is typically
   intended to create an information channel from 'M' to 'B' (for 'B' to
   feed back to 'A').  Therefore the goals of IPsec and ECN are mutually

   With respect to the DS or ECN fields, S.5.1.2 of RFC4301 says,
   "controls are provided to manage the bandwidth of this [covert]
   channel".  Using the ECN processing rules of RFC4301, the channel
   bandwidth is two bits per datagram from 'A' to 'M' and one bit per
   datagram from 'M' to 'A' (because 'E' limits the combinations of the
   2-bit ECN field that it will copy).  In both cases the covert channel
   bandwidth is further reduced by noise from any real congestion
   marking.  RFC4301 therefore implies that these covert channels are
   sufficiently limited to be considered a manageable threat.  However,
   with respect to the larger (6b) DS field, the same section of RFC4301
   says not copying is the default, but a configuration option can allow
   copying "to allow a local administrator to decide whether the covert
   channel provided by copying these bits outweighs the benefits of
   copying".  Of course, an administrator considering copying of the DS
   field has to take into account that it could be concatenated with the
   ECN field giving an 8b per datagram covert channel.

   Thus, for tunnelling the 6b Diffserv field two conceptual models have
   had to be defined so that administrators can trade off security
   against the needs of traffic conditioning [RFC2983]:

   The uniform model:  where the DIffserv field is preserved end-to-end
      by copying into the outer header on encapsulation and copying from
      the outer header on decapsulation.

   The pipe model:  where the outer header is independent of that in the
      inner header so it hides the Diffserv field of the inner header
      from any interaction with nodes along the tunnel.

   However, for ECN, the new IPsec security architecture in RFC4301 only
   standardised one tunnelling model equivalent to the uniform model.
   It deemed that simplicity was more important than allowing

Briscoe                Expires September 25, 2009              [Page 29]

Internet-Draft               ECN Tunnelling                   March 2009

   administrators the option of a tiny increment in security, especially
   given not copying congestion indications could seriously harm
   everyone's network service.

A.2.  Control Constraints

   Congestion control requires that any congestion notification marked
   into packets by a resource will be able to traverse a feedback loop
   back to a function capable of controlling the load on that resource.
   To be precise, rather than calling this function the data source, we
   will call it the Load Regulator.  This will allow us to deal with
   exceptional cases where load is not regulated by the data source, but
   usually the two terms will be synonymous.  Note the term "a function
   _capable of_ controlling the load" deliberately includes a source
   application that doesn't actually control the load but ought to (e.g.
   an application without congestion control that uses UDP).


                     Figure 6: Simple Tunnel Scenario

   We now consider a similar tunnelling scenario to the IPsec one just
   described, but without the different security domains so we can just
   focus on ensuring the control loop and management monitoring can work
   (Figure 6).  If we want resources in the tunnel to be able to
   explicitly notify congestion and the feedback path is from 'B' to
   'A', it will certainly be necessary for 'E' to copy any CE marking
   from the outer header to the inner header for onward transmission to
   'B', otherwise congestion notification from resources like 'M' cannot
   be fed back to the Load Regulator ('A').  But it doesn't seem
   necessary for 'I' to copy CE markings from the inner to the outer
   header.  For instance, if resource 'R' is congested, it can send
   congestion information to 'B' using the congestion field in the inner
   header without 'I' copying the congestion field into the outer header
   and 'E' copying it back to the inner header.  'E' can still write any
   additional congestion marking introduced across the tunnel into the
   congestion field of the inner header.

   It might be useful for the tunnel egress to be able to tell whether
   congestion occurred across a tunnel or upstream of it.  If outer
   header congestion marking was reset by the tunnel ingress ('I'), at
   the end of a tunnel ('E') the outer headers would indicate congestion
   experienced across the tunnel ('I' to 'E'), while the inner header
   would indicate congestion upstream of 'I'.  But similar information
   can be gleaned even if the tunnel ingress copies the inner to the
   outer headers.  At the end of the tunnel ('E'), any packet with an

Briscoe                Expires September 25, 2009              [Page 30]

Internet-Draft               ECN Tunnelling                   March 2009

   _extra_ mark in the outer header relative to the inner header
   indicates congestion across the tunnel ('I' to 'E'), while the inner
   header would still indicate congestion upstream of ('I').  Appendix C
   gives a simple and precise method for a tunnel egress to infer the
   congestion level introduced across a tunnel.

   All this shows that 'E' can preserve the control loop irrespective of
   whether 'I' copies congestion notification into the outer header or
   resets it.

   That is the situation for existing control arrangements but, because
   copying reveals more information, it would open up possibilities for
   better control system designs.  For instance, Appendix E describes
   how resetting CE marking at a tunnel ingress confuses a proposed
   congestion marking scheme on the standards track.  It ends up
   removing excessive amounts of traffic unnecessarily.  Whereas copying
   CE markings at ingress leads to the correct control behaviour.

A.3.  Management Constraints

   As well as control, there are also management constraints.
   Specifically, a management system may monitor congestion markings in
   passing packets, perhaps at the border between networks as part of a
   service level agreement.  For instance, monitors at the borders of
   autonomous systems may need to measure how much congestion has
   accumulated since the original source, perhaps to determine between
   them how much of the congestion is contributed by each domain.

   Therefore, when monitoring the middle of a path, it should be
   possible to establish how far back in the path congestion markings
   have accumulated from.  In this document we term this the baseline of
   congestion marking (or the Congestion Baseline), i.e. the source of
   the layer that last reset (or created) the congestion notification
   field.  Given some tunnels cross domain borders (e.g. consider M in
   Figure 6 is monitoring a border), it would therefore be desirable for
   'I' to copy congestion accumulated so far into the outer headers
   exposed across the tunnel.

   Appendix B.2 discusses various scenarios where the Load Regulator
   lies in-path, not at the source host as we would typically expect.
   It concludes that a Congestion Baseline is determined by where the
   Load Regulator function is, which should be identified in the
   transport layer, not by addresses in network layer headers.  This
   applies whether the Load Regulator is at the source host or within
   the path.  The appendix also discusses where a Load Regulator
   function should be located relative to a local tunnel encapsulation

Briscoe                Expires September 25, 2009              [Page 31]

Internet-Draft               ECN Tunnelling                   March 2009

Appendix B.  Relative Placement of Tunnelling and In-Path Load

B.1.  Identifiers and In-Path Load Regulators

   The Load Regulator is the node to which congestion feedback should be
   returned by the next downstream node with a transport layer feedback
   function (typically but not always the data receiver).  The Load
   Regulator is often, but not always the data source.  It is not always
   (or even typically) the same thing as the node identified by the
   source address of the outermost exposed header.  In general the
   addressing of the outermost encapsulation header says nothing about
   the identifiers of either the upstream or the downstream transport
   layer functions.  As long as the transport functions know each
   other's addresses, they don't have to be identified in the network
   layer or in any link layer.  It was only a convenience that a TCP
   receiver assumed that the address of the source transport is the same
   as the network layer source address of an IP packet it receives.

   More generally, the return transport address for feedback could be
   identified solely in the transport layer protocol.  For instance, a
   signalling protocol like RSVP [RFC2205] breaks up a path into
   transport layer hops and informs each hop of the address of its
   transport layer neighbour without any need to identify these hops in
   the network layer.  RSVP can be arranged so that these transport
   layer hops are bigger than the underlying network layer hops.  The
   host identity protocol (HIP) architecture [RFC4423] also supports the
   same principled separation (for mobility amongst other things), where
   the transport layer sender identifies its transport address for
   feedback to be sent to, using an identifier provided by a shim below
   the transport layer.

   Keeping to this layering principle deliberately doesn't require a
   network layer packet header to reveal the origin address from where
   congestion notification accumulates (its Congestion Baseline).  It is
   not necessary for the network and lower layers to know the address of
   the Load Regulator.  Only the destination transport needs to know
   that.  With forward congestion notification, the network and link
   layers only notify congestion forwards; they aren't involved in
   feeding it backwards.  If they are (e.g. backward congestion
   notification (BCN) in Ethernet [IEEE802.1au] or EFCI in ATM
   [ITU-T.I.371]), that should be considered as a transport function
   added to the lower layer, which must sort out its own addressing.
   Indeed, this is one reason why ICMP source quench is now deprecated
   [RFC1254]; when congestion occurs within a tunnel it is complex
   (particularly in the case of IPsec tunnels) to return the ICMP
   messages beyond the tunnel ingress back to the Load Regulator.

Briscoe                Expires September 25, 2009              [Page 32]

Internet-Draft               ECN Tunnelling                   March 2009

   Similarly, if a management system is monitoring congestion and needs
   to know the Congestion Baseline, the management system has to find
   this out from the transport; in general it cannot tell solely by
   looking at the network or link layer headers.

B.2.  Non-Dependence of Tunnelling on In-path Load Regulation

   We have said that at any point in a network, the Congestion Baseline
   (where congestion notification starts from zero) should be the
   previous upstream Load Regulator.  We have also said that the ingress
   of an IP in IP tunnel must copy congestion indications to the
   encapsulating outer headers it creates.  If the Load Regulator is in-
   path rather than at the source, and also a tunnel ingress, these two
   requirements seem to be contradictory.  A tunnel ingress must not
   reset incoming congestion, but a Load Regulator must be the
   Congestion Baseline, implying it needs to reset incoming congestion.

   In fact, the two requirements are not contradictory, because a Load
   Regulator and a tunnel ingress are not the names of machines, but the
   names of functions within a machine that typically occur in sequence
   on a stream of packets, not at the same point.  Figure 7 is borrowed
   from [RFC2983] (which was making a similar point about the location
   of Diffserv traffic conditioning relative to the encapsulation
   function of a tunnel).  An in-path Load Regulator can act on packets
   either at [1 - Before] encapsulation or at [2 - Outer] after
   encapsulation.  Load Regulation does not ever need to be integrated
   with the [Encapsulate] function (but it can be for efficiency).
   Therefore we can still mandate that the [Encapsulate] function always
   copies CE into the outer header.

     >>-----[1 - Before]--------[Encapsulate]----[3 - Inner]---------->>
                                           +--------[2 - Outer]------->>

     Figure 7: Placement of In-Path Load Regulator Relative to Tunnel

   Then separately, if there is a Load Regulator at location [2 -
   Outer], it might reset CE to ECT(0), say.  Then the Congestion
   Baseline for the lower layer (outer) will be [2 - Outer], while the
   Congestion Baseline of the inner layer will be unchanged.  But how
   encapsulation works has nothing to do with whether a Load Regulator
   is present or where it is.

   If on the other hand a Load Regulator resets CE at [1 - Before], the

Briscoe                Expires September 25, 2009              [Page 33]

Internet-Draft               ECN Tunnelling                   March 2009

   Congestion Baseline of both the inner and outer headers will be [1 -
   Before].  But again, encapsulation is independent of load regulation.

B.3.  Dependence of In-Path Load Regulation on Tunnelling

   Although encapsulation doesn't need to depend on in-path load
   regulation, the reverse is not true.  The placement of an in-path
   Load Regulator must be carefully considered relative to
   encapsulation.  Some examples are given in the following for

   In the traditional Internet architecture one tends to think of the
   source host as the Load Regulator for a path.  It is generally not
   desirable or practical for a node part way along the path to regulate
   the load.  However, various reasonable proposals for in-path load
   regulation have been made from time to time (e.g. fair queuing,
   traffic engineering, flow admission control).  The IETF has recently
   chartered a working group to standardise admission control across a
   part of a path using pre-congestion notification (PCN) [PCNcharter].
   This is of particular relevance here because it involves congestion
   notification with an in-path Load Regulator, it can involve
   tunnelling and it certainly involves encapsulation more generally.

   We will use the more complex scenario in Figure 8 to tease out all
   the issues that arise when combining congestion notification and
   tunnelling with various possible in-path load regulation schemes.  In
   this case 'I1' and 'E2' break up the path into three separate
   congestion control loops.  The feedback for these loops is shown
   going right to left across the top of the figure.  The 'V's are arrow
   heads representing the direction of feedback, not letters.  But there
   are also two tunnels within the middle control loop: 'I1' to 'E1' and
   'I2' to 'E2'.  The two tunnels might be VPNs, perhaps over two MPLS
   core networks.  M is a congestion monitoring point, perhaps between
   two border routers where the same tunnel continues unbroken across
   the border.
        ______     _______________________________________      _____
       /      \   /                                        \   /     \
      V        \ V                                M         \ V       \

                     Figure 8: Complex Tunnel Scenario

   The question is, should the congestion markings in the outer exposed
   headers of a tunnel represent congestion only since the tunnel
   ingress or over the whole upstream path from the source of the inner
   header (whatever that may mean)?  Or put another way, should 'I1' and
   'I2' copy or reset CE markings?

Briscoe                Expires September 25, 2009              [Page 34]

Internet-Draft               ECN Tunnelling                   March 2009

   Based on the design principles in Section 4.3, the answer is that the
   Congestion Baseline should be the nearest upstream interface designed
   to regulate traffic load--the Load Regulator.  In Figure 8 'A', 'I1'
   or 'E2' are all Load Regulators.  We have shown the feedback loops
   returning to each of these nodes so that they can regulate the load
   causing the congestion notification.  So the Congestion Baseline
   exposed to M should be 'I1' (the Load Regulator), not 'I2'.
   Therefore I1 should reset any arriving CE markings.  In this case,
   'I1' knows the tunnel to 'E1' is unrelated to its load regulation
   function.  So the load regulation function within 'I1' should be
   placed at [1 - Before] tunnel encapsulation within 'I1' (using the
   terminology of Figure 7).  Then the Congestion Baseline all across
   the networks from 'I1' to 'E2' in both inner and outer headers will
   be 'I1'.

   The following further examples illustrate how this answer might be

   o  We argued in Appendix E that resetting CE on encapsulation could
      harm PCN excess rate marking, which marks excess traffic for
      removal in subsequent round trips.  This marking relies on not
      marking packets if another node upstream has already marked them
      for removal.  If there were a tunnel ingress between the two which
      reset CE markings, it would confuse the downstream node into
      marking far too much traffic for removal.  So why do we say that
      'I1' should reset CE, while a tunnel ingress shouldn't?  The
      answer is that it is the Load Regulator function at 'I1' that is
      resetting CE, not the tunnel encapsulator.  The Load Regulator
      needs to set itself as the Congestion Baseline, so the feedback it
      gets will only be about congestion on links it can relieve itself
      (by regulating the load into them).  When it resets CE markings,
      it knows that something else upstream will have dealt with the
      congestion notifications it removes, given it is part of an end-
      to-end admission control signalling loop.  It therefore knows that
      previous hops will be covered by other Load Regulators.
      Meanwhile, the tunnel ingresses at both 'I1' and 'I2' should
      follow the new rule for any tunnel ingress and copy congestion
      marking into the outer tunnel header.  The ingress at 'I1' will
      happen to copy headers that have already been reset just
      beforehand.  But it doesn't need to know that.

   o  [Shayman] suggested feedback of ECN accumulated across an MPLS
      domain could cause the ingress to trigger re-routing to mitigate
      congestion.  This case is more like the simple scenario of
      Figure 6, with a feedback loop across the MPLS domain ('E' back to
      'I').  I is a Load Regulator because re-routing around congestion
      is a load regulation function.  But in this case 'I' should only
      reset itself as the Congestion Baseline in outer headers, as it is

Briscoe                Expires September 25, 2009              [Page 35]

Internet-Draft               ECN Tunnelling                   March 2009

      not handling congestion outside its domain, so it must preserve
      the end-to-end congestion feedback loop for something else to
      handle (probably the data source).  Therefore the Load Regulator
      within 'I' should be placed at [2 - Outer] to reset CE markings
      just after the tunnel ingress has copied them from arriving
      headers.  Again, the tunnel encapsulation function at 'I' simply
      copies incoming headers, unaware that the load regulator will
      subsequently reset its outer headers.

   o  The PWE3 working group of the IETF is considering the problem of
      how and whether an aggregate edge-to-edge pseudo-wire emulation
      should respond to congestion [I-D.ietf-pwe3-congestion-frmwk].
      Although the study is still at the requirements stage, some
      (controversial) solution proposals include in-path load regulation
      at the ingress to the tunnel that could lead to tunnel
      arrangements with similar complexity to that of Figure 8.

   These are not contrived scenarios--they could be a lot worse.  For
   instance, a host may create a tunnel for IPsec which is placed inside
   a tunnel for Mobile IP over a remote part of its path.  And around
   this all we may have MPLS labels being pushed and popped as packets
   pass across different core networks.  Similarly, it is possible that
   subnets could be built from link technology (e.g. future Ethernet
   switches) so that link headers being added and removed could involve
   congestion notification in future Ethernet link headers with all the
   same issues as with IP in IP tunnels.

   One reason we introduced the concept of a Load Regulator was to allow
   for in-path load regulation.  In the traditional Internet
   architecture one tends to think of a host and a Load Regulator as
   synonymous, but when considering tunnelling, even the definition of a
   host is too fuzzy, whereas a Load Regulator is a clearly defined
   function.  Similarly, the concept of innermost header is too fuzzy to
   be able to (wrongly) say that the source address of the innermost
   header should be the Congestion Baseline.  Which is the innermost
   header when multiple encapsulations may be in use?  Where do we stop?
   If we say the original source in the above IPsec-Mobile IP case is
   the host, how do we know it isn't tunnelling an encrypted packet
   stream on behalf of another host in a p2p network?

   We have become used to thinking that only hosts regulate load.  The
   end to end design principle advises that this is a good idea
   [RFC3426], but it also advises that it is solely a guiding principle
   intended to make the designer think very carefully before breaking
   it.  We do have proposals where load regulation functions sit within
   a network path for good, if sometimes controversial, reasons, e.g.
   PCN edge admission control gateways [I-D.ietf-pcn-architecture] or
   traffic engineering functions at domain borders to re-route around

Briscoe                Expires September 25, 2009              [Page 36]

Internet-Draft               ECN Tunnelling                   March 2009

   congestion [Shayman].  Whether or not we want in-path load
   regulation, we have to work round the fact that it will not go away.

Appendix C.  Contribution to Congestion across a Tunnel

   This specification mandates that a tunnel ingress determines the ECN
   field of each new outer tunnel header by copying the arriving header.
   Concern has been expressed that this will make it difficult for the
   tunnel egress to monitor congestion introduced only along a tunnel,
   which is easy if the outer ECN field is reset at a tunnel ingress
   (RFC3168 full functionality mode).  However, in fact copying CE marks
   at ingress will still make it easy for the egress to measure
   congestion introduced across a tunnel, as illustrated below.

   Consider 100 packets measured at the egress.  It measures that 30 are
   CE marked in the inner and outer headers and 12 have additional CE
   marks in the outer but not the inner.  This means packets arriving at
   the ingress had already experienced 30% congestion.  However, it does
   not mean there was 12% congestion across the tunnel.  The correct
   calculation of congestion across the tunnel is p_t = 12/(100-30) =
   12/70 = 17%.  This is easy for the egress to to measure.  It is the
   packets with additional CE marking in the outer header (12) as a
   proportion of packets not marked in the inner header (70).

   Figure 9 illustrates this in a combinatorial probability diagram.
   The square represents 100 packets.  The 30% division along the bottom
   represents marking before the ingress, and the p_t division up the
   side represents marking along the tunnel.

     |     |         |
     | 30  |         |
     |     |         |       The large square
     |     +---------+p_t    represents 100 packets
     |     |   12    |
     0    30%       100%
     inner header marking

       Figure 9: Tunnel Marking of Packets Already Marked at Ingress

Briscoe                Expires September 25, 2009              [Page 37]

Internet-Draft               ECN Tunnelling                   March 2009

Appendix D.  Why Not Propagating ECT(1) on Decapsulation Impedes PCN

   Multi-level congestion notification is currently on the IETF's
   standards track agenda in the Congestion and Pre-Congestion
   Notification (PCN) working group.  The PCN working group eventually
   requires three congestion states (not marked and two increasingly
   severe levels of congestion marking) [I-D.ietf-pcn-architecture].
   The aim is for the less severe level of marking to stop admitting new
   traffic and the more severe level to terminate sufficient existing
   flows to bring a network back to its operating point after a serious

   Although the ECN field gives sufficient codepoints for these three
   states, current ECN tunnelling RFCs prevent the PCN working group
   from using three ECN states in case any tunnel decapsulations occur
   within a PCN region (see Appendix A of
   [I-D.ietf-pcn-baseline-encoding]).  If a node in a tunnel sets the
   ECN field to ECT(0) or ECT(1), this change will be discarded by a
   tunnel egress compliant with RFC4301 or RFC3168.  This can be seen in
   Figure 2 (Section 3.2), where ECT values in the outer header are
   ignored unless the inner header is the same.  Effectively one ECT
   codepoint is wasted; the ECT(0) and ECT(1) codepoints have to be
   treated as just one codepoint when they could otherwise have been
   used for their intended purpose of congestion notification.

   As a consequence, the PCN w-g has initially confined itself to two
   encoding states as a baseline encoding
   [I-D.ietf-pcn-baseline-encoding].  And it has had to propose an
   experimental extension using extra Diffserv codepoint(s) to encode
   the extra states [I-D.moncaster-pcn-3-state-encoding], using up the
   rapidly exhausting DSCP space while leaving ECN codepoints unused.
   Another PCN encoding has been proposed that would survive tunnelling
   without an extra DSCP [I-D.menth-pcn-psdm-encoding], but it requires
   the PCN edge gateways to somehow share state so the egress can
   determine which marking a packet started with at the ingress.  Also a
   PCN ingress node can game the system by initiating packets with
   inappropriate markings.  Yet another work-round to the ECN tunnelling
   problem proposes a more involved marking algorithm in the forwarding
   plane to encode the three congestion notification states using only
   two ECN codepoints [I-D.satoh-pcn-st-marking].  Still another
   proposal compromises the precision of the admission control
   mechanism, but manages to work with just two encoding states and a
   single marking algorithm [I-D.charny-pcn-single-marking].

   Rather than require the IETF to bless any of these work-rounds, this
   specification fixes the root cause of the problem so that operators
   deploying PCN can simply ask that tunnel end-points within a PCN
   region should comply with this new ECN tunnelling specification.

Briscoe                Expires September 25, 2009              [Page 38]

Internet-Draft               ECN Tunnelling                   March 2009

   Then PCN can use the trivially simple experimental 3-state ECN
   encoding defined in [I-D.briscoe-pcn-3-in-1-encoding].

D.1.  Alternative Ways to Introduce the New Decapsulation Rules

   There are a number of ways for the new decapsulation rules to be

   o  They could be specified in the present standards track proposal
      (preferred) or in an experimental extension;

   o  They could be specified as a new default for all Diffserv PHBs
      (preferred) or as an option to be configured only for Diffserv
      PHBs requiring them (e.g.  PCN).

   The argument for making this change now, rather than in a separate
   experimental extension, is to avoid the burden of an extra standard
   to be compliant with and to be backwards compatible with--so we don't
   add to the already complex history of ECN tunnelling RFCs.  The
   argument for a separate experimental extension is that we may never
   need this change (if PCN is never successfully deployed and if no-one
   ever needs three ECN or PCN encoding states rather than two).
   However, the change does no harm to existing mechanisms and stops
   tunnels wasting of quarter of a bit (a 2-bit codepoint).

   The argument for making this new decapsulation behaviour the default
   for all PHBs is that it doesn't change any expected behaviour that
   existing mechanisms rely on already.  Also, by ending the present
   waste of a codepoint, in the future a use of that codepoint could be
   proposed for all PHBs, even if PCN isn't successfully deployed.

   In practice, if these new decapsulation rules are specified
   straightaway as the normative default for all PHBs, a network
   operator deploying 3-state PCN would be able to request that tunnels
   comply with the latest specification.  Implementers of non-PCN
   tunnels would not need to comply but, if they did, their code would
   be future proofed and no harm would be done to legacy operations.
   Therefore, rather than branching their code base, it would be easiest
   for implementers to make all their new tunnel code comply with this
   specfication, whether or not it was for PCN.  But they could leave
   old code untouched, unless it was for PCN.

   The alternatives are worse.  Implementers would otherwise have to
   provide configurable decapsulation options and operators would have
   to configure all IPsec and IP in IP tunnel endpoints for the
   exceptional behaviour of certain PHBs.  The rules for tunnel
   endpoints to handle both the Diffserv field and the ECN field should
   'just work' when handling packets with any Diffserv codepoint.

Briscoe                Expires September 25, 2009              [Page 39]

Internet-Draft               ECN Tunnelling                   March 2009

Appendix E.  Why Resetting CE on Encapsulation Impedes PCN

   Regarding encapsulation, the section of the PCN architecture
   [I-D.ietf-pcn-architecture] on tunnelling says that header copying
   (RFC4301) allows PCN to work correctly.  Whereas resetting CE
   markings confuses PCN marking.

   The specific issue here concerns PCN excess rate marking
   [I-D.ietf-pcn-marking-behaviour], i.e. the bulk marking of traffic
   that exceeds a configured threshold rate.  One of the goals of excess
   rate marking is to enable the speedy removal of excess admission
   controlled traffic following re-routes caused by link failures or
   other disasters.  This maintains a share of the capacity for traffic
   in lower priority classes.  After failures, traffic re-routed onto
   remaining links can often stress multiple links along a path.
   Therefore, traffic can arrive at a link under stress with some
   proportion already marked for removal by a previous link.  By design,
   marked traffic will be removed by the overall system in subsequent
   round trips.  So when the excess rate marking algorithm decides how
   much traffic to mark for removal, it doesn't include traffic already
   marked for removal by another node upstream (the `Excess traffic
   meter function' of [I-D.ietf-pcn-marking-behaviour]).

   However, if an RFC3168 tunnel ingress intervenes, it resets the ECN
   field in all the outer headers, hiding all the evidence of problems
   upstream.  Thus, although excess rate marking works fine with RFC4301
   IPsec tunnels, with RFC3168 tunnels it typically removes large
   volumes of traffic that it didn't need to remove at all.

Author's Address

   Bob Briscoe
   B54/77, Adastral Park
   Martlesham Heath
   Ipswich  IP5 3RE

   Phone: +44 1473 645196

Briscoe                Expires September 25, 2009              [Page 40]