PCN Working Group                                               D. Satoh
Internet-Draft                                                  Y. Maeda
Intended status: Informational                              O. Phanachet
Expires: September 10, 2009                                      H. Ueno
                                                                  NTT-AT
                                                           March 9, 2009


  Single PCN Threshold Marking by using PCN baseline encoding for both
                   admission and termination controls
                     draft-satoh-pcn-st-marking-01

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on September 10, 2009.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (http://trustee.ietf.org/license-info).
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.






Satoh, et al.          Expires September 10, 2009               [Page 1]


Internet-Draft                 ST Marking                     March 2009


Abstract

   [I-D.ietf.pcn.architecture] defines two rates, admissible and
   supportable, per link that divide PCN traffic load into three states.
   PCN admission control and flow termination mechanisms operate in
   accordance with these three states.  [I-D.ietf.pcn.baseline.encoding]
   defines one bit for packet marking.  This document proposes an
   algorithm for marking and metering by using pre-congestion
   notification (PCN) baseline encoding for both flow admission and flow
   termination.  The ratio of marked packets determines the three link
   states: no packets marked, some packets marked, and all packets
   marked.  To achieve this marking behaviour, we use two token buckets.
   One is not used for marking but for a marking switch; the other is
   used for marking.  The token bucket for marking has two thresholds.
   One is TBthreshold.threshold, already defined in [I-D.ietf-pcn-
   marking-behaviour], and the other is a new threshold, which is set to
   be the number of bits of a metered-packet smaller than the token
   bucket size.  Therefore, the new threshold is larger than
   TBthreshold.threshold.  If the amount of tokens is less than
   TBthreshold.threshold, all the packets are marked as defined in
   [I-D.ietf-pcn-marking-behaviour].  If the amount of tokens is less
   than the new threshold and greater than TBthreshold.threshold, one-
   Nth packets are marked.  We evaluated the performance of admission
   control and flow termination using a simulation.  For admission
   control, the results show that the performance of the algorithm was
   almost the same as, but slightly inferior to, that of CL
   [draft-briscoe-tsvwg-cl-phb-03].  For flow termination, the
   performance of the algorithm was almost the same as CL when the load
   was 1.2 times the supportable rate, but it was superior to CL when
   the load was high (two times the supportable rate).  Furthermore, in
   the algorithm, over termination percentages of all the bottleneck
   links are almost the same in the case of multi-bottleneck.  In CL,
   the over termination percentages of all the bottleneck links are
   different and those at upstream bottleneck links are higher than
   those at downstream bottleneck links because of accumulation of
   marked packets.















Satoh, et al.          Expires September 10, 2009               [Page 2]


Internet-Draft                 ST Marking                     March 2009


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  5
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Single Threshold-Marking . . . . . . . . . . . . . . . . . . .  5
     3.1.  Operation at PCN-interior-node . . . . . . . . . . . . . .  6
     3.2.  Operation at PCN-egress-node . . . . . . . . . . . . . . . 10
       3.2.1.  Operation for Admission Decision . . . . . . . . . . . 10
       3.2.2.  Operation for Flow Termination Decision  . . . . . . . 10
     3.3.  Operation at the PCN-ingress-node  . . . . . . . . . . . . 11
       3.3.1.  Admission Decision . . . . . . . . . . . . . . . . . . 11
       3.3.2.  Flow Termination Decision  . . . . . . . . . . . . . . 11
   4.  Admission Issues . . . . . . . . . . . . . . . . . . . . . . . 11
     4.1.  Effect of Synchronization  . . . . . . . . . . . . . . . . 11
   5.  Termination Issues . . . . . . . . . . . . . . . . . . . . . . 12
     5.1.  Effect of dropping packets to Termination  . . . . . . . . 12
     5.2.  Effect of multi-bottleneck . . . . . . . . . . . . . . . . 13
     5.3.  Speed and Accuracy of Termination  . . . . . . . . . . . . 13
   6.  Performance evaluation . . . . . . . . . . . . . . . . . . . . 13
   7.  Impact on PCN marking behaviour  . . . . . . . . . . . . . . . 13
   8.  Changes from -00 version . . . . . . . . . . . . . . . . . . . 14
   9.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 14
   10. Appendix A: Simulation Setup and Environment . . . . . . . . . 14
     10.1. Network and Signaling Models . . . . . . . . . . . . . . . 15
     10.2. Traffic Models . . . . . . . . . . . . . . . . . . . . . . 17
     10.3. Performance Metrics  . . . . . . . . . . . . . . . . . . . 18
     10.4. Parameter Settings for STM . . . . . . . . . . . . . . . . 19
     10.5. Parameter Settings for CL  . . . . . . . . . . . . . . . . 19
     10.6. Simulation Environment . . . . . . . . . . . . . . . . . . 20
   11. Appendix B: Admission Control Simulation . . . . . . . . . . . 20
     11.1. Parameter Settings for Admission Control . . . . . . . . . 20
     11.2. Sensitivity to EWMA Weight and CLE Threshold . . . . . . . 20
     11.3. Basic Evaluation . . . . . . . . . . . . . . . . . . . . . 22
     11.4. Effect of Ingress-Egress Aggregation . . . . . . . . . . . 23
       11.4.1. CBR  . . . . . . . . . . . . . . . . . . . . . . . . . 23
       11.4.2. VBR  . . . . . . . . . . . . . . . . . . . . . . . . . 24
       11.4.3. SVD  . . . . . . . . . . . . . . . . . . . . . . . . . 25
     11.5. Effect of Multi-bottleneck . . . . . . . . . . . . . . . . 26
     11.6. Fairness among Different Ingress-Egress Pairs  . . . . . . 26
   12. Appendix C: Flow termination . . . . . . . . . . . . . . . . . 27
     12.1. Basic evaluation . . . . . . . . . . . . . . . . . . . . . 28
     12.2. Effect of Ingress-Egress Aggregation . . . . . . . . . . . 28
       12.2.1. CBR  . . . . . . . . . . . . . . . . . . . . . . . . . 29
       12.2.2. VBR  . . . . . . . . . . . . . . . . . . . . . . . . . 29
       12.2.3. SVD  . . . . . . . . . . . . . . . . . . . . . . . . . 30
     12.3. Effect of Multi-bottleneck . . . . . . . . . . . . . . . . 31
   13. Appendix D: Why is TBthreshold.shallow.threshold set to be
       smaller than the token bucket size by the bit-size of a



Satoh, et al.          Expires September 10, 2009               [Page 3]


Internet-Draft                 ST Marking                     March 2009


       metered packet?  . . . . . . . . . . . . . . . . . . . . . . . 33
   14. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 35
   15. Security Considerations  . . . . . . . . . . . . . . . . . . . 35
   16. Informative References . . . . . . . . . . . . . . . . . . . . 35
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36














































Satoh, et al.          Expires September 10, 2009               [Page 4]


Internet-Draft                 ST Marking                     March 2009


1.  Introduction

   Pre-congestion notification (PCN) provides information to support
   admission control and flow termination in order to protect the
   quality of service (QoS) of inelastic flows.  Although several
   algorithms (e.g., [I-D.briscoe-tsvwg-cl-phb], [I-D.charny-pcn-single-
   marking], and [I-D.babiarz-pcn-3sm]) have been proposed to achieve
   PCN, only the single marking algorithm (SM) [I-D.charny-pcnsingle-
   marking] meets the requirement of baseline encoding.

   This document proposes an algorithm for marking and metering by using
   PCN baseline encoding for both flow admission and flow termination.
   Our algorithm uses PCN-threshold marking while SM uses PCN-excess-
   traffic marking.  Although our algorithm uses an elaborate mechanism,
   it chooses PCN-admissible and PCN-supportable rates independently,
   and it explicitly detects whether PCN traffic is greater than the
   PCN-supportable-rate.  Furthermore, our algorithm is little affected
   by synchronization.  Therefore, it can avoid degradation of admission
   accuracy caused by synchronization.


2.  Terminology

   The terminology used in this document conforms to the terminology of
   [ID.ietf-pcn-architecture] and [I-D.ietf-pcn-marking-behaviour].


3.  Single Threshold-Marking

   We describe an algorithm of marking and metering by using PCN
   baseline encoding for both flow admission and flow termination.  This
   algorithm uses only the PCN-threshold-rate as the PCN-supportable-
   rate and does not use the PCN-excess-rate.  We show a schematic of
   how the PCN admission control and flow termination mechanisms operate
   as the rate of PCN-traffic increases for a PCN-domain with three
   types of ratios of PCN-threshold-marked packets and three states of
   marking ratio in Fig. 1.  Only one bit can be used for marking when
   PCN baseline encoding is used.  We use one bit marking for the
   marking ratio to distinguish the three states.  As shown in Fig. 1,
   no packets are PCN-marked at a rate less than the PCN-admissible-
   rate, some packets are PCN-marked at rates between the PCN-
   admissible- and PCN-supportable-rates, and all the packets are PCN-
   marked at rates greater than the PCN-supportable-rate.  Admission
   control using this marking stops traffic from being admitted when the
   fraction of marked traffic for an ingress-egress aggregate exceeds a
   configured threshold: the congestion level estimate (CLE).  When the
   egress receives a certain sequential marked packets, it sends a
   message of the receiving rate to an ingress.  Then the ingress



Satoh, et al.          Expires September 10, 2009               [Page 5]


Internet-Draft                 ST Marking                     March 2009


   terminates flows based on the information of the sending and
   receiving rates.

   The remainder of this section describes the possible operation of the
   system.


          PCN traffic rate
           100%
             |                      Terminate some
             |All the packets       admitted flows
             |PCN-threshold-marked       &          Marking.frequency: 1
  PCN-       |                      Block new flows
  supportable|
   rate      +-------------------------------------------------------
  (PCN-      |
  threshold- |
  rate)      | Some packets            Block     Marking.frequency: 1/N
             | PCN-threshold-marked   new flows
  PCN-       |
  admissible |
  rate       +-------------------------------------------------------
             |
             | No packets          Admit new flows   Not defined
             | PCN-marked
             |
          0% +-------------------------------------------------------


   Figure 1: Ratio of PCN-threshold-marked packets, control operations,
                           and Marking.frequency

3.1.  Operation at PCN-interior-node

   We explain here how to distinguish the three link states.  The single
   threshold-marking algorithm (STM) uses two token buckets.  One is not
   used for marking but as a marking switch, which we call the switch
   token bucket (SwTB).  The other is used for marking, which we call
   the marking token bucket (MkTB).  We also explain how to mark by
   using two token buckets in cooperation.

   SwTB can be used when the traffic is below the admissible rate so
   that no packets are marked.  Metering and marking using SwTB is
   similar to threshold metering and marking, although there is a
   difference between actual marking and having the marking switch ON.
   SwTB has one threshold, which is termed the SwTB threshold.  Tokens
   of SwTB are added at the PCN-admissible-rate.  Tokens are removed
   equal to the bit-size of the metered packet.  These additions and



Satoh, et al.          Expires September 10, 2009               [Page 6]


Internet-Draft                 ST Marking                     March 2009


   removals are independent of MkTB.  If the metered traffic is
   sustained at a level greater than the PCN-admissible-rate, the tokens
   are less than the SwTB threshold.  When the tokens are less than the
   SwTB threshold, the marking switch of SwTB is set to ON and the
   metered packet is threshold-marked in accordance with the meter based
   on MkTB.  Otherwise, the marking switch of SwTB is OFF.  When the
   marking switch is OFF, no packet is marked regardless of the meter of
   MkTB.  Thus, the state of admitting new flows (seen in Fig. 1) is
   achieved.

   MkTB can be used to mark packets in accordance with traffic rates.
   Metering and marking using MkTB is modified threshold metering and
   marking.  MkTB has two thresholds.  One, termed
   TBthreshold.shallow.threshold, is set to be smaller than the token
   bucket size by the bit-size of a metered packet.  The other is
   TBthreshold.threshold in [I-D. pcn-marking-behaviour].  Regardless of
   the marking switch being ON or OFF, tokens of MkTB are added at the
   PCN-supportable-rate and tokens are removed equal to the bit-size of
   the metered packet.  These additions and removals are independent of
   SwTB.

   If the metered traffic is sustained at a level greater than the PCN-
   supportable-rate, the tokens are less than the TBthreshold.threshold.
   When the tokens are less than TBthreshold.threshold, all the metered
   packets are threshold-marked.  The marking.frequency in this state is
   1 because all the metered packets are marked.  Relations among
   marking.frequency, marking behaviour, and PCN mechanisms are shown in
   Fig. 1.  Thus, the state of terminating certain admitted flows and
   blocking new flows (seen in Fig. 1) is achieved.

   If the metered traffic is between the PCN-admissible-rate and PCN-
   supportable-rate, the tokens are greater than the
   TBthreshold.threshold.  If the metered traffic has rate fluctuations,
   the tokens are often less than TBthreshold.shallow.threshold.  Note
   that the probability that the tokens are less than
   TBthreshold.shallow.threshold is approximately the ratio between the
   rate of the metered traffic and the PCN-supportable-rate [see
   Appendix D].  If the tokens are less than
   TBthreshold.shallow.threshold (the tokens are less than MkTB.size
   before the tokens of the arrived packet.size are removed), a metered
   packet is PCN-threshold-marked for every one-Nth.  Note that one-Nth
   arrived packets are not marked.  The marking.frequency in this state
   is 1/N. This marking with marking.frequency 1/N avoids accommodating
   marking through multi-bottleneck links to distinguish partial marking
   from marking of all packets.

   This marking is expressed by the following pseudo code:




Satoh, et al.          Expires September 10, 2009               [Page 7]


Internet-Draft                 ST Marking                     March 2009


   Input: pcn packet
   SwTB.fill = min(SwTB.size, SwTB.fill + SwTB.rate*(now - lastUpdate));
   MkTB.fill = min(MkTB.size, MkTB.fill + MkTB.rate*(now - lastUpdate));
   //add tokens to the two token buckets
   SwTB.fill = max( 0 , SwTB.fill - packet.size);
   MkTB.fill = max( 0 , MkTB.fill - packet.size);
   //remove tokens from the two token buckets
   IF (SwTB.fill <= SwTB.threshold) THEN  //marking switch is ON
       IF  (MkTB.fill < MkTBthreshold.shallow.threshold) THEN
           IF (MkTB.fill > MkTBthreshold.threshold) THEN
               markCnt++;
               IF mod(markCnt, N) == 0 THEN //Marking.frequency = 1/N
                  markCnt = 0;
                  packet.mark = TM;
               ENDIF
           ELSE
               markCnt = 0;
               packet.mark = TM;
           ENDIF
       ENDIF
   ELSE //marking switch is OFF
   ENDIF
   output: void


   It can be rewritten as follows.

























Satoh, et al.          Expires September 10, 2009               [Page 8]


Internet-Draft                 ST Marking                     March 2009


   Input: pcn packet

   SwTB.fill = min(SwTB.size, SwTB.fill + SwTB.rate*(now - lastUpdate));
   MkTB.fill = min(MkTB.size, MkTB.fill + MkTB.rate*(now - lastUpdate));
   //add tokens to the two token buckets
   lastUpdate = now;
   markOpe = ((SwTB.fill - packet.size) >= SwTB.Threshold) ? false:true;
   IF (markOpe) THEN
      IF (MkTB.fill < MkTB.size) THEN
         markCnt ++;
         IF (MkTB.fill < MkTB.Threshold) THEN
            packet.mark = TM;
            markCnt = 0;
         ELSE IF (markCnt == N) THEN
            packet.mark = TM;
            markCnt = 0;
         ENDIF
      ENDIF
   ENDIF
   SwTB.fill = max( 0 , SwTB.fill - packet.size);
   MkTB.fill = max( 0 , MkTB.fill - packet.size);
   //remove tokens from each token bucket
   Output: void


   Here, SwTB.fill and MkTB.fill (TBthreshold.fill in [I-D.pcn-marking-
   behaviour]) represent the amount of tokens in the SwTB and MkTB,
   SwTB.size and MkTB.size (TBthreshold.max in [I-D.pcn-marking-
   behaviour]) represent the maximum values of SwTB.fill and MkTB.fill,
   and SwTB.threshold and MkTB.threshold represent the SwTB threshold
   and MkTB threshold (TBthreshold.threshold), respectively.  Tokens of
   SwTB are added at the SwTB.rate whose value is the PCN-admissible-
   rate.  Tokens of MkTB are added at the MkTB.rate whose value is the
   PCN-supportable-rate (PCN-threshold-rate).  The time variables now
   and lastUpdate are the current time and the time when the fill state
   of TB was last updated, respectively.  The byte of an arrived packet
   is packet.size.  TM represents the state of the threshold-marked
   packet, packet.mark represents the state of the mark of a packet, and
   markCnt represents the number of marked packets.

   The relation between PCN traffic rate and marking ratio is shown Fig.
   2, where AR and SR represent PCN-admissible- and PCN-supportable-
   rates.








Satoh, et al.          Expires September 10, 2009               [Page 9]


Internet-Draft                 ST Marking                     March 2009


                 ^
 marking ratio   |
               1 +                                     +----------------
                 |                                     |
                 |                                     |
                 |                                     |
                 |                                     |
                 |                                     |
             1/N +                                    _|
                 |                                  _-
                 |                                _-
                 |                              _-
                 |                            _-
                 |                          _-
                 |                        _-
       AR/(N*SR) +                     _-
                 |                    |
                 |                    |
                 |                    |
                 |                    |
                 ---------------------+-----------------+-------------->
                0                     AR                SR   PCN traffic
                                                                   rate


       Figure 2: Relation between PCN traffic rate and marking ratio

3.2.  Operation at PCN-egress-node

3.2.1.  Operation for Admission Decision

   A PCN-egress-node measures the ratio of PCN-threshold-marked packets
   on a per-ingress basis and reports to the PCN-ingress-node the
   congestion level estimate (CLE), which is the fraction of the marked
   traffic received from the ingress node and is exactly the same as
   that of CL, SM, and three state PCN marking algorithms (3sM).

3.2.2.  Operation for Flow Termination Decision

   The PCN-egress-node always measures the receiving rate per PCN-
   ingress-node regardless of whether receiving marked packets or not.
   When the PCN-egress-node receives L-sequential marked packets from a
   PCN-ingress-node, it sends a packet of information about the
   receiving rate to the PCN-ingress-node at an interval that is shorter
   than the measuring interval of the PCN-ingress-node.  If the PCN-
   egress-node receives M-sequential unmarked packets, it sends a packet
   for canceling termination to the PCN-ingress-node.




Satoh, et al.          Expires September 10, 2009              [Page 10]


Internet-Draft                 ST Marking                     March 2009


3.3.  Operation at the PCN-ingress-node

3.3.1.  Admission Decision

   Just as in CL and SM, the admission decision is based on the CLE.
   The ingress node stops admitting new flows if the CLE is greater than
   a predefined threshold (CLE threshold).  The CLE threshold is chosen
   under the following maximum value.  The maximum of the CLE threshold
   MUST be

              CLE threshold = (1/N)*rho  ,                 (1)


   where rho represents the minimum ratio between the PCN-admissible-
   rate and PCN-supportable-rate via all the links between the PCN-
   ingress and egress nodes and N is the denominator of
   marking.frequency 1/N. Note that the marking ratio approaches the
   maximum CLE threshold described above when the load is the PCN-
   admissible-rate at a bottleneck link via all the links between the
   PCN-ingress and egress nodes as the number of superposition of flows
   approaches infinity, as described in Appendix D.

3.3.2.  Flow Termination Decision

   As soon as a PCN-ingress-node receives a packet with information
   about a receiving rate from a PCN-egress-node, it starts measuring
   the sending rate to the PCN-egress-node.  It receives the packet
   multiple times during the measuring interval and maintains the
   minimum value of the receiving rate.  At the end of the measurement
   interval, it terminates flows whose bandwidths are the same as the
   sending rate - (1-y)* min(receiving rate, sending rate).  Because
   flow termination affects the receiving rate after transmission delay,
   the receiving rate can be larger than the sending rate.  Therefore,
   min(receiving rate, sending rate) is used.  The value of y is small
   enough to tolerate.  It stops terminating flows or measuring the
   sending rate as soon as it receives the packet for canceling
   termination from the PCN-egress-node.  This termination takes more
   time when the PCN-supportable-rate is much lower than the physical
   rate because the sending rate is almost the same as the receiving
   rate even under a congested situation.


4.  Admission Issues

4.1.  Effect of Synchronization

   CL and SM suffer from synchronization [I-D.zhang-pcn-performance-
   evaluation, I-D.charny-pcn-single-marking].  Synchronization is



Satoh, et al.          Expires September 10, 2009              [Page 11]


Internet-Draft                 ST Marking                     March 2009


   defined in [I-D.zhang-pcn-performance-evaluation] as the phenomenon
   of some flows having all their packets marked, while other flows have
   none of their packets marked.  This phenomenon easily occurs in the
   case of CBR flows.  If the duration time of flows is infinity, a
   superposition of homogeneous CBR flows shows periodic behaviour for
   every interval between two sequential packets of a CBR flow.  Even if
   the duration time is limited, the superposition of homogeneous CBR
   flows shows periodic behaviour when the number of flows is not
   changed.  When excess-traffic-marking is applied, the periodic
   behaviour of the packets leads to periodic marking behaviour because
   the periodic behaviour of the packets leads to the periodic behaviour
   of tokens, which causes periodic marking behaviour.  The period of
   marking behaviour is the same as that of the behaviour of the
   packets.  The periodic marking behaviour lasts as long as the
   periodic behaviour of packets.  Synchronization makes the admission
   and termination of CL and SM less accurate [I-D.zhang-pcn-
   performance-evaluation, I-D.charny-pcn-single-marking].

   Although STM uses partial marking for admission control, the marking
   is not excess-traffic-marking but a kind of threshold-marking.  It is
   true that the periodic behaviour of packets leads to the periodic
   behaviour of tokens, but the period of the marking behaviour is
   different from that of behaviour of packets when the
   marking.frequency is not 1.  The period of marking behaviour is
   longer than the periodic behaviour of packets.  Thus, no flows have
   all their packets marked.  Because the marking of STM is threshold
   marking, more marking occurs for metered packets when 1 is chosen as
   N at the marking of STM than when excess-traffic-marking is applied.
   The ratio of marked packets to all the packets is approximately the
   same as that of traffic load to the PCN-supportable-rate as shown in
   Appendix D.  Thus, the ratio of flows that have no packets marked in
   STM is smaller than that in the excess-traffic-marking although some
   flows have no packets marked at all.  Furthermore, when PCN traffic
   is close to the admissible rate, no packets are marked in a short
   interval because of traffic fluctuations.  A metered packet is not
   marked when the rate of the PCN traffic is less than the admissible
   rate in the marking of STM.  This breaks the periodic marking
   behaviour.  Almost the same performance results were achieved with
   and without randomized traffic, which supports the description above.
   Therefore, STM is little affected by synchronization.


5.  Termination Issues

5.1.  Effect of dropping packets to Termination

   When the PCN traffic is greater than the PCN-supportable-rate, all
   the packets are threshold-marked in STM.  Dropping packets themselves



Satoh, et al.          Expires September 10, 2009              [Page 12]


Internet-Draft                 ST Marking                     March 2009


   does not influence the performance of STM termination.  Therefore, a
   node can drop any packets.  Preferential dropping packets is not
   necessary.

5.2.  Effect of multi-bottleneck

   In the case of multi-bottleneck links, over termination percentages
   of STM are almost the same among all the bottleneck links although CL
   makes over termination percentages of upstream bottleneck links worse
   because CL is affected by accumulation of marking.

5.3.  Speed and Accuracy of Termination

   Because a PCN-ingress-node terminates flows every measurement
   interval, the amount of termination can be roughly evaluated.
   Therefore, by using the ratio between the physical link speed and the
   PCN-supportable-rate, the value of y is estimated roughly in order to
   finish termination within a certain interval.  However, this
   termination has a trade-off between termination speed and accuracy.


6.  Performance evaluation

   We compared the performance of STM with that of CL.  For admission,
   an over admission percentage was chosen, and for flow termination,
   over termination and termination time were chosen as performance
   metrics.

   For admission control, the performance of STM was almost the same as
   that of CL, although STM was slightly inferior to CL.

   For flow termination, when the load was slightly larger than the PCN-
   supportable-rate and the traffic type was CBR, the over termination
   percentages of STM were inferior to those of CL.  However, when the
   traffic type was VBR or SVD, the over termination percentages of STM
   were almost the same as those of CL.  When the load was much larger
   than the PCN-supportable-rate, our simulation showed that the over
   termination percentages of STM were superior to those of CL.  STM
   took more termination time than CL because STM terminates flows
   little by little.  In the multi-bottleneck case, over termination
   percentages of STM are almost the same among all the bottleneck liks
   because STM is not affected by accumulation of marking.


7.  Impact on PCN marking behaviour

   The goal of this section is to propose two minor changes to the PCN-
   marking-behaviour framework as currently described in [I-D.ietf-pcn-



Satoh, et al.          Expires September 10, 2009              [Page 13]


Internet-Draft                 ST Marking                     March 2009


   marking-behaviour] in order to enable the single threshold-marking
   approach.  We propose additions of a threshold meter function with
   two thresholds and a combination of the meter functions.  A new
   threshold of the threshold meter function is larger than
   TBthreshold.threshold.  If the amount of tokens is less than the new
   threshold of the threshold meter function, the metered packets are
   partially marked and not all marked.  This marking function includes
   the ramp-marking in [I-D.briscoe-tsvwg-cl-phb].  The combination of
   SwTB and MkTB is an example of the combination of the meter
   functions.


8.  Changes from -00 version

   o  Minor changes to flow termination (Section 3)

   o  Added description of effect of synchronization to STM (Section 4)

   o  Added discussion of sensitivity to EWMA weight and CLE threshold
      (Appendix B)

   o  Added simulation results for admission (Appendix B)

   o  Added simulation results for flow termination (Appendix C)

   o  Added explanation about marking using TBthrehold.shallow.threshold
      and queueing theory (Appendix D)

   o  Other minor edits


9.  Acknowledgements

   This research was partially supported by the National Institute of
   Information and Communications Technology (NICT), Tokyo, Japan.

   We thank Mr. Takayuki Uchida at U-software Corporation for helping
   with the simulations and Professor Shigeo Shioda at Chiba University
   for fruitful discussions.  We also thank Assistant Professor Michael
   Menth at the University of Wuerzburg for his careful reading of this
   draft and fruitful discussions.


10.  Appendix A: Simulation Setup and Environment







Satoh, et al.          Expires September 10, 2009              [Page 14]


Internet-Draft                 ST Marking                     March 2009


10.1.  Network and Signaling Models

   We used the three types of network topologies shown in the figures
   below for simulations.  They are the same as those in [I-D.charny-
   pcn-single-marking] and [I-D.zhang-pcn-performance-evaluation], and
   the first two figures are the same as those in [I-D. briscoe-tsvwg-
   cl-phb].  The first type of network topology is single link (Fig.
   A.1).  The second type is a multi-link network with a single
   bottleneck (termed "RTT") (Fig. A.2).  The third type is a range of
   multi-bottleneck topologies (termed "Parking Lot") (Fig. A.3).

   A single link between an ingress and an egress node is shown in Fig.
   A.1, where all flows enter at node A and depart from node B. This
   topology is used to basically verify the behaviour of the algorithms
   with respect to a single ingress-egress aggregate (IEA) in isolation.

                                  A --- B


                Figure A.1: Simulated single-link network.

                                 A
                                 \
                                 B - D - F
                                 /
                                 C


                 Figure A.2: Simulated multi-link network.

                 A--B--C    A--B--C--D     A--B--C--D--E--F
                 |  |  |    |  |  |  |     |  |  |  |  |  |
                 |  |  |    |  |  |  |     |  |  |  |  |  |
                 D  E  F    E  F  G  H     G  H  I  J  K  L
                   (a)         (b)                (c)


                 Figure A.3: Simulated multi-link network.

   As shown in Fig. A.2, a set of ingresses (A, B, and C) are connected
   to an interior node in the network (D).  This topology is used to
   study the behaviour of the algorithm where many IEAs share a single
   bottleneck link.  The number of ingresses varied in different
   simulation experiments from 2 - 1800.  All links generally have
   different propagation delays.  Thus, these propagation delays were
   chosen randomly in the range of 1 - 100 ms.  This node D in turn is
   connected to the egress (F).  In this topology, different sets of
   flows between each ingress and the egress converge on a single link



Satoh, et al.          Expires September 10, 2009              [Page 15]


Internet-Draft                 ST Marking                     March 2009


   D-F, where the PCN algorithm is enabled.  The capacities of the
   ingress links are not limiting, and hence no PCN is enabled on them.
   The bottleneck link D-F was modeled with a 10-ms propagation delay in
   all simulations.  Therefore, the range of round-trip delays in the
   experiments was from 22 - 220 ms.

   Another type of network of interest is that with a parking lot (PLT)
   topology, which has multi-bottleneck links.  The simplest PLT with
   two bottlenecks is illustrated in Fig. A.3(a).  A traffic matrix with
   this network on this topology is as follows.

   o  an aggregate of "2-hop" flows entering the network at A and
      leaving at C (via the two links A-B-C)

   o  an aggregate of "1-hop" flows entering the network at D and
      leaving at E (via A-B)

   o  an aggregate of "1-hop" flows entering the network at E and
      leaving at F (via B-C)

   In the 2-hop PLT shown in Fig. A.3(a), the points of congestion are
   links A-B and B-C.  The capacities of all the other links are not
   limited.  We also experimented with larger PLT topologies with three
   bottlenecks (Fig. A.3(b)) and five bottlenecks (Fig. A.3(c)).  In all
   cases, we simulated one ingress-egress pair that carried the
   aggregate of "long" flows traversing all N bottlenecks (where N is
   the number of bottleneck links in the PLT topology) and N ingress-
   egress pairs that carried flows traversing a single bottleneck link
   and exited at the next "hop".  In all cases, only the "horizontal"
   links in Fig. A.3 were the bottlenecks, with non-limited capacities
   of all "vertical" links.  We named the bottleneck IDs in the order of
   lowest (upstream) to highest (downstream), e.g., bottleneck ID 1 is
   the upstream link between nodes A and B and bottleneck ID 2 is the
   downstream link between nodes B and C in Fig. A.3 (a).  The
   propagation delays for all links in all PLT topologies were set to 1
   ms.

   Our simulations concentrated primarily on the range of capacities of
   "bottleneck" links with sufficient aggregation - above 128 Mbps for
   voice and 2.4 Gbps for SVD.  Higher link speeds will generally result
   in higher levels of aggregation and hence generally better
   performance of measurement-based algorithms.  Therefore, it seems
   reasonable to believe that the studied link speeds do provide
   meaningful evaluation targets.

   In the simulation model, a call request arrives at the ingress, which
   immediately sends a message to the egress.  The message arrives at
   the egress after the propagation time plus link processing time (but



Satoh, et al.          Expires September 10, 2009              [Page 16]


Internet-Draft                 ST Marking                     March 2009


   no queuing delay).  When the egress receives this message, it
   immediately responds to the ingress with the current CLE.  If the CLE
   is below the specified CLE threshold, the call is admitted.  If
   otherwise, it is rejected.  An admitted call sends packets in
   accordance with one of the chosen traffic models for the duration of
   the call (see next section).  The propagation delay from the source
   to the ingress and from the destination to the egress is assumed to
   be negligible and is not modeled.

   In the admission control simulation, the PCN-admissible-rate and PCN-
   supportable rates were half the link speed and 90% of the link speed,
   respectively, and in the flow termination simulation, the PCN-
   admission-rate and PCN-supportable-rate were 20% and 40% of the link
   speed in all the links.  The actual queue size was 80,000 packets,
   which was not the byte size because queue size was set by only the
   number of packets in the NS2 simulator.

10.2.  Traffic Models

   We use the same types of traffic as those of [I-D.briscoe-tsvwg-cl-
   phb], [I-D.charny-pcn-single-marking], and [I-D.zhang-pcn-
   performance-evaluation].  These are CBR voice, on-off traffic
   approximating voice with silence compression (termed "VBR"), and on-
   off traffic with higher peak and mean rates (we termed the latter
   "synthetic video" (SVD)).  On-off traffic (VBR and SVD) is described
   with a two-state Markov chain, and on and off periods were
   exponentially distributed with the specified mean.

   Each flow arrives according to the Poisson process.  The distribution
   of flow duration was chosen to be exponentially distributed with a
   mean of 1 min, regardless of the traffic type, which is the same as
   that in [I-D.briscoe-tsvwg-cl-phb] and [I-D.charny-pcn-single-
   marking] in admission simulation.  The flow duration was infinity in
   termination simulation in order to observe the effect by only flow
   termination.

   Traffic parameters for each type are summarized below.

   CBR voice

   o  Packet length: 160 bytes

   o  Packet inter-arrival time: 20ms ((160*8)/(64*1000) s)

   o  Average rate: 64 Kbps

   On-off traffic approximating voice with silence compression




Satoh, et al.          Expires September 10, 2009              [Page 17]


Internet-Draft                 ST Marking                     March 2009


   o  Packet length: 160 bytes

   o  Packet inter-arrival time during on period: 20 ms

   o  Long-term average rate: 21.76 Kbps

   o  On period mean duration: 340ms; during the on period traffic is
      sent with the CBR voice parameters described above

   o  Off period mean duration: 660 ms; no traffic is sent for the
      duration of the off period

   SVD

   o  Packet length: 1500 bytes

   o  Packet inter-arrival time during on period: 1 ms

   o  Long term average rate: 4 Mbps

   o  On period mean duration: 340 ms

   o  Off period mean duration: 660 ms

10.3.  Performance Metrics

   In all our experiments, we used the percent deviation of the mean
   rate of the expected load level as a performance metric.  We term
   these "over-admission" and "over-termination" percentages, depending
   on the type of experiment.

   In our experiments of admission control, we compared the actual
   achieved average throughput to the desired traffic load (PCN-
   admissible-rate).  The desired traffic load is not the exact
   admissible rate because a signal packet for a new flow is sent for
   admission.  Therefore, the desired traffic load is the admissible
   rate minus the amount of signal packets for non-admitted flows.

   In our experiments of flow termination, we compared the actual
   achieved average throughput to the desired traffic load (PCN-
   supportable-rate) and measured termination time.  Because a
   termination discontinues and restarts when the traffic type is bursty
   traffic such as VBR and SVD, a termination was regarded as the last
   termination when it takes more than 1 s until the next termination.







Satoh, et al.          Expires September 10, 2009              [Page 18]


Internet-Draft                 ST Marking                     March 2009


10.4.  Parameter Settings for STM

   All the simulations were run with the following values.

   o  SwTB.size = 20 ms at PCN-admissible-rate

   o  SwTB.threshold = 10 ms at PCN-admissible-rate

   o  MkTB.size = 10 ms at PCN-supportable-rate

   o  MkTB.threshold = 8 ms at PCN-supportable-rate

   o  Marking.frequency = 1/3 and 1

   o  CLE threshold is varied in Table B.1

   o  EWMA weight is varied in Table B.1

   o  Number of sequential marked packets for termination = 100

   o  Number of sequential unmarked packets for stopping termination =
      20

   o  Extra percentage of receiving or sending rate more than the
      difference between sending and receiving rates for termination at
      one time = 5.0

   o  Interval between sending the receiving rate = 33.3 ms

   o  Interval for measuring sending rate = 100 ms

10.5.  Parameter Settings for CL

   All the simulations were run with 5 ms at the PCN-supportable-rate as
   the token bucket size for termination and with the following virtual
   queue thresholds.

   o  Min-marking-threshold: 5 ms at the PCN-admissible-rate

   o  Max-marking-threshold: 15 ms at the PCN-admissible-rate

   o  Virtual-queue-upper-limit: 20 ms at the PCN-admissible-rate

   The virtual-queue-upper-limit puts an upper bound on how much the
   virtual queue can grow.  In the admission control simulation, the CLE
   threshold and EWMA weight were set as 0.05 and 0.3, respectively.
   All of the parameters of the virtual queue were the same as those in
   [I-D.briscoe-tsvwg-cl-phb], [I-D.charny-pcn-single-marking], and



Satoh, et al.          Expires September 10, 2009              [Page 19]


Internet-Draft                 ST Marking                     March 2009


   [I-D.zhang-pcn-performance-evaluation].  The value of 5 ms at the
   PCN-supportable-rate as the token bucket size corresponds to 500
   packets in the case of CBR, 170 packets in the case of VBR, and 666
   packets in the cases of SVD.  The token bucket depth was set to 64
   packets for CBR, and for on-off traffic, to 128 or higher (in
   [I-D.briscoe-tsvwg-cl-phb]) and to 256 packets (in [I-D.zhang-pcn-
   performance-evaluation]).

10.6.  Simulation Environment

   We used NS2 for our simulation experiments.  Simulations were run on
   a Dell Power Edge 1950, Intel Quad-core Xeon 2.66GHz, 32GB RAM
   computer running Red Hat Enterprise Linux (64bit).


11.  Appendix B: Admission Control Simulation

11.1.  Parameter Settings for Admission Control

   We evaluated over-admission percentages when the load of the
   bottleneck was five times the admissible rate, i.e., 2.5 times the
   link speed.  The simulation time was 300 s, and simulation results
   during the time interval between 200 and 300 s were used for
   performance metrics to obtain the results in the steady state.

   Egress measurement parameters for STM: In our simulations, CLE
   threshold was varied in Table B.1.  The CLE was computed as an
   exponential weighted moving average (EWMA).  The weight was varied in
   Table B.1.  The CLE was computed on a per-packet basis.

   Egress measurement parameters for CL: In our simulations, the chosen
   CLE threshold was 0.05.  The CLE was computed as an exponential
   weighted moving average (EWMA) with a weight of 0.3.  The CLE was
   computed on a per-packet basis.  These values were the same as those
   in Sect. 8.2.3 in [I-D.charny-pcn-single-marking].

11.2.  Sensitivity to EWMA Weight and CLE Threshold

   We simulated admission control of STM when the CLE threshold and EWMA
   weight were the six cases shown in Table B.1.  The results of case 1
   were shown in the previous draft.  We simulated the admission control
   five times for each case.  The smallest and largest over-admission
   percentage values of the averages of the five simulations are shown
   in Table B.2.  The smallest and largest values of all five links are
   shown when the topology was PLT(c).  The smallest and largest values
   can be over-admission percentages in a different link.  Case 1, whose
   results were shown in the previous draft, was the worst case of the
   six in all of the simulations from the viewpoint of absolute



Satoh, et al.          Expires September 10, 2009              [Page 20]


Internet-Draft                 ST Marking                     March 2009


   deviation from no over- and under-admission.  The results of case 6
   are shown in the following sections because that case was the best.


                    ------------------------------------
                    Case | CLE threshold | EWMA weight
                    ------------------------------------
                     1   |     0.05      |    0.0005
                    ------------------------------------
                     2   |     0.05      |    0.01
                    ------------------------------------
                     3   |     0.05      |    0.03
                    ------------------------------------
                     4   |     0.05      |    0.3
                    ------------------------------------
                     5   |     0.00001   |    0.3
                    ------------------------------------
                     6   |     0.000001  |    0.3
                    ------------------------------------


       Table B.1: CLE thresholds and EWMA weights in our simulation





























Satoh, et al.          Expires September 10, 2009              [Page 21]


Internet-Draft                 ST Marking                     March 2009


                    ------------------------------------
                    Traffic| Topo. | Smallest | Largest
                    ------------------------------------
                           |S.Link |  0.028   |  0.285
                           |----------------------------
                           |RTT10  |  0.585   |  0.844
                           |----------------------------
                     CBR   |RTT70  | -1.656   |  0.942
                           |----------------------------
                           |RTT600 | -5.218   |  0.234
                           |----------------------------
                           |RTT1000| -6.030   | -0.179
                           |----------------------------
                           |PLT(c) |  0.005   |  0.103
                    ------------------------------------
                           |S.Link |  0.979   |  1.300
                           |----------------------------
                           |RTT10  | -0.753   |  1.562
                           |----------------------------
                     VBR   |RTT70  | -3.249   |  1.462
                           |----------------------------
                           |RTT600 | -5.427   |  0.699
                           |----------------------------
                           |RTT1800| -5.870   | -0.077
                           |----------------------------
                           |PLT(c) |  0.408   |  0.752
                    ------------------------------------
                           |S.Link |  4.308   |  4.741
                           |----------------------------
                           |RTT10  |  1.593   |  4.982
                           |----------------------------
                     SVD   |RTT35  | -1.981   |  5.999
                           |----------------------------
                           |RTT140 | -5.502   |  4.679
                           |----------------------------
                           |RTT300 | -7.192   |  1.302
                           |----------------------------
                           |PLT(c) |  0.993   |  2.382
                    ------------------------------------


    Table B.2:Smallest and largest values of over-admission percentages
               of all pairs of CLE threshold and EWMA weight

11.3.  Basic Evaluation

   Over-admission percentage statistics were evaluated using the single
   link topology.  Table B.3 gives results of an admission control



Satoh, et al.          Expires September 10, 2009              [Page 22]


Internet-Draft                 ST Marking                     March 2009


   simulation when the load was five times the admissible rate.  When
   the traffic type is CBR, the link speed is 128 Mbps and the load is
   the rate of 5000 flows.  The call interval per ingress-egress
   aggregate (IEA) is 0.012 sec.  When the traffic type is VBR, the link
   speed is 78.3 Mbps and the load is the rate of 9000 flows.  The call
   interval per IEA is 0.0067 sec.  When the traffic type is SVD, the
   link speed is 2.45 Gbps and the load is the rate of 1500 flows.  The
   call interval per IEA is 0.04 sec.  We show the average of the
   results of five simulations with different random seeds for each
   traffic type.  The performance of STM was very good although it was
   inferior to that of CL.


                         -------------------------
                         Type | Over Admission %
                              |-------------------
                              |    STM  |   CL
                         -------------------------
                          CBR |   0.065 |  0.028
                         -------------------------
                          VBR |   1.249 |  0.979
                         -------------------------
                          SVD |   4.678 |  4.476
                         -------------------------


   Table B.3: Over-admission percentage statistics obtained with single
                                   link

11.4.  Effect of Ingress-Egress Aggregation

   We evaluated the effect of ingress-egress aggregation (IEA) using RTT
   topology.  As with the simulations in [I-D.charny-pcn-single-
   marking], the aggregate load on the bottleneck was the same across
   each traffic type with the aggregate load being evenly divided among
   all ingresses.

11.4.1.  CBR

   Simulation results with traffic load conditions when the traffic type
   was CBR are shown in Table B.4.  The link speed of the bottleneck was
   128 Mbps for all the cases with varying numbers of ingresses.  It
   corresponded to 2000 CBR flows.  Thus, the PCN-admissible-rate
   corresponded to 1000 CBR flows.  The load corresponded to 5000 flows.
   The second column in Table B.4 and the following tables is the
   expected admitted number of connections per IEA.  STM(r) in Table B.4
   represents the results of STM with randomized CBR traffic.  Each
   packet of the randomized CBR traffic has an added delay distributed



Satoh, et al.          Expires September 10, 2009              [Page 23]


Internet-Draft                 ST Marking                     March 2009


   uniformly from 0 to 50 ms.  The results of both CBR traffic and
   randomized CBR traffic were almost the same, as shown in Table B.4.
   These simulation results show that the accuracy of STM was little
   affected by synchronization because synchronization effects are
   largest in the case of CBR with low IEA [I-D.charny-pcn-single-
   marking].


   ----------------------------------------------
    No. of  |Expected   |    Over Admission
   ingresses|No.  of    |          (%)
            |connections|------------------------
            |per IE pair|  STM  |STM (r)|  CL
   ----------------------------------------------
      10    |    100    | 0.750 | 0.786 | 0.761
   ----------------------------------------------
      70    |    14.3   | 0.890 | 0.867 | 0.727
   ----------------------------------------------
     600    |     1.6   | 0.122 |-0.085 | 0.038
   ----------------------------------------------
    1000    |      1    |-0.241 |-0.433 |-0.179
   ----------------------------------------------


   Table B.4: Over admission percentage statistics with CBR, RTT

11.4.2.  VBR

   Simulation results with traffic load conditions when the traffic type
   was VBR are shown in Table B.5.  The link speed of the bottleneck was
   78 Mbps for all the cases with varying numbers of ingresses.  This
   link speed was 1800 times higher than the average rate of a VBR flow.
   Thus, the PCN-admissible-rate was 900 times higher than the average
   rate of a VBR flow.  The load was 2.5 times higher than the link
   speed.  The result of STM is almost the same as that of CL.
















Satoh, et al.          Expires September 10, 2009              [Page 24]


Internet-Draft                 ST Marking                     March 2009


                  ----------------------------------------
                   No. of  |Expected   |Over Admission
                  ingresses|No.  of    |      (%)
                           |connections|------------------
                           |per IE pair|   STM  |  CL
                  ----------------------------------------
                     10    |    180    |  1.562 |  1.345
                  ----------------------------------------
                     70    |    25.7   |  1.334 |  1.462
                  ----------------------------------------
                    600    |     3     | -0.483 |  0.699
                  ----------------------------------------
                   1800    |     1     | -1.822 | -0.077
                  ----------------------------------------


     Table B.5: Over-admission percentage statistics with VBR and RTT

11.4.3.  SVD

   Simulation results with traffic load conditions when the traffic type
   was SVD are shown in Table B.6.  The link speed of the bottleneck was
   2448 Mbps for all the cases with varying numbers of ingresses.  This
   link speed was 600 times higher than the average rate of a SVD flow.
   Thus, the PCN-admissible-rate was 300 times higher than the average
   rate of a SVD flow.  The load was 2.5 times higher than the link
   speed.  Although the result of STM is almost the same as that of CL,
   CL is slightly superior to STM.

                  ----------------------------------------
                   No. of  |Expected   |Over Admission
                  ingresses|No.  of    |      (%)
                           |connections|------------------
                           |per IE pair|   STM  |  CL
                  ----------------------------------------
                     10    |     30    |  4.982 |  4.870
                  ----------------------------------------
                     35    |     8.4   |  5.261 |  4.898
                  ----------------------------------------
                    140    |      2    |  3.595 |  2.597
                  ----------------------------------------
                    300    |      1    | -0.033 |  0.221
                  ----------------------------------------


     Table B.6: Over-admission percentage statistics with SVD and RTT





Satoh, et al.          Expires September 10, 2009              [Page 25]


Internet-Draft                 ST Marking                     March 2009


11.5.  Effect of Multi-bottleneck

   We evaluated the effect of a multi-bottleneck with PLT topology.  The
   over- admission-percentage of each bottleneck ID is shown in Table
   B.7.  The bottleneck IDs are named in the order of lowest (upstream)
   to highest (downstream).  The alphabets in Table B.7 shows nodes in
   Fig. A.3(c).  When CBR was used as the traffic type, the link speed
   of all the bottlenecks was 128 Mbps and the call-interval per IEA was
   0.024 s.  When VBR was used as the traffic type, the link speed of
   all the bottlenecks was 78 Mbps and the call-interval per IEA was
   0.01 s.  When SVD was used as the traffic type, the link speed of all
   the bottlenecks was 2448 Mbps and the call-interval per IEA was 0.08
   s.  Both show good performance.  In the cases of CBR and VBR, CL is
   slightly superior to STM.  In the case of SVD, STM is slightly
   superior to CL.

       -------------------------------------------------------------
        Traffic   |Algorithm|         Bottleneck ID
        Type      |         | 1(A-B)| 2(B-C)| 3(C-D)| 4(D-E)| 5(E-F)
       -------------------------------------------------------------
         CBR      |   STM   | 0.060 | 0.043 | 0.038 | 0.062 | 0.056
                  |    CL   | 0.019 | 0.025 | 0.015 | 0.005 | 0.009
       -------------------------------------------------------------
         VBR      |   STM   | 0.690 | 0.684 | 0.708 | 0.749 | 0.752
                  |    CL   | 0.560 | 0.540 | 0.585 | 0.614 | 0.620
       -------------------------------------------------------------
         SVD      |   STM   | 1.441 | 1.910 | 1.348 | 1.602 | 1.551
                  |    CL   | 1.887 | 1.908 | 1.952 | 2.123 | 2.382
       -------------------------------------------------------------


        Table B.7: Over admission percentage with multi-bottleneck

11.6.  Fairness among Different Ingress-Egress Pairs

   In [I-D.charny-pcn-single-marking], fairness is illustrated using the
   ratio between the bandwidth of the long-haul aggregates and the
   short-haul aggregates.  We used CBR traffic with the load of each
   bottleneck being five times the PCN-admissible-rate.  The fairness
   results for different topologies are shown in Table B.9.  We measured
   the ratios of the average throughput of short-haul aggregates to that
   of long-haul aggregates during the simulation time at the first
   bottleneck.  The average of five simulations is displayed.  The link
   speed and call interval conditions in Table B.7 were used in this
   experiment.  Both show good performance.






Satoh, et al.          Expires September 10, 2009              [Page 26]


Internet-Draft                 ST Marking                     March 2009


     -----------------------------------------------------------------
          |Topo|                  Simulation Time (s)
          |    |  10  |  20  |  30  |  40  |  50  |  60  |  70  |  80
     -----------------------------------------------------------------
     STM  |PLT2| 0.98 | 1.00 | 1.05 | 1.11 | 1.18 | 1.25 | 1.32 | 1.39
      CL  |    | 1.02 | 1.04 | 1.08 | 1.15 | 1.22 | 1.30 | 1.38 | 1.46
     -----------------------------------------------------------------
     STM  |PLT3| 0.98 | 1.04 | 1.13 | 1.24 | 1.35 | 1.46 | 1.57 | 1.67
      CL  |    | 0.98 | 1.00 | 1.05 | 1.12 | 1.21 | 1.30 | 1.41 | 1.52
     -----------------------------------------------------------------
     STM  |PLT5| 1.02 | 1.06 | 1.17 | 1.31 | 1.46 | 1.62 | 1.79 | 1.97
      CL  |    | 1.02 | 1.05 | 1.14 | 1.28 | 1.43 | 1.59 | 1.75 | 1.93
     -----------------------------------------------------------------


                      Table B.8: Fairness performance


12.  Appendix C: Flow termination

   We evaluated over-termination percentages and the time to terminate
   the necessary amount of traffic (termination time) when the load of
   the bottleneck was 1.25 and 2.0 times the PCN-supportable-rate.
   Over-termination percentages are defined as (PCN-supportable-rate -
   the rate after termination)/PCN-supportable-rate) expressed as a
   percentage.  The PCN-admissible-rate and PCN-supportable-rate were
   20% and 40% of the link speed, respectively, in all the links.  The
   load was lower than the PCN-admissible-rate at the beginning of the
   simulation.  The simulation time was 100 s.  The duration of all the
   flows was infinity.  One IE-pair was generated at half the simulation
   time (50 s).  Each flow of the IE-pair arrived in accordance with
   uniform distribution within the average of the packet interval in a
   flow.  This simulated the change of a route when there was a failure.
   Over-termination percentages were calculated using the average rate
   during the time interval between 80 and 100 s.  In all the tables,
   termination time is shown as the time between the detection time of
   termination and the last flow termination.  However, latency one-way
   time from the PCN-egress-node to PCN-ingress-node is not included.
   Because of bursty traffic such as VBR and SVD, an accurate
   termination often makes intermittent terminations.  Therefore, a
   termination was regarded as the last termination when it took more
   than 1 s until the next termination.  In all the simulation, PCN
   traffic rate was less than PCN-supportable rate just after the last
   termination.  Over-termination percentages in the following tables
   were larger than those just after the last termination in the cases
   of VBR and SVD because intermittent terminations occurred.

   When the traffic type was CBR, the link speed was 320 Mbps and the



Satoh, et al.          Expires September 10, 2009              [Page 27]


Internet-Draft                 ST Marking                     March 2009


   load was the rate of 2500 and 4000 flows.  When the traffic type was
   VBR, the link speed was 109 Mbps and the load was the rate of 2500
   and 4000 flows.  When the traffic type was SVD, the link speed was 4
   Gbps and the load was the rate of 500 and 800 flows.  We used
   randomized CBR, VBR, and SVD in termination simulations.  Each packet
   of the randomized CBR, VBR, and SVD traffic has an added delay
   distributed uniformly from 0 to 50 ms.  We show the average of the
   results of five simulations with different random seeds for each
   traffic type.

12.1.  Basic evaluation

   The results of a termination control simulation with the single link
   topology are shown in Table C.1.  When the load was 1.25 times the
   PCN-supportable-rate (SR in Table), the accuracy of the proposed
   control was almost the same as that of the CL control in the cases of
   VBR and SVD, although the proposed control was inferior to the CL
   control in the case of CBR.  However, when the load was 2.0 times the
   PCN-supportable-rate, the proposed control was more accurate than the
   CL control.  We show the average of results of five simulations with
   different random seeds for each traffic type.

       -------------------------------------------------------------
       Traffic | Load    | Over termination(%)| Termination time (s)
         Type  |(x SR)   |------------------------------------------
               |         |   STM   |    CL    |  STM    |   CL
       -------------------------------------------------------------
          CBR  |         |   3.260 |   0.531  |  0.50   |   0.50
          VBR  |   1.25  |  13.201 |  13.785  |  0.44   |   0.52
          SVD  |         |  12.367 |  14.552  |  1.51   |   0.64
       -------------------------------------------------------------
          CBR  |         |   3.461 |  12.441  |  1.40   |   0.40
          VBR  |   2.0   |   4.910 |  25.833  |  1.57   |   0.40
          SVD  |         |  12.527 |  24.503  |  2.05   |   0.40
       -------------------------------------------------------------



     Table C.1: Over-termination percentage statistics and termination
                      time obtained with single link

12.2.  Effect of Ingress-Egress Aggregation

   We evaluated the effect of ingress-egress aggregation (IEA) with RTT
   topology.  As with the simulations in [I-D.charny-pcn-single-
   marking], the aggregate load on the bottleneck was the same across
   each traffic type, with the aggregate load being evenly divided among
   all ingresses.



Satoh, et al.          Expires September 10, 2009              [Page 28]


Internet-Draft                 ST Marking                     March 2009


12.2.1.  CBR

   Simulation results obtained with traffic load conditions when the
   traffic type was CBR are shown in Table C.2.  SR in the second column
   represents the PCN-supportable-rate.  The link speed of the
   bottleneck was 40.5 Mbps and the propagation delay between the
   ingress and egress nodes was 1 ms for all the cases of varying
   numbers of ingresses.  When the load is 1.25 times of PCN-
   supportable-rate, over termination percentages of STM are larger than
   those of CL and termination times of STM are slightly larger than
   those of CL.  When the load is 2.0 times of PCN-supportable-rate,
   over termination percentages of STM are smaller than those of CL and
   termination times of STM are over one second larger than those of CL.

    -------------------------------------------------------------------
     No. of   |Load    |   Over termination (%) |  Termination time (s)
     Ingress  |(x SR)  |-----------------------------------------------
              |        |    STM    |     CL     |   STM    |    CL
    -------------------------------------------------------------------
        2     |        |   6.310   |   2.810    |   0.73   |  0.54
       10     |  1.25  |   6.070   |   3.270    |   0.71   |  0.60
       35     |        |   7.451   |   4.170    |   0.85   |  0.61
       70     |        |   6.341   |   4.780    |   0.92   |  0.66
    -------------------------------------------------------------------
        2     |        |   7.211   |  12.960    |   1.61   |  0.55
       10     |  2.0   |   6.560   |  12.560    |   1.69   |  0.63
       35     |        |   6.171   |  13.370    |   1.82   |  0.64
       70     |        |   5.851   |  14.300    |   1.94   |  0.65
    -------------------------------------------------------------------


     Table C.2: Over-termination percentage statistics and termination
                      time obtained with CBR and RTT

12.2.2.  VBR

   Simulation results obtained with traffic load conditions when the
   traffic type was VBR are shown in Table C.3.  The link speed of the
   bottleneck was 41.1 Mbps and the propagation delay between the
   ingress and egress nodes was 1 ms for all the cases of varying
   numbers of ingresses.  When the load is 1.25 times of PCN-
   supportable-rate, over termination percentages of STM are smaller
   than those of CL except the case of the number of ingress 10and
   termination times of STM are slightly larger than those of CL.  When
   the load is 2.0 times of PCN-supportable-rate, over termination
   percentages of STM are much smaller than those of CL and termination
   times of STM are over one second larger than those of CL.




Satoh, et al.          Expires September 10, 2009              [Page 29]


Internet-Draft                 ST Marking                     March 2009


    -------------------------------------------------------------------
     No. of   |Load    |   Over termination (%) |  Termination time (s)
     Ingress  |(x SR)  |-----------------------------------------------
              |        |    STM    |     CL     |   STM    |    CL
    -------------------------------------------------------------------
        2     |        |   9.696   |   11.320   |   0.65   |   0.61
       10     |  1.25  |  11.226   |   10.564   |   0.76   |   0.73
       35     |        |   9.971   |   12.857   |   0.80   |   0.70
      100     |        |   8.112   |   15.422   |   0.82   |   0.73
    -------------------------------------------------------------------
        2     |        |   5.377   |   20.768   |   1.54   |   0.49
       10     |  2.0   |   5.536   |   22.791   |   1.61   |   0.57
       35     |        |   5.271   |   22.670   |   2.38   |   0.62
      100     |        |   4.534   |   25.033   |   2.46   |   0.67
    -------------------------------------------------------------------


     Table C.3: Over-termination percentage statistics and termination
                      time obtained with VBR and RTT

12.2.3.  SVD

   Simulation results obtained with traffic load conditions when the
   traffic type was SVD are shown in Table C.4.  The link speed of the
   bottleneck was 613 Mbps and the propagation delay between the ingress
   and egress nodes was 1 ms for all the cases of varying numbers of
   ingresses.  When the load is 1.25 times of PCN-supportable-rate, over
   termination percentages of STM are smaller than those of CL and
   termination times of STM are over one second larger than those of CL.
   When the load is 2.0 times of PCN-supportable-rate, over termination
   percentages of STM are much smaller than those of CL and termination
   times of STM are over 1.5 second larger than those of CL.

    -------------------------------------------------------------------
     No. of   |Load    |   Over termination (%) |  Termination time (s)
     Ingress  |(x SR)  |-----------------------------------------------
              |        |    STM    |     CL     |   STM    |    CL
    -------------------------------------------------------------------
        2     |        |  12.078   |   15.748   |   1.94   |   0.71
       10     |  1.25  |  12.906   |   16.016   |   1.46   |   0.74
       35     |        |  12.616   |   20.396   |   1.42   |   0.73
    -------------------------------------------------------------------
        2     |        |  12.214   |   24.970   |   2.61   |   0.54
       10     |  2.0   |  11.592   |   26.763   |   2.05   |   0.60
       35     |        |  13.390   |   31.583   |   1.97   |   0.66
    -------------------------------------------------------------------





Satoh, et al.          Expires September 10, 2009              [Page 30]


Internet-Draft                 ST Marking                     March 2009


     Table C.4: Over-termination percentage statistics and termination
                      time obtained with SVD and RTT

12.3.  Effect of Multi-bottleneck

   We evaluated the effect of multi-bottlenecks using PLT(c) topology.
   The over-termination-percentage of each bottleneck ID and termination
   time of the long-haul IEA are shown in Table C.5.  These bottleneck
   IDs are the same as those in Table B.7.  The link speed of each
   traffic type was the same as that in simulations of the effect of
   IEA.  The propagation delay between the ingress and egress nodes was
   1 ms.  Over termination percentages of STM are almost the same among
   all the bottleneck links although those of CL worsen as the
   bottleneck IDs decrease (upstream).  CL is affected by accumulation
   of marking.  STM is not affected by it because all the packets are
   marked in termination.

   The results of termination time show in Table C.6.  When the load is
   1.25 times of PCN-supportable-rate, termination times of STM are
   almost the same as those of CL.  When the load is 2.0 times of PCN-
   supportable-rate, termination times of STM are over one second larger
   than those of CL.





























Satoh, et al.          Expires September 10, 2009              [Page 31]


Internet-Draft                 ST Marking                     March 2009


     ------------------------------------------------------------------
             |       |    |              Over termination (%)
     Load    |Traffic|Alg.|--------------------------------------------
     (x SR)  |Type   |    |          Bottleneck ID
             |       |    | 1(A-B) | 2(B-C) | 3(C-D) | 4(D-E) | 5(E-F)
     ------------------------------------------------------------------
             |CBR    |STM |  4.491 |  4.551 |  4.321 |  4.311 |  4.370
             |       |CL  |  6.351 |  2.742 |  1.341 |  0.882 |  0.672
             |---------------------------------------------------------
       1.25  |VBR    |STM | 15.443 | 15.893 | 15.729 | 15.134 | 15.741
             |       |CL  | 17.321 | 13.343 | 12.459 | 11.201 | 11.127
             |---------------------------------------------------------
             |SVD    |STM | 15.941 | 15.489 | 15.219 | 15.877 | 15.229
             |       |CL  | 18.467 | 15.535 | 14.487 | 13.950 | 14.536
     ------------------------------------------------------------------
             |CBR    |STM |  4.371 |  4.552 |  4.301 |  4.241 |  4.741
             |       |CL  | 20.291 | 12.601 |  9.571 |  8.050 |  7.550
             |---------------------------------------------------------
       2.0   |VBR    |STM |  8.941 |  6.447 |  6.619 |  6.486 |  6.332
             |       |CL  | 27.015 | 14.756 | 14.408 | 14.510 | 14.242
             |---------------------------------------------------------
             |SVD    |STM | 18.019 | 15.547 | 16.355 | 16.334 | 15.983
             |       |CL  | 27.115 | 15.151 | 14.097 | 14.263 | 13.472
     ------------------------------------------------------------------


     Table C.5: Over-termination percentage statistics and termination
                    time obtained with multi-bottleneck























Satoh, et al.          Expires September 10, 2009              [Page 32]


Internet-Draft                 ST Marking                     March 2009


       -------------------------------------------------------------
       Load (x SR)    |Traffic Type|Algorithm | Termination time (s)
       -------------------------------------------------------------
                      |CBR         |   STM    |       0.53
                      |            |   CL     |       0.50
                      |---------------------------------------------
         1.25         |VBR         |   STM    |       0.45
                      |            |   CL     |       0.53
                      |---------------------------------------------
                      |SVD         |   STM    |       0.45
                      |            |   CL     |       0.54
       -------------------------------------------------------------
                      |CBR         |   STM    |       1.45
                      |            |   CL     |       0.42
                      |---------------------------------------------
         2.0          |VBR         |   STM    |       1.92
                      |            |   CL     |       0.48
                      |---------------------------------------------
                      |SVD         |   STM    |       1.75
                      |            |   CL     |       0.64
       -------------------------------------------------------------


        Table C.6: Termination time obtained with multi-bottleneck


13.  Appendix D: Why is TBthreshold.shallow.threshold set to be smaller
     than the token bucket size by the bit-size of a metered packet?

   The origin of the threshold marking is a virtual queue [I-D.briscoe-
   tsvwg-cl-phb].  Therefore queueing theory is applied to marking.
   Little's formulas are one of the most powerful relationship for G/G/1
   in queueing theory where G/G/1 is described using Kendall's notation.
   Both arrival process and service time distribution of G/G/1 is
   described as a general distribution and 1 represents the number of
   servers.  Furthermore, Little's formulas are valid for any queue
   discipline such as (first-in, first-out) FIFO, (Last-in, first-out)
   LIFO, and (service in random order) SIRO.
   TBthreshold.shallow.threshold is introduced to apply the Little's
   formulas to marking.

   At first, we explain Little's formulas in queueing theory [see Gross
   and Harris].  Little's formulas are

                        L = lambda W             (D.1)


   and



Satoh, et al.          Expires September 10, 2009              [Page 33]


Internet-Draft                 ST Marking                     March 2009


                       L_q = lambda W_q,          (D.2)


   where L represents the mean number of customers in the system and L_q
   represents the expected number of customers in queue, lambda
   represents the average rate of customers entering the queueing
   system, W_q represents the expected time a customer spends waiting in
   the queue prior to entering service, and W represents the expected
   total time a customer spends in the queueing system.  In queueing
   theory, the term customer is used in a general sense and does not
   imply necessarily a human customer.

   From eqs.  (D.1) and (D.2), the following equation is derived,

          L - L_q = lambda(W-W_q) = lambda/mu = rho           (D.3)


   where mu represents the mean service rate, that is, 1/mu is the mean
   service time and rho is traffic intensity.  The left-hand side of
   eq.(D.3) is the expected number of customers in service in the steady
   state.  Furthermore,

             L = 1*p_1 + 2*p_2 + 3*p_3 + ......            (D.4)


   and

            L_q = 0*p_1 + 1*p_2 + 2*p_3 + ......            (D.5)


   where p_n represents the probability that the number of customers in
   the system is n in the steady state.  From eqs.  (D.4) and (D.5),

    L - L_q = p_1 + p_2 + p_3 + ...... = 1 - p_0 = p_b            (D.5)


   where p_b is the probability that the server is busy in the steady
   state.

   If we get p_b, we get the traffic intensity.  Marking by using
   TBthreshold.shallow.threshold is a way to get p_b.  The marking makes
   each packet marked when the previous packet is in service.  Each
   packet is not marked only when no packet is in a virtual queue at the
   packet's arrived time.  The marking ratio gives an approximation of
   p_b.  Both are different generally because the time average and the
   customer average are different, and p_b is the time average when the
   server is busy and the marking ratio is the customer average.
   However, both are the same when the arrival proces is a Poisson



Satoh, et al.          Expires September 10, 2009              [Page 34]


Internet-Draft                 ST Marking                     March 2009


   process.  This property is called PASTA, which is abbreviation that
   Poisson arrivals see time averages.  Furthermore, if an arrival
   process is described as a renewal process, an infinite superposition
   of them is the Poisson process {Cox].  The renewal process is a large
   class of stochastic processes.  The inter-arrival time is independent
   and identically distributed (IID).  Therefore the marking ratio by
   using TBthreshold.shallow.threshold is expected to give a good
   approximation of the traffic intensity.


14.  IANA Considerations

   TBD


15.  Security Considerations

   TBD


16.  Informative References

   [Cox]      Cox, D.R., "Renewal Theory", John Wiley & Sons, 1962.

   [Gross and Harris]
              Gross, D. and C. Harris, "Fundamentals of Queueing
              Theory", John Wiley & Sons, 1998.

   [I-D.babiarz-pcn-3sm]
              Babiarz, J., Liu, X-G., Chan, K., and M. Menth, "Three
              State PCN Marking", November 2007.

   [I-D.briscoe-tsvwg-cl-phb]
              Briscoe, B., Eardley, P., Songhurst, D., Le Faucheur, F.,
              Charny, A., Liatsos, V., Babiarz, J., Chan, K., Dudley,
              S., Karagiannis, G., Bader, A., and L. Westberg, "Pre-
              Congestion Notification Marking", October 2006.

   [I-D.charny-pcn-single-marking]
              Charny, A., Xhang, J., Le Faucheur, F., and V. Liatsos,
              "Pre-Congestion Notification Using Single Marking for
              Admission and Termination", November 2007.

   [I-D.ietf-pcn-architecture]
              Eardley, P., "Pre-Congestion Notification Architecture",
              January 2009.

   [I-D.ietf-pcn-baseline-encoding]



Satoh, et al.          Expires September 10, 2009              [Page 35]


Internet-Draft                 ST Marking                     March 2009


              Moncaster, T., Briscoe, B., and M. Menth, "Baseline
              Encoding and Transport of Pre-Congestion Information",
              Feburary 2009.

   [I-D.ietf-pcn-marking-behaviour]
              Eardley, P., "Marking behaviour of PCN-nodes",
              October 2008.

   [I-D.westberg-pcn-load-control]
              Westberg, L., "LC-PCN: The Load Control PCN Solution",
              November 2008.

   [I-D.zhang-pcn-performance-evaluation]
              Zhang, J., Charny, A., Liatsos, V., and F. Le Faucheur,
              "Performance Evaluation of CL-PHB Admission and
              Termination Algorithms", July 2007.


Authors' Addresses

   Daisuke Satoh
   NTT Advanced Technology Corporation
   1-19-18, Nakacho
   Musashino-shi, Tokyo  180-0006
   Japan

   Email: daisuke.satoh@ntt-at.co.jp


   Yukari Maeda
   NTT Advanced Technology Corporation

   Email: yukari.maeda@ntt-at.co.jp


   Oratai Phanachet
   NTT Advanced Technology Corporation

   Email: oratai.phanachet@ntt-at.co.jp


   Harutaka Ueno
   NTT Advanced Technology Corporation

   Email: harutaka.ueno@ntt-at.co.jp






Satoh, et al.          Expires September 10, 2009              [Page 36]