Internet Engineering Task Force Ming Hou
Differentiated Services Working Group Hussein T. Mouftah
Internet Draft Queen's University,Canada
Expires September 1999 June, 1999
Performance Evaluation of Premium Service
<draft-hou-diffserv-premium-eval-00.txt>
Status of this Memo
This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026.
This document is a product of the IETF Differentiated Services Working
Group. Comments are solicited and should be directed to the working group
mailing list.
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other groups may
also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any time.
It is inappropriate to use Internet Drafts as reference material or to cite
them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Distribution of this memo is unlimited.
Abstract
The performance evaluation of QoS required services helps in observing
the behavior of the Diffserv, and is an important part of the work of the
Diffserv Working Group. In this document we evaluate the Premium service in
terms of packet loss and jitter under Diffserv.
A postscript version of this document is also available.
1. Introduction
The Premium service, which was proposed in [7], requires
a low jitter and low packet loss during data transmission. It can
be implemented by the EF PHB.
By Diffserv protocol, the arriving Premium packets are geared to the EF
aggregates on the node output interface. The EF queue has a higher priority
in being forwarded than other queues, and packets in it are forwarded to
exhaust before the packets in other queues. This provides a very short
EF queue. Thus the Premium packets will experience a little queuing delay.
Packets from other services different from the Premium service, will be
dropped with higher priority when there is not enough buffer at the node
output interface. Since the Premium packets are forwarded prior to others,
the Premium service is expected to have a relatively lower packet loss than
other services.
This report is organized as follows: Section 2 describes the IP
simulator which is used in evaluating the Premium service; Section 3
presents the simulation results for Premium service in terms of jitter and
packet loss; Section 4 gives the conclusive remarks.
2. Description of simulator
In order to observe the various aspect of Diffserv, we have developed
the Queen's University IP Simulator (QUIPS). This real-time discrete-event
simulator is written in JAVA language and runs on UNIX platform.
QUIPS has been designed in a modular way, which contains three modules:
the Graphical User Interface (GUI), the network module and the control
module. This modular design fashion makes it easy for further extension.
In the simulation model, the network has been decomposed into four
components: the link, the node, the sender and the receiver. A sender and a
receiver represent the data producer and the data consumer, respectively.
The sender sends packets with different service requirement to the receiver,
and the receiver computes the end-to-end delay and other QoS parameters
after receiving the packet. A link is modeled as a uni-directional link,
since the QoS requirements on the two directions are not always the same.
The node component represents a router in a computer network. Both the
routing and the forwarding engine are implemented in the node.
The control module has an event queue, which handles the operation of
different network elements. The monitor in the control module gathers
statistics during simulation runs.
Although only Diffserv is implemented in QUIPS, new functions and IP
protocols can be easily added to the simulator without a significant change
to the codes because of its modular design.
3. Simulation results and analysis
Using QUIPS, both the Assured service and the Premium service can be
simulated and evaluated under Diffserv. This study focuses only on the
performance of the Premium service while the investigation into the Assured
service is left to future work.
The EF PHB has the highest priority in being forwarded, thus it can provide
a low delay, low loss service. The EF PHB is assigned to the Premium traffic
in order to implement the Premium service.
There is a number of mechanisms that can be used in implementing the
EF PHB. The simplest one is a priority queue where the arrival rate of
the queue is strictly less than its departure rate. As jitter comes from the
queueing delay along the path, a feature of this implementation is that
EF-marked microflows will see very little jitter at their subscribed rate if
all DS nodes along the path use this implementation since packets spend
little time in queues.
Since the Premium Service is a low jitter, low loss service, its performance
evaluation is focused on its behavior in terms of jitter and packet loss
under Diffserv.
Different network parameters may have different effect on the performance of
the Premium traffic. First is the output buffer size. When a Premium packet
arrives, if there is no enough output buffer for it, it will be either
dropped, which affects the packet loss; or be delayed, that is, be waited
withing the router until resources available, which increases the amount of
jitter. Secondly, the link bandwidth occupitation of the Premium traffic
is an important parameter in performance evaluation. As the density of the
Premium traffic increases, new Premium packets will arrive at the router
before the old ones can be sent. In this case, although the Premium packets
have higher priority in getting network resources than the other type of
traffic, newly arrived Premium packets will be lost in the competition of
resources with the Premium packets which have arrived ealier but have not
been sent yet. Thus, the jitter and the packet loss will increase. The
packet length also has some effects on the performace measurements. Assuming
a lengthy best-effort packet is transmitting while a Premium packet is
arriving. The Premium packet shall be delayed although it has a higher
priority. This causes the increase of jitter, and, if a number of Premium
packets are jammed, the packet loss of Premium packets will increase. All
the above parameters play an important role in the behavior of Premium
traffic under Diffserv, thus the evaluations in this draft are performed by
choosing different values of the output buffer size, the Premium traffic
allocation and the packet length.
A two-hop DS domain with the link bandwidth of 100 Mbps is modeled, as shown
in Figure 1. It is assumed that the SLAs and the TCAs have been established,
and remain unchanged during simulation. For the EF-marked packets, or the
Premium Service required packets, sources have been set up to produce
packets at a constant bit rate with a variation of +/- 10% of the subscribed
peak rate. A mixture of other kinds of traffic, the best effort service, is
used to fill the link.
4. Jitter Evaluation
In evaluating the packets' jitter, the jitter is recorded as the added
delay normalized by the time to send a packet at the subscribed peak
rate. The simulation parameters are changed and the Premium
packets' behavior is observed under Diffserv in three aspects: for different
output buffer size of routers, for different packet size, and for
different Premium service allocation. During each kind of simulation,
only one parameter of the three is changed and the other two remain
fixed.
4.1 Evaluation with Different Buffer Size
This jitter evaluation of Premium packets is simulated under four
different output buffer size: 2, 5, 10 packets and an infinite buffer. The
packet length is assumed to be 1500 bytes for all simulation runs, while the
Premium traffic is subscribed as 30% of the link bandwidth.
The average delay of Premium service packets and the jitter Premium
packets experienced are shown in Figures 2 and 3, respectively.
>From Figure 2, one can see that after a short transient period at the
beginning of a simulation run, the average delay tends to be constant. The
transient area should be ignored because of the system unsteadyness at the
begining of a simulation run. The average delay of Premium packets in all
the cases is very small, with a slight difference among them. The average
delay decreases as the number of output buffer increases. However,
the results of the infinite buffer size are very close to the results
of the 10-packet size buffer.
The jitter for all test cases are very close, as shown in Figure 3. A
detailed comparison of the jitter at different Premium packet percentile is
listed in Table 1. Both the figure and the table imply that the variaration
of the output buffer size does not have much
effect on the jitter behavior of Premium packets.
====================================================================
|Buffer|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter|
--------------------------------------------------------------------
| 2 | 50 | 0.015 | 75 | 0.043| 95 |0.0505|
--------------------------------------------------------------------
| 5 | 50 | 0.0205 | 75 | 0.036| 95 |0.0505|
--------------------------------------------------------------------
| 10 | 50 | 0.0205 | 75 | 0.036| 95 |0.0505|
--------------------------------------------------------------------
|infinity| 50 | 0.0205 | 75 | 0.036| 95 |0.0505|
====================================================================
Table 1. Comparison of Jitter with Different Buffer Size
4.2 Evaluation with Different Packet Size
Here, the node output buffer is chosen to be infinite; thus there is
no packet loss during transmission. The individual source rates of
generating the Premium packets are picked as the 30% of the link
bandwidth or 30 Mbps.
Figure 4 shows the delay each 1500-byte EF packet experienced. One may
observe that a few packets have a longer delay than the others because of
the congestion at the node output interface.
However, from the graph of packet percentile vs jitter (Figure 5), one can
see that most of the packets experience low jitter. The absolute value of
the jitter for different packet size (1000 bytes, 1500 bytes and 2500 bytes)
are compared. From Figure 5, one can see that the jitter value is
bigger as the packet size increases. However, the normalized jitters
are almost the same, and are all less than 0.7 packet-time (Table 2).
A comparison of the average packet delay experienced with different
packet sizes is shown in Figure 6. Ignore the transient period,
the average delay tends to be constant. One can see that the average
delay of the Premium packets is increasing as the packet size
increases. Long packet will cause larger delay, because at the same
speed it will take longer time to transmit a long packet than to
transmit a short packet. New packet will arrive before previous one
completely transmitted, thus the new one have to be waited in the
queue and increases the end-to-end delay in reault.
=========================================================================
|Packet Size|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter|
-------------------------------------------------------------------------
| 1000 | 50 | 0.18 | 75 | 0.36 | 95 | 0.62 |
-------------------------------------------------------------------------
| 1500 | 50 | 0.16 | 75 | 0.35 | 95 | 0.7 |
-------------------------------------------------------------------------
| 2500 | 50 | 0.16 | 75 | 0.31 | 95 | 0.61 |
=========================================================================
Table 2. Comparison of Normalized Jitter with Different Packet Size
4.3 Evaluation with Different Premium service allocation
In this section, the effect of varying the Premium traffic allocation
on the jitter behavior is investigated. The Premium traffic to be
allocated is chosen as a small percentage of the link bandwidth. The
allocation is chosen to be 5%, 10%, 20% and 30% of the link bandwidth.
Figure 7 shows the simulation results of the average delay, which support
the notion that the density of the Premium traffic will cause the average
delay to be increased. With very small percentage of Premium traffic, such
as 5\% or 10\% of the link bandwidth, the average delay is small, but with a
small change on the allocation, the average delay will change significantly.
The small allocation of Premium traffic also has an outstanding jitter
performance. As shown in Figure 8, the 5% allocation has better jitter
performance than the others. A comparison of the jitter value for each test
case is presented in Table 3.
As the traffic allocation for the Premium packet increases, more
Premium packets will apprear at the output interface of routers, and
probability of more than one Premium packet competing for the same
network resources will increase. The subsequent Premium packets are put
into the waiting queue, when there is a Premium packet being transmitted.
========================================================================
|Allocation|Percentile|Jitter(mSec)|Percentile|Jitter|Percentile|Jitter|
------------------------------------------------------------------------
| 5% | 50 | 0.008 | 75 | 0.01 | 95 | 0.044|
------------------------------------------------------------------------
| 10% | 50 | 0.008 | 75 | 0.01 | 95 | 0.044|
------------------------------------------------------------------------
| 20% | 50 | 0.013 | 75 |0.035 | 95 | 0.052|
------------------------------------------------------------------------
| 30% | 50 | 0.021 | 75 |0.037 | 95 | 0.051|
========================================================================
Table 3. Normalized Jitter with Different Premium Traffic Allocation
5. Packet loss evaluation
In this section, the behavior of the Premium packets under Diffserv in terms
of packet loss is investigated. Similar to the case of the jitter
evaluation, the simulation parameters are varied and the Premium packets'
behavior is observed under Diffserv in three aspects: for different
output buffer size of routers, for different packet size, and for
different Premium service allocation. During each type of simulation,
only one parameter of the three is changed and the other two remain
fixed.
The simulation results show the loss probability of packets for the network
topology shown in Figure 1. The output buffer of the nodes is
assumed to be of two-packet size, and the network load ranges from zero to
three times of the link bandwidth among which is 30% of the Premium traffic.
5.1 Evaluation with Different Packet Size
The packet loss for different packet size with an output buffer
of two packets is shown in Figure 9. One can see that varying of the packet
length does not have much effect on the packet drop.
5.2 Evaluation with Different Buffer Size
In Figure 10, a comparison of the Premium packet loss probability and the
best effort packet loss probability is provided. Since the Premium packets
have the highest priority in forwarding among the traffic streams, they will
experience less amount of loss than the best effort packets. One can see
that the best effort packets have a very high probability of being lost
, while the Premium packets have relatively lower probability of packet
loss. However, as the network load increases, the Premium packets will be
dropped much more because of the congestion at the nodes output interface.
The packet loss behavior of the Premium traffic as the output buffer
is increased is tested as shown in Figure 11. Also, the overall packet loss
with different buffer size has been investigated, as shown in Figure 12. The
buffer sizes are picked as 2, 3, 5 and 10 packets size, respectively. One
can see that the packet drop is decreasing as the buffer size increases.
5.3 Evaluation with Different Premium traffic allocation
Three Premium traffic allocations, 5%, 10% and 30%, have been adopted for
the investigation results reported in this section.
The results of the packet loss probability for the Premium packets under
different traffic allocation are shown in Figure 13. As expected, the loss
probability increases as the traffic allocation increases. Even a small
change in the allocation, will have a big effect on the packet loss
probability. This is because when the Premium traffic increases, at
the output interface of the routers, more Premium packets will compete
for the network resource, and the routers have to drop more packets in order
to satisfy others. For the same reason, the packet loss probability for
the Premium packets together with the best-effort packets will increase as
the Premium traffic allocation increases, as shown in Figure 14.
Finally, a comparison of the probability of packet loss for the Premium
packet with the best-effort packets, for different Premium traffic
allocation is provided in Figure 15.
6. Conclusion
This draft has briefly described an IP simulator developed by Queen's
University computer networks group. The performance of the Premium service
has been evaluated by QUIPS under Diffserv. The QoS performance measures are
chosen to be the jitter and the packet loss; and the different network
parameters, such as buffer size, packet size and percentile allocation of
Premium service, are used for different simulation runs. The simulation
results show that the Premium service can be provided using Diffserv.
Diffserv.
7. References
[1] C. Alaettiniglu, A. U. Shanker, etc. Design and Implementation of MaRS:
A Routing Testbed, September 1992, http://www.ccs.neu.edu/home/matta
[2] C. Alaettiniglu, A. U. Shanker, etc. MaRs - version 1.0 Programmer's
Manual, June 1991, http://www.ccs.neu.edu/home/matta
[3] D. Black, S. Blake, M. Carlson, E. Davies, Z. Wang, W. Weiss, An
Architecture for Differentiated Services, Internet rfc2475, December 1998
[4] Y. Bernet, D. Durham, F. Reichmeyer, Requirements of Diff-serv Boundary
Routers, Internet draft, draft-bernet-diffedge-01.txt, November 1998
[5] Y. Bernet, et al., A Framework for Differentiated Services, Internet
draft, draft-ietf-diffserv-framework-01.txt, October 1998
[6] V. Jacobson, K. Nichols, K. Poduri, An Expedited Forwarding PHB,
Internet draft, draft-ietf-diffserv-phb-ef-01.txt, November 1998
[7] K. Nichols, V. Jacobson, and L. Zhang, A Two-bit Differentiated Services
Architecture for the Internet, Internet Draft,
draft-nichols-diff-svc-arch-00.txt, November 1997.
8. Authors' Addresses
Ming Hou
Department of Electrical and Computer Engineering
Queen's University
Kingston, Ontario
Canada K7L 3N6
houm@eleceng.ee.queensu.ca
Hussein T. Mouftah
Department of Electrical and Computer Engineering
Queen's University
Kingston, Ontario
Canada K7L 3N6
mouftah@eleceng.ee.queensu.ca