Network Working Group Sudhin Jacob
Internet Draft Juniper Networks
Intended Status: Informational Praveen Ananthasankaran
Expires: August 03,2017 Nokia
Februrary 06, 2017
Benchmarking of Y1731 Performance Monitoring
draft-jacpra-bmwg-pmtest-03
Abstract
The draft defines the methodologies for benchmarking of the Y1731
performance monitoring on DUT in various methods like Calculation
of near-end and far-end data. Measurement is done in scenarios by
using pre-defined COS and without COS in the network.The test
includes Impairment test,High Availability test and soak tests.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 03, 2017.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Praveen & Sudhin Expires August 03,2017 [Page 1]
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
1.2. Terminologies. . . . . . . . . . . . . . . . . . . . . . 3
2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Test Procedure . . . . . . . . . . . . . . . . . . . . . . . 5
5.Test cases
5.1 Y.1731 Two-way Delay Measurement Test procedure. . . . . . . . 5
5.2 Y.1731 One-way Delay Measurement Test procedure. . . . . . . . 7
5.3 Loss measurement without COS Test Procedure. . . . . . . . . . 9
5.4 Loss measurement with COS Test Procedure. . . . . . . . . . . .12
5.5. Synthetic Loss Measurement Test Procedure. . . . . . . . . . .15
6.Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . 18
7. Security Considerations. . . . . . . . . . . . . . . . . . . . . 18
8.IANA Considerations. . . . . . . . . . . . . . . . . . . . . . . .18
Praveen & Sudhin Expires August 03,2017 [Page 2]
1. Introduction
Performance monitoring is explained in ITU Y1731.This document defines
the methodologies for benchmarking performance of Y1731 over a point to
point service. Performance Monitoring has been implemented with
many varying designs in order to achieve their intended network functionality.
The scope of this document is to define methodologies for benchmarking Y1731
performance measurement. The following protocols under Y.1731 will be benchmarked.
1. Two-way delay measurement
2. One-way delay measurement
3. Loss measurement
4. Synthetic loss measurement
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
1.2. Terminologies
PM Performance monitoring
COS Class of Service
In-profile CIR termed as green packets.
Out-profile EIR Yellow/Amber packet.
LMM Loss Measurement Message
LMR Loss Measurement Reply
DMM Delay Measurement Message
DMR Delay MEasurement Reply
P Router Provider Router.
PE Router Provider Edge Router
CE Router customer Edge Router
DUT Device under Test.
CCM Continuity check messages
Praveen & Sudhin Expires August 03,2017 [Page 3]
2.1 Test Topology
| Traffic Generator
+----------+
| |
| PE2 |
| |
+----------+
|
|
+----------+
| |
| Core |
| router |
+----------+
|
|
+----------+
| |
| DUT |
| PE1 |
+----------+
|
|--- Traffic Generator
Praveen & Sudhin Expires August 03,2017 [Page 4]
3. Network
The benchmarking topology consists of 3 routers and 2 traffic generators.
DUT is PE1 connected to CE. The core router is the P router mentioned in the
topology. There is layer two(point-to-point) services running from PE1 to PE2.
On the top of that performance monitoring such as loss,delay and synthetic
measurements are running.PE1 is acting as DUT.The traffic will be layer 2
with vlan tag.The frame size will be 64,128,512,1024 and 1400.The tests are
carried out using these various frame size.The traffic will be uni directional
or bi directional.
4. Test Procedure
The tests are defined to benchmark the Y1731 performance monitoring in
High Availability,Impairment,SOAK,Scale,with traffic of various line rate
and frame sizes.
4.1 Performance Monitoring with traffic
Traffic is send with different .1p priorities,line rate and frame size of 64,
128,512,1024,1400.The PM values are measured with each frame size with various
line rates.
4.2 High Availability
The traffic is flowing bi-direction.Then traffic is flowing at "P" packets per
sec. The traffic generator is measuring the Tx and Rx packets, while the routing
engine failover there should not any packet loss the router tester must show both
"P" packet per seconds.The PM historical data should not reset.
4.3 Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions with
Performance monitoring running over it.There should not be any crashes,memory
leaks.
4.4 SOAK
This test is used to measure the performance of DUT over a period of time,with scaled
configuration and traffic over a period of time "T'". In each interval "t1" the
parameters measured are CPU usage, memory usage and crashes.
4.5 Measurement Statistics
The test is repeated for "N" times and the value is taken by averaging the values.
Praveen & Sudhin Expires August 03,2017 [Page 5]
5 Test Cases
5.1 Y.1731 Two-way Delay Measurement Test procedure
Basic Testing Objective
Check the round trip delay of the network in different conditions of
traffic load in the network.
Test Procedure
Configure a layer 2 point-to-point service between PE1 and PE2.
Configure Y.1731 Two way delay measurement over the service.Observe
the delay measurement in the following conditions of traffic in the network
a. Send 80% of Line-rate traffic with different priorities and frame size.
b. Send 40% of Line-rate traffic with different priorities and frame size.
c. Without any line traffic
The result of all the 3 conditions above are noted and correlated.
Test Measurement
The following factors needs are to be measured to benchmark the result
1. The average two-way delay
2. The average two-way delay variation
In the above 3 conditions the results obtained must be similar
1. Ideal case
In this case the hardware aspects of processing capacity and the link
level anomalies are not considered. The benchmark is just on the protocol
functioning.In such environment where for an ideal case the system should
expect delay variation to be zero.
Praveen & Sudhin Expires August 03,2017 [Page 6]
2. Practical case
This case is used to benchmark results when delay measurement is done on
physical hardware (like a router).The factors of packet process jitter and
link level delays needs to be considered here.The delay variation in such
cases will defer based on the above parameters on different hardware systems.
Result will very base on the exact hardware.
Delay Variation
+
^ |
| |
| |
+ |
|
| +----------+
| | |
| | |
+------------+----------+-------------+
Time
----->
Traffic (0 to 100 percent line rate)
Impairment
This is to benchmark two-way delay measurement even when both data and PDUs
are dropped in the network using the impairment tool.
Measurement
The results must show similar results before and after this test.
High Availability
During routing engine failover the historical data must not reset.
Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions
with Performance monitoring running over it.There should not be any
crashes,memory leaks.
Soak
The bi directional traffic is send over service over 24 to 48
hours and measure after the stipulated time there must not be
any change in behavior in the network for performance monitoring
Measurement
There should not be any core or crashes,memory leaks.
Praveen & Sudhin Expires August 03,2017 [Page 7]
5.2 One-Way delay measurement Test Procedure
Basic Testing Objective
The test defined to measure the one-way delay measurement.
One-way delay measurement as defined in Y.1731 is the delay
of the packet to originate from a specific end-point till it
reached the other end of the network. The measurement of
this mandates the clock to be accurately synchronized as
the delay is computed based on the time of two different end-points.
Test Procedure
Configure a layer2 point-to-point service between PE1 and PE2.
Configure Y.1731 one-way delay measurement over the service.
Observe the delay measurement delay measurement in the following
conditions of traffic in the network
a. Send 80% of Line-rate traffic with different priorities with different
frame size.
b. Send 40% of Line-rate traffic with different priorities with
different frame size.
c. Without any line traffic
The result of all the 3 conditions above are noted and correlated.
Test Measurement
The following factors needs to be measured to benchmark the result
The average one-way delay
The average one-way delay variation
In the above 3 cases results obtained must be similar.
Praveen & Sudhin Expires August 03,2017 [Page 8]
1. Ideal case
In this case the hardware aspects of processing capacity and the link
level anomalies are not considered. The benchmark is just on the protocol
functioning.In such environment where for an ideal case the system should
expect delay variation to be zero.
2. Practical case
This case is used to benchmark results when delay measurement is done on
physical hardware (like a router).The factors of packet process jitter and
link level delays needs to be considered here.The delay variation in such
cases will defer based on the above parameters on different hardware systems.
Result will very base on the exact hardware.
Delay Variation
+
^ |
| |
| |
+ |
|
| +----------+
| | |
| | |
+------------+----------+-------------+
Time
----->
Traffic (0 to 100 percent line rate)
Impairment
This is to benchmark one-way delay measurement even when both data
and PDUs are dropped in the network using the impairment tool.
Measurement
The results must show similar results before and after this test.
High Availability
During routing engine failover the historical data must not reset.
Praveen & Sudhin Expires August 03,2017 [Page 9]
Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions
with Performance monitoring running over it.There should not be any
crashes,memory leaks.
Soak
The bi directional traffic is send over service over 24 to 48
hours and measure after the stipulated time there must not be
any change in behavior in the network for performance monitoring
Measurement
There should not be any core or crashes,memory leaks.
5.3 Loss measurement without COS Test Procedure
Basic Testing Objective
The test defined methodology for benchmarking data loss in the network
on real customer traffic. The Y.1731 indicates to consider only
in-profile (green) packet for loss measurement. For this, the testing
needs to be done in multiple environment where
a.All data packets from traffic generator are sent with single 802.1p
priority and the network do not have a COS profile defined.
b.All data packets from traffic generator are sent with 0 to 7
values for 802.1p priority and the network do not have a COS profile
defined.
The objective is to benchmark the protocol behavior under different
networking conditions and correlate the data.The objective is not
to test the actual functioning of Y.1731 Loss measurement.The loss
measurement must count only in profile packet, since there is no COS
defined.All the packets must be recorded as green.
Praveen & Sudhin Expires August 03,2017 [Page 10]
Test Procedure
Configure a layer2 point-to-point service between PE1 and PE2.
Configure Y.1731 loss measurement over the service.
Observe the loss measurement in the following conditions of traffic in
the network
a.Send 80% of Line-rate traffic with different priorities with different
frame size.
b.Send 40% of Line-rate traffic with different priorities with different
frame size.
c.Without any line traffic
The result of all the 3 conditions above are noted and correlated.
Test Measurement
The factors which need to be considered is the acceptable absolute
loss for the given network.
Impairment
This is to benchmark loss measurement even when both data and PDUs
are dropped in the network using the impairment tool.
Measurement
When the data is dropped it must show the loss correctly and PM PDUs
are dropped the counting should not be affected,ther should not be
any abnormal output.
High Availability
During routing engine failover the historical data must not reset.
Ideal case there must be 0 packet loss.
Praveen & Sudhin Expires August 03,2017 [Page 11]
Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions
with Performance monitoring running over it.There should not be any
crashes,memory leaks.Each session must record loss measurement correctly.
Soak
The bi directional traffic is send over service over 24 to 48
hours and measure after the stipulated time there must not be
any change in behavior in the network for performance monitoring
Measurement
There should not be any core or crashes,memory leaks.
Result
+----------------------------------+
| Traffic sent |Loss measurement|
| over the service|(without cos) |
| for bi direction| |
+----------------------------------+
| 7 Streams at | Near End = 100%|
| 100% line rate | Far End = 100% |
| with priority | |
| from 0 to 7 | |
+----------------------------------+
| Dropping 50% | Near End 50% |
| of line rate | Far end 100% |
| at near end. | Near End loss |
| | observed 50% |
| | |
+----------------------------------+
| Dropping 50% |Near End 100% |
| of line rate | Far end 50% |
| at far end. | Far End Loss |
| | observed 50% |
+-----------------+----------------+
Praveen & Sudhin Expires August 03,2017 [Page 12]
5.4. Loss measurement with COS Test Procedure
Basic Testing Objective
The test defined methodology for benchmarking data loss in the network on
real customer traffic. The Y.1731 indicates to consider only in-profile(green)
packet for loss measurement. For this, the testing needs to be done in multiple
environment where
a. All data packets from traffic generator are sent with single 802.1p
priority and the network have pre-defined COS profile defined.
b. All data packets from traffic generator are sent with 0 to 7 values
for 802.1p priority and the network have pre-defined COS profile defined.
The COS profile defined needs to have 2 factors
a.COS needs to treat different 802.1p as separate class of packets.
b.Each Class of packets needs to be an defined CIR for the
specific network.
The objective is to benchmark the protocol behavior under different
networking conditions and correlate the data. The objective is not
to test the actual functioning of Y.1731 Loss measurement.
The loss measurement must show in profile packet for each COS levels.
Each COS level must count only its own defined in profile packets.
The Packets, which are termed, as out profile by COS marking must
not be counted.When the traffic is send with single 802.1p priority
the loss measurement must record value only for that particular COS level.
Test Procedure
Configure a layer2 point-to-point service between PE1 and PE2.
Configure Y.1731 loss measurement over the service.
Observe the loss measurement in the following conditions of traffic
in the network.
Praveen & Sudhin Expires August 03,2017 [Page 13]
d.Send 80% of Line-rate traffic with different priorities with
different frame size.
e.Send 40% of Line-rate traffic with different priorities with different
frame size.
f. Without any line traffic
The result of all the 3 conditions above are noted and correlated.
Test Measurement
The factors which need to be considered is the acceptable absolute
loss for the given network.
Impairment
This is to benchmark loss measurement even when both data and PDUs
are dropped in the network using the impairment tool.
Measurement
When the data is dropped it must show the loss correctly and PM PDUs
are dropped the counting should not be affected,there should not be
any abonormal output.
High Availability
During routing engine failover the historical data must not reset.
Ideal case there must be 0 packet loss.
Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions
with Performance monitoring running over it.There should not be any
crashes,memory leaks.Each session must record loss measurement correctly.
Soak
The bi directional traffic is send over service over 24 to 48
hours and measure after the stipulated time there must not be
any change in behavior in the network for performance monitoring
Praveen & Sudhin Expires August 03,2017 [Page 14]
Measurement
There should not be any core or crashes,memory leaks.
Result
+----------------------------------+
| Traffic sent |Loss measurement|
| over the service|(With cos) |
| for bi direction| |
+----------------------------------+
| 7 Streams at | Near End = 100%|
| 100% line rate | Far End = 100% |
| with priority | |
| from 0 to 7 | |
+----------------------------------+
| Dropping 50% | Near End 50%|
| of line rate | Far end 100% |
| at near end | Near End loss |
| for priority | observed 50% |
| marked 0 | (priority 0) |
+----------------------------------+
| Dropping 50% |Near End 100%|
| of line rate | Far end 50% |
| at far end for | Far End Loss |
| priority 0 | observed 50% |
| | (priority 0) |
+-----------------+----------------+
Praveen & Sudhin Expires August 03,2017 [Page 15]
5.5. Synthetic Loss Measurement Test Procedure
5.5.1 Basic Testing Objective
The test defined methodology for benchmarking synthetic loss in the network.
The testing needs to be done in multiple environment where
a. All data packets from traffic generator are sent with single 802.1p
priority and the network do not have a COS profile defined.
The synthetic loss measurement also uses the same 802.1p priority as that
of traffic.
b. All data packets from traffic generator are sent with single 802.1p
priority and the network have pre-defined COS profile defined.The synthetic
loss measurement also uses the same 802.1p priority as that of traffic.
c. All data packets from traffic generator are sent with 0 to 7
values for 802.1p priority and the network do not have a COS profile
defined. The synthetic loss measurement also uses the same 802.1p priority
as that of traffic. Hence 8 sessions are tested in parallel.
d. All data packets from traffic generator are sent with 0 to 7
values for 802.1p priority and the network have pre-defined COS profile defined.
The synthetic loss measurement also uses the same 802.1p priority as that of
traffic. Hence 8 sessions are tested in parallel.
The COS profile defined needs to have 2 factors
1.COS needs to treat different 802.1p as separate class of packets.
2.Each Class of packets needs to have defined CIR for the specific network.
The objective is to benchmark the protocol behavior under different networking
conditions and correlate the data. The objective is not to test the
actual functioning of Y.1731 Loss measurement.
Test Procedure
Configure a layer2 point-to-point service between PE1 and PE2.
Configure Y.1731 loss measurement over the service. Observe the synthetic
loss measurement in the following conditions of traffic in the network
a. Send 80% of Line-rate traffic with different priorities
b. Send 40% of Line-rate traffic with different priorities
c. Without any line traffic
The result of all the 3 conditions above are noted and correlated.
Praveen & Sudhin Expires August 03,2017 [Page 16]
Test Measurement
The factors which need to be considered is the acceptable absolute loss
for the given network.
Impairment
This is to benchmark synthetic loss measurement even when both data
and PDUs are dropped in the network using the impairment tool.
Measurement
When the data is dropped it must not affect the SLM counters but if
synthetic frames are dropped the loss must be shown accordingly.
High Availability
During routing engine failover the historical data must not reset.
Scale
This is to measure the performance of DUT in scaling to "X" CFM sessions
with Performance monitoring running over it.There should not be any
crashes,memory leaks.
Soak
The bi directional traffic is send over service over 24 to 48
hours and measure after the stipulated time there must not be
any change in behavior in the network for performance monitoring
Measurement
There should not be any core or crashes,memory leaks.
Praveen & Sudhin Expires August 03,2017 [Page 17]
6. Acknowledgements
We would like to thank Al Morton of (ATT) for their support and encouragement.
We would like to thank Fioccola Giuseppe of Telecom Italia reviewing our
draft and commenting it.
7. Security Considerations
NA
8.IANA Considerations
NA
Appendix A. Appendix
Authors' Addresses
Sudhin Jacob
Juniper Networks
Bangalore
Email: sjacob@juniper.net
sudhinjacob@rediffmail.com
Praveen Ananthasankaran
Nokia
Manyata Embassy Tech Park,
Silver Oak (Wing A), Outer Ring Road,
Nagawara, Bangalore-560045
Email: praveen.ananthasankaran@nokia.com
Praveen & Sudhin Expires August 03,2017 [Page 18]