Internet Engineering Task Force S. Jacob, Ed.
Internet-Draft K. Tiruveedhula
Intended status: Standards Track Juniper Networks
Expires: December 18, 2016 June 16, 2016
Benchmarking Methodology for PBB-EVPN
draft-kishjac-bmwg-pbbevpn-00
Abstract
This document defines the methodologies for benchmarking performance
of PBB-EVPN.PBB-EVPN is defined in RFC 7623. It is being deployed in
provider network.This document provides the benchmarking
methodologies for PBB-EVPN convergance,data plane,control plane
learning of mac.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 18, 2016.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 1]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
2. Test Topology . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Test Procedure . . . . . . . . . . . . . . . . . . . . . . . 5
4.1. Customer MAC Learning in Data Plane and B-MAC in control
plane . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2. MAC flush for locally learned and remote learned MAC . . 5
4.3. High Availablity . . . . . . . . . . . . . . . . . . . . 5
4.4. Reliablity . . . . . . . . . . . . . . . . . . . . . . . 5
4.5. Convergence Time . . . . . . . . . . . . . . . . . . . . 5
4.6. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.7. SOAK . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.8. Measurement Statistics . . . . . . . . . . . . . . . . . 6
5. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 6
5.1. MAC Learning in Control plane and Data Plane . . . . . . 6
5.1.1. To check the time taken to learn the mac address in
DUT . . . . . . . . . . . . . . . . . . . . . . . . . 6
5.1.2. To check the time taken to learn 100,000 routes from
remote peer by DUT. . . . . . . . . . . . . . . . . . 7
5.1.3. To check the time taken to flush the local entry due
to CE link failure . . . . . . . . . . . . . . . . . 7
5.1.4. To check the time taken by DUT to flush 100,000
macs learned from remote PE after R1 traffic
generator link failure . . . . . . . . . . . . . . . 8
5.1.5. To measure the mac ageing time. . . . . . . . . . . . 8
5.1.6. To check the time taken by DUT to age from
remote PE after stopping the traffic at remote PE. . 9
5.2. Convergence . . . . . . . . . . . . . . . . . . . . . . . 9
5.2.1. To check the time taken by DUT to learn 100,000
routes from local and 100,000 from remote and
measure the time of flood from DUT. . . . . . . . . . 9
5.2.2. To measure the time taken to elect a new DF by adding
a a MHPE. . . . . . . . . . . . . . . . . . . . . . . 10
5.3. Reliability . . . . . . . . . . . . . . . . . . . . . . . 10
5.3.1. To check the whether there is traffic loss due to
routing engine failover for redundancy test. . . . . 10
5.3.2. To check the whether DF election is taking place
properly after the one of the MH PE reboot. . . . . 11
5.4. Scale . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.4.1. To Scale the DUT to 4k PBB-EVPN instances and clear
bgp in DUT without traffic. . . . . . . . . . 11
5.4.2. To Scale the DUT to 4k PBB-EVPN instances and clear
bgp in DUT with traffic. Measure the convergence
time . . . . . . . . . . . . . . . . . . . . . . . . 12
5.5. Soak Test . . . . . . . . . . . . . . . . . . . . . . . . 12
Jacob & Tiruveedhula Expires December 18, 2016 [Page 2]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
5.5.1. To Scale the DUT to 4k PBB-EVPN instances in DUT with
traffic and run the set up for 24hrs . . . . . 13
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
8. Security Considerations . . . . . . . . . . . . . . . . . . . 13
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1. Normative References . . . . . . . . . . . . . . . . . . 13
9.2. Informative References . . . . . . . . . . . . . . . . . 14
Appendix A. Appendix . . . . . . . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14
1. Introduction
PBB-EVPN which is defined in RFC 7623 which describes procedures for
BGP MPLS-based PBB-EVPN.This document defines the methodologies for
benchmarking performance of PBB-EVPN.PBB-EVPN has been implemented
with many varying designs in order to achieve their intended network
functionality.The scope of this document is to provide methodologies
for benchmarking pbb-evpn data, control plane mac learning, mac
flush, mac ageing, convergence, high availablity, reliablity,scale.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. Test Topology
Jacob & Tiruveedhula Expires December 18, 2016 [Page 3]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
Topology Diagram
| Traffic Generator
+----------+
| |
| R1 |
| |
+----------+
|
|
+----------+
| |
| RR |
| |
+----------+-------------|
| |
| |
+----------+ +-----------+
| | | MHPE2 |
| DUT | | |
| MHPE1 | | |
+----------+ +-----------+
| |
+----------+------------
| |
| CE |
| |
+----------+------------ Traffic Generator
Topology Diagram
Figure 1
3. Network
The network consists of 5 routers and 2 traffic generator ports. DUT
is acting as one of MH PE to CE.The RR is acting as route reflector
and core router.R1 is a Single home router running pbbevpn. All four
routers except CE are running mpls,bgp. CE is a dual home connected
to DUT and MH PE1. The testing will be done on DUT to bench mark the
service.DUT and MHPE2 is running PBB-EVPN with SA/AA with CE.The DUT
and and other PE's will be running 100 PBB-EVN instances on 100 sub
interfaces.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 4]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
4. Test Procedure
The test defined to bench mark the performance of PBB-EVPN mac
learning in control pland and data plane. Mac flush,High
Availablity, convergence time in link failures,Reliablity,scale
scenarios.
4.1. Customer MAC Learning in Data Plane and B-MAC in control plane
The Customer MAC will be learned in data plane, test is to measure
the time taken to learn the "X" number of mac's in time "T". The
data plane learning will from the locally connected interface. The
control plane learning of B-MAC is through BGP advertisements from
the remote PE.Let the local learning time be "T" and control plane
learning of the B-MAC time be "T'".
4.2. MAC flush for locally learned and remote learned MAC
The time taken to flush the "X" locally learned mac, let it be "T1"
once the traffic is stopped. The time taken flush the remote mac
which learned by data plane that "X" macs when the traffic is
stopped at remote side.Let the measured time is "T2".
4.3. High Availablity
The traffic is flowing bi direction. The bgp is converged, let the
"X" numbers of macs learned locally and remotely. Then traffic is
flowing at "P" packets per sec. The traffic generator is measuring
the Tx and Rx packets, while the routing engine failover there should
not any packet loss the router tester must show both "P" packet per
seconds.
4.4. Reliablity
This is to measure during any events like adding a new PE or
rebooting on of the MHPE in the same ethernet segment,there should
not any issues in DF election. There should not be any loop. The DF
elction must take place in t time where 4>t>3. The DF election must
work on V mod N. There should not be any loop in traffic.
4.5. Convergence Time
During any events like link failure, hard reset measure the time
taken to learn "X" mac's locally and remotely or time taken to learn
"X" mac locally and remote.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 5]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
4.6. Scale
This is to measure the performance of DUT in scaling to "X" PBB-EVPN
instances. The measured parameters are CPU usage, memory
leak,crashes.
4.7. SOAK
This is measuring the performance of DUT running with scaled
configuration with traffic over a peroid of time "T'". In each
interval "t1" the parameters measured are CPU usage, memory usage,
crashes.
4.8. Measurement Statistics
The test is repeated for "N" times and the value is taken by
averaging the values.
5. Test Cases
5.1. MAC Learning in Control plane and Data Plane
The following tests are conducted to measure mac learning local and
remote.
5.1.1. To check the time taken to learn the mac address in DUT
Objective:
To check the time taken to learn the mac address locally and time
taken to send the locally learned routes to peers.
a. Send 100,000 unicast frames from CE to MHPE1(DUT) working in SA
mode where DUT is the DF so that it can forward the traffic with
different source and destination address,Measure the time taken
to learn these mac in forwarding table.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT (traffic generator) to
send the traffic to the routers. Measurement The DUT mac table must
learn the 100,000 mac address. Measurement The DUT mac table must
Jacob & Tiruveedhula Expires December 18, 2016 [Page 6]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
learn the 100,000 measure the time taken to learn the 100,000 mac
address from locally.
5.1.2. To check the time taken to learn 100,000 routes from remote peer
by DUT.
Objective:
Send 100,000 frames with different SA and DA to DUT from R1 using
traffic generator. Measure the time taken to learn these 100,000
macs from remote peer and program the mac address table.
Procedure:
Configure PBB-EVPN EVI in R1, MHPE2, DUT. All 4 routers except CE
are running mpls, bgp, RR is acting as route reflector to R1, MHPE2
and DUT.Once the bgp comes up check the DUT mac table. For MH PE ESI
must be configured per IFD/Interface. Using RT(traffic generator) to
send the traffic to the routers.
Measurement:
The DUT mac table must learn the 100,000 measure the time taken to
learn the 100,000 mac address from remote peers.
5.1.3. To check the time taken to flush the local entry due to CE link
failure
Objective:
Send 100,000 frames with different SA and DA to DUT from CE using
traffic generator. Then wait to learn all 100,000 mac address. Then
fail the DUT CE link and measure the time taken to flush these
100,000 routes from the mac table.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and DUT.
Once the bgp comes up check the DUT mac table. For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement:
The DUT mac table must learn the 100,000 measure the time taken for
flushing these 100,000 mac address.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 7]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
5.1.4. To check the time taken by DUT to flush 100,000 routes learned
from remote PE after R1 traffic generator link failure
Objective:
Send 100,000 frames with different SA and DA to DUT from R1 using
traffic generator. Bring down the link between R1 and traffic
generator. Then measure the time taken to flush the DUT mac address
table.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT mac table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement:
Measure the time taken to flush 100,000 remote routes from mac table
of DUT.
5.1.5. To measure the mac ageing time.
Objective:
Send 100,000 frames with different SA and DA to DUT from CE using
traffic generator. Then wait to learn all 100,000 mac address. Then
stop the traffic.Wait to see how long it takes to flush these mac
entries due to ageing.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT. All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT mac table. For MH PE ESI
must be configured per IFD/Interface. Using RT(traffic generator) to
send the traffic to the routers.
Measurement:
The DUT mac table must learn the 100,000 measure the time taken for
flushing these 100,000 mac address due to ageing.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 8]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
5.1.6. To check the time taken by DUT to age from remote PE after
stopping the traffic at remote PE.
Objective:
Send 100,000 frames with different SA and DA to DUT from R1 using
traffic generator.After stopping the traffic at remote PE(R1) traffic
generator.Measure the time taken to remove these macs from DUT mac
table.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT mac table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement:
Measure the time taken to flush 100,000 remote macs from mac table of
DUT due to ageing.
5.2. Convergence
The following tests are executed to measure the convergence time in
case of an event or by lerning the mac without any external trigger.
5.2.1. To check the time taken by DUT to learn 100,000 macs from
local and 100,000 from remote and measure the time of flood from
DUT.
Objective:
Send 100,000 frames with different SA and DA to DUT from R1 using
traffic generator. Send 100,000 frames with different SA and DA from
traffic generator connected to CE. The SA and DA of flows must be
complimentary to have unicast flows. Measure the time taken by the
DUT to learn 200,000 in mac table.Measure the flood time period of
DUT.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI
must be configured per IFD/Interface. Using RT(traffic generator) to
send the traffic to the routers.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 9]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
Measurement:
Measure the time taken to learn 200,000 mac address table in DUT.
and measure the flood time of DUT.
5.2.2. To measure the time taken to elect a new DF by adding a a MHPE.
Objective:
Send 100,000 frames from CE to DUT from traffic generator with
different SA and DA. Wait to learn all 100,000 mac address. Then
add R1 to the same Ethernet segment by configuring the same ESI value
configured in DUT,MHPE2 in IFD. Then the new DF election take place
during that time there should not be any loop and measure the time
taken to come up the new DF.Measure the time taken to elect the new
DF.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table. For MH PE ESI
must be configured per IFD/Interface. Using RT(traffic generator) to
send the traffic to the routers.
Measurement:
Measure the time taken for new DF election in DUT and there should
not be any loop or forwarding the BUM traffic back to the same
segment.
5.3. Reliability
These tests are conducted to check after an event there wont be any
change in functionality.
5.3.1. To check the whether there is traffic loss due to routing engine
failover for redundancy test.
Objective:
Send 100,000 frames from CE to DUT from traffic generator with
different SA and DA. Send 100,000 frames from traffic generator to
R1 with different SA and DA so that 200,000 mac address will be
learned in DUT. There is a bi directional traffic flow with 100,000
pps in each direction. Then do a routing engine failover.
Procedure:
Jacob & Tiruveedhula Expires December 18, 2016 [Page 10]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement:
There should not be any traffic loss,No change in the DF role. No
withdraw of any routes.
5.3.2. To check the whether DF election is taking place properly after
the one of the MH PE reboot.
Objective:
Send 100,000 frames from CE to DUT from traffic generator with
different SA and DA. Send 100,000 frames from traffic generator to
R1 with different SA and DA so that 200,000 mac address will be
learned in DUT. There is a bi directional traffic flow with 100,000
pps in each direction. Then reboot DUT since there are 2 MH PE's the
other PE will become the DF then after DUT comes back, then DF
election has to re initiate.
Procedure:
Configure PBB-EVPN EVI in R1,MHPE2,DUT.All 4 routers except CE are
running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement
The DF election has to take place again once the DUT comes back
online.There should not be any DF stuck casefor 100 PBB-EVPN
instances.
5.4. Scale
5.4.1. To Scale the DUT to 4k PBB-EVPN instances and clear bgp in DUT
without traffic.
Objective:
The main purpose of the test the DUT performance to scale 4k PBB-EVPN
instances. Then clear bgp neighbor. There should not be any loss of
routes or any crashes.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 11]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
Procedure:
Configure PBB-EVPN instances in R1,MHPE2,DUT.All 4 routers except CE
are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement
There should not be any loss of route types 2,3 and 4 in DUT.
5.4.2. To Scale the DUT to 4k PBB-EVPN instances and clear bgp in DUT
with traffic. Measure the convergence time
Objective:
The main purpose of the test the DUT performance to scale 4k pbb-evpn
instances with traffic.Then clear bgp neighbor.There should not be
any loss of routes or any crashes. Send 100,000 frames from CE to
DUT from traffic generator with different SA and DA for 5 pbb-evpn
instances.Send 100,000 frames from traffic generator to R1 with
different SA and DA for 5 pbb-evpn instances.so that 1,000,000 mac
address will be learned in DUT. There is a bi directional traffic
flow with 500,000 pps in each direction. Then clear the bgp nei in
DUT after the bgp comes up and started learning the routes, measure
the time taken to learn all 1,000,000 macs in DUT.
Procedure:
Configure PBB-EVPN instances in R1,MHPE2,DUT.All 4 routers except CE
are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Measurement:
The DUT must learn all 1,000,000 mac address.Measure the time taken
to learn 1,000,000 mac in DUT, measure the flood traffic time "T" of
DUT
5.5. Soak Test
Jacob & Tiruveedhula Expires December 18, 2016 [Page 12]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
5.5.1. To Scale the DUT to 4k PBB-EVPN instances in DUT with traffic
and run the set up for 24hrs
Objective:
Configure PBB-EVPN instances in R1,MHPE2,DUT.All 4 routers except CE
are running mpls,bgp,RR is acting as route reflector to R1,MHPE2 and
DUT.Once the bgp comes up check the DUT evpn table.For MH PE ESI must
be configured per IFD/Interface. Using RT(traffic generator) to send
the traffic to the routers.
Procedure:
Measurement:
Take the hourly reading of CPU,process memory.There should not be any
leak,crashes,CPU spikes.
6. Acknowledgements
We would like to thank Al Morton of (ATT) for their support and
encouragement. We would like to thank Fioccola Giuseppe of Telecom
Italia reviewing our draft and commenting it.
7. IANA Considerations
This memo includes no request to IANA.
8. Security Considerations
There is no additional consideration from RFC 6192.
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544,
DOI 10.17487/RFC2544, March 1999,
<http://www.rfc-editor.org/info/rfc2544>.
Jacob & Tiruveedhula Expires December 18, 2016 [Page 13]
Internet-Draft PBB-EVPN Benchmarking Methodology June 2016
9.2. Informative References
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
2015, <http://www.rfc-editor.org/info/rfc7432>.
Appendix A. Appendix
Authors' Addresses
Sudhin Jacob (editor)
Juniper Networks
Bangalore
India
Phone: +91 8061212543
Email: sjacob@juniper.net
Kishore Tiruveedhula
Juniper Networks
10 Technology Park Dr
Westford, MA 01886
USA
Phone: +1 9785898861
Email: kishoret@juniper.net
Jacob & Tiruveedhula Expires December 18, 2016 [Page 14]