Network Working Group Silvija Dry
Internet Draft Fernando Calabria
Expires: April 2007 Ian Yee Yan Fung
Cisco Systems, Inc.
October 11, 2006
Multicast VPN Scalability Benchmarking
draft-sdry-bmwg-mvpnscale-00.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that
any applicable patent or other IPR claims of which he or she is
aware have been or will be disclosed, and any of which he or she
becomes aware will be disclosed, in accordance with Section 6 of
BCP 79.
This document may only be posted in an Internet-Draft.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on April 11, 2007.
Abstract
Multicast VPN (MVPN) is a service deployed by VPN service providers
to enable their customers to use IP multicast applications over VPNs.
With the increased popularity the scalability of deploying such a
Dry, et al. Expires April 11, 2007 [Page 1]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
service is becoming of a great interest. This document defines
standard metric and test methodology for characterizing and comparing
control plane MVPN scalability of Provider Edge (PE) devices that
implement MVPN service.
Table of Contents
1 Introduction...................................................3
2 Document Scope.................................................4
3 Key Words to Reflect Requirements..............................5
4 MVPN Metric Definition.........................................5
5 Test Environment...............................................6
5.1 Test Topologies..........................................6
5.2 Unicast Control Plane Setup..............................7
5.3 Multicast Control Plane Setup............................7
5.4 Data Traffic Characteristics.............................8
5.5 Test Apparatus Considerations............................8
5.6 Considerations for distributed architecture platforms....9
6 Test Categories, Stimulus and Execution Methodology............9
6.1 Steady State Testing....................................10
6.2 Failure Recovery Testing................................11
7 Results Content and Reporting Format..........................13
7.1 Steady State Testing....................................13
7.2 Failure Recovery Testing................................14
8 Test Cases - Steady State Testing.............................14
8.1 "Empty" MVPNs Scale.....................................14
8.2 PIM Enabled VPN C-Interfaces Scale......................16
8.3 PIM Neighborships Scale.................................18
8.4 Default MDT's Multicast State Scale.....................20
8.5 VRF Multicast State Scale...............................22
8.6 VRF Multicast OIF Scale.................................24
8.7 Joined Data MDT Scale...................................26
8.8 Sourced Data MDT Scale..................................28
8.9 Data MDT Reuse..........................................30
8.10 PIM J/P Suppression Effectiveness.......................31
8.11 Additional Tests for Devices Lacking "Efficient" Join/Prune
Suppression...................................................34
8.12 Scale of mVPNs spanning large number of PEs.............34
8.13 Scale of mVPNs with larger amount of state..............36
8.14 Scale of "average" size mVPNs...........................38
9 Security Considerations.......................................40
10 IANA Considerations........................................40
11 Acknowledgments............................................41
12 References.................................................41
12.1 Normative References....................................41
12.2 Informative References..................................41
Dry, et al. Expires April 11, 2007 [Page 2]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Author's Addresses...............................................42
Intellectual Property Statement..................................42
Disclaimer of Validity...........................................42
Copyright Statement..............................................43
Acknowledgment...................................................43
1 Introduction
MVPN is a technology deployed by SPs (Service Providers) as an
overlay of their existing BGP VPN service offering. It enables SP's
customers to use IP multicast applications over BGP VPNs. With the
increased popularity, the scalability of deploying MVPN is becoming
of a great interest. There is, however, no standard method defined to
measure and compare different implementations. This document defines
a metric and methodology for testing and comparing control plane MVPN
scalability of PE devices.
There are multiple proposals on architectures and protocols currently
in IETF for implementing MVPN service. This draft will describe
methodology for benchmarking MVPN scalability only for the
implementations following [ROSEN-8] or portion of [L3VPN-MCAST] which
uses PIM protocol to create 'tunnels' that instantiate MI-PMSIs (or
MDT tunnel) and S-PMSIs (or data MDT). We are limiting scope only to
above mentioned implementation as that is the only one deployed
today.
Before describing the detailed test methodology, it is important to
review the key factors that impact scalability of MVPN deployments:
o The MVPN Metric: a set of variables that when combined indicates
the scalability capabilities of a PE device. MVPN scalability is
multi-dimensional and can not be quantified with single parameter
thus defining such a metric set is necessary. MVPN Metric will be
defined in section 4. The rest of this document focuses on a
methodology that characterizes different aspects of MVPN Metric.
o MVPN design and operational choices (such as choice of PIM
protocol variant or extent of data MDT usage) SP makes impact
overall MVPN scalability. More details on these choices with their
tradeoffs are discussed in [MVPN-DEPLOY]. In this document design
choices most suitable for a goal of any given test case will be
used which may not necessarily be the same as recommended design
choice for realistic deployment.
o MVPN is a service that is never deployed in isolation as it
requires underlying unicast VPN offering. Typically SPs add MVPN
Dry, et al. Expires April 11, 2007 [Page 3]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
service on PE devices that are already deployed and are providing
a large number of other services such as unicast L3VPNs, L2VPNs,
internet access, etc. Therefore, when considering MVPN scalability
in realistic deployments one needs to take into consideration the
level to which PE resources are already utilized and the available
headroom amount remaining. In this document it will be assumed
that MVPN service is deployed on the top of a "minimized" unicast
control plane.
o MVPN Scalability of a PE device is different when the system is
subjected to different stimuli. For example scalability achieved
in steady state is typically higher than when the system is
subjected to network and device specific failures. In this
document limited set of mandatory test stimuli will be defined.
We choose to limit the scope of this document so it is suitable for
standard comparison of disparate implementations. We included set of
test cases (8.12-8.14) that will be helpful to operators with network
engineering for their deployments. Choices of values of variables in
test cases 8.12-8.14 were made using information from the MVPN
requirements survey conducted as part of [MVPN-REQ].
2 Document Scope
This document will describe the MVPN metric and a test methodology to
compare the MVPN control plane scalability of PE devices in the
standardized way.
Test methodology will define standard set of steady state and failure
recovery test cases, their test execution procedures and results
content and reporting format. Standard test environment will also be
defined for each test case.
DUT (Device Under Test) term will be used interchangeably with MVPN
PE device. VPN related terms used in this document are defined in
RFC4364 and RFC2547bis. MVPN related terms used in this document are
defined in [ROSEN-8] and [L3VPN-MCAST].
Note, the deployment of MVPN also consumes resources on P devices in
support of creation and maintenance of MDTs. But, since MVPN
functionality does not reside on P and CE routers, they are beyond
the scope of this document.
Dry, et al. Expires April 11, 2007 [Page 4]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
3 Key Words to Reflect Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119
[Br97]. RFC 2119 defines the use of these key words to help make the
intent of standards track documents as clear as possible. While this
document uses these keywords, this document is not a standards track
4 MVPN Metric Definition
MVPN control plane scalability of PE device can not be described with
single parameter but it requires a set of variables. We call such a
set "MVPN Metric" and define it further in this section.
When providing scalability capabilities of a PE device one MUST
provide values for all of the MVPN metric variables that were used
during the test. For example, one should never claim that a PE device
supports X number of MVPNs without disclosing the values of other
MVPN Metric variables.
The MVPN Metric is defined as a tuple of the following 13 variables:
Global Domain Related Variables
1. Num_mVPN: Number of multicast VPN routing instances configured on
DUT that have MI-PMSI active and forwarding.
2. Num_*G_P: Total number of (*,G) multicast routes on DUT capable of
forwarding and created by PIM P-instance on DUT.
3. Num_SG_P: Total number of (S,G) multicast routes on DUT capable of
forwarding and created by PIM P-instance.
4. Num_OIF_P: Total number of OIFs (outgoing interfaces) on DUT across
all multicast routes created by PIM P-instance.
VPN Domains Related Variables:
1. Num_MC_C_ints: Number of PIM C-interfaces on DUT
2. Num_PIM_C_neigh: Total number of PIM neighbors in PIM C-instances
across all mVPNs on DUT.
3. Num_*G_C: Total number of (*,G) multicast routes across all MVPNs
on DUT capable of forwarding and created by PIM C-instances.
Dry, et al. Expires April 11, 2007 [Page 5]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
4. Num_SG_C: Total number of (S,G) multicast routes across all MVPNs
on DUT capable of forwarding and created by PIM C-instances.
5. Num_OIF_C: Total number of OIFs on DUT across all multicast routes
created by PIM C-instances.
Data MDT (S-PMSI) Related Variables:
1. Num_DataMDT_Src: Total number of data MDTs (S-PMSIs) across all
mVPNs on DUT that are sourced by DUT.
2. Num_DataMDT_Rx: Total number of data MDTs (S-PMSIs)across all mVPNs
on DUT for which DUT is a receiver.
3. Num_DataMDT_SrcFlows: Total number of C-instance (S,G) flows across
all mVPNs on DUT that are mapped to Num_DataMDT_Src.
4. Num_DataMDT_RxFlows: Total number of C-instance (S,G) flows across
all mVPNs on DUT that are mapped to Num_DataMDT_Rx.
5 Test Environment
5.1 Test Topologies
___________ _________ ________ ________ ___________
/ \ / \ / \ / \ / \
| Test |A1 | (DUT) |D2 | RR |B2 | |B4 | Test |
| Apparatus |====| PE1 |====| P |====| PE2 |====| Apparatus |
| | D1| | B1| | B3| | A2| |
\___________/ \_________/ \________/ \________/ \___________/
||
|| _____________
|| / \
|| A3| Test |
++======================| Apparatus |
| (Emulating |
|_PE routers)_|
\_____________/
Figure 1. Test Topology 1
Legend:
D1 (DUT's C-facing interface): DUT's interface that connects to
customer premise router (C-router).
D2 (DUT's P-facing interface): DUT's interface that connects to SP's
core router (P-router).
Dry, et al. Expires April 11, 2007 [Page 6]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
RR/P (Route Reflector/P-router) - single router that will be
performing roles of both P-router and route reflector
PE2 - Will also be referred to as "Remote PE" and is the router
performing PE functionality to assist with evaluation of DUT PE
router.
5.2 Unicast Control Plane Setup
All P facing interfaces MUST use OSPF as IGP. This requirement is
made to provide a standard way to compare end to end convergence
times which depend on the underlying unicast protocol. Only a minimum
number of IGP routes required to establish connectivity should be
seen on the DUT.
All PE routers in the topology including the DUT and emulated PE's
MUST have one iBGP peer to the Route Reflector. DUT SHOULD NOT have
any additional iBGP peering. Only the minimum number of VPNv4 iBGP
routes required to establish site to site VPN connectivity should be
imported on the DUT. There SHOULD NOT be any internet/ipv4 routes
seen on the DUT.
A DUT MUST use static unicast routing on all C facing VPN interfaces.
Only the minimum number of static routes required to establish end to
end connectivity should be seen on the DUT. No dynamic unicast
routing protocol is used in order to minimize processing overhead.
5.3 Multicast Control Plane Setup
In any given test, all default MDT groups MUST use the same multicast
routing protocol. Different tests may require different protocols for
default MDT groups, so refer to individual test cases for the
appropriate multicast configuration. In any test case where PIM-SM
(Protocol Independent Multicast - Sparse mode) is the multicast
routing protocol, a DUT MUST NOT be the RP (Rendezvous Point). Also
dynamic RP discovery protocols SHOULD NOT be used.
For data MDT groups PIM-SSM (Source Specific Multicast) routing
protocol MUST be used.
If there are multiple sources per group in a C-instance then they
MUST be located behind the same PE router.
Dry, et al. Expires April 11, 2007 [Page 7]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Sources emulated by test apparatus ports that are physically directly
connected to DUT (port A1 in Figures 1 and 2) MUST not have IP
address from DUT's connected subnets, i.e. the DUT MUST not be the
first hop router.
Test apparatus ports that are physically directly connected to DUT
(port A1 in Figures 1 and 2) MUST not use IGMP protocol to emulate
multicast receivers. Instead PIM protocol must be used, i.e. the DUT
MUST not be the last hop router.
5.4 Data Traffic Characteristics
For every C-instance mroute there MUST be traffic flow associated
with it and forwarded by DUT.
All C-instance flows SHOULD be transmitted with the same traffic rate
and packet size.
As the focus of this document is on the control plane scalability and
not on forwarding performance the data rate and packet size of
traffic flows can be chosen by user but it MUST be reported in the
test results. However it is suggested to use 10% of "idle system"
throughput [RFC1242] so that it can be easily detected if hardware
forwarding platforms start forwarding in software and at the same
time in case of software forwarding platforms there will be enough
processor headroom left for control plane scaling. By "idle system"
we refer to system with all of MVPN metric variables minimized and
single VPN traffic flow in each direction.
5.5 Test Apparatus Considerations
Different test tools must generate PIM protocol control messages in a
consistent way since they are directly connected to the DUT.
The following MUST be implemented on all PIM sessions on the test
apparatus:
1) PIM Join/Prune aggregation MUST be supported and set such that
80 PIM J/P messages are aggregated in each PDU
Dry, et al. Expires April 11, 2007 [Page 8]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
2) PIM Join/Prune aggregated PDUs MUST be sent at 10 PDUs/sec rate
per PIM session, i.e. this translates to maximum of
80*10*60=48,000 state per minute.
3) In order to closer mimic realistic deployments test apparatus
MUST send all control plane messages in 10 equal size batches
with at least 5 seconds between each batch.
5.6 Considerations for distributed architecture platforms
To fairly evaluate platforms with distributed architectures one MUST
utilize at least two C-facing line cards in the system.
Configuration MUST be such that total number of mVPNs is distributed
evenly across multiple line cards.
6 Test Categories, Stimulus and Execution Methodology
Following are two major test categories that this document addresses:
. Steady State Testing: DUT and network as a whole are not subject to
any failure stimulus/control plane events.
. Failure Recovery Testing: DUT and or network components are subject
to different failure stimulus that introduce one or more control
plane events.
Each test case specified in section 8 MUST be executed for steady
state and for each of six mandatory failure stimulus listed below.
Optionally one can use methodology defined in this document for
additional stimulus.
Mandatory failure stimulus:
1) DUT Power Cycle: Physical power cycle of DUT. All convergence
times MUST be measured from the time DUT's power is turned back
on. This time will be referred to as Tf (the time of failure
recovery action).
2) Main Processor Card Switchover: Physical removal of the active
main processor card in the redundant system. All convergence times
should be measured from the time active processor card is
physically disconnected from the chassis (Tf). This stimulus can
be omitted only for platforms that do not support redundant main
processor cards.
Dry, et al. Expires April 11, 2007 [Page 9]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
3) P-facing Line Card OIR (online insertion and removal): Physical
removal and insertion of P-facing line card. Time between removal
and insertion SHOULD be at least 300 seconds. All convergence
times should be measured from the time line card is physically
inserted into chassis (Tf/Time frame).
4) C-facing Line Card OIR: Physical removal and insertion of C-facing
line card. Time between removal and insertion SHOULD be at least
300 seconds. All convergence times should be measured from the
time line card is physically re-inserted into chassis (Tf).
5) P-facing Link Flap: Physical removal and insertion of the cable
from P router side that is connected to P-facing interface of DUT.
Time between removal and insertion SHOULD be at least 300 seconds.
All convergence times should be measured from the time cable is
physically re-inserted (Tf).
6) C-facing Link Flap: Physical removal and insertion of the cable
from test apparatus side that is connected to C-facing interface
of DUT. Time between removal and insertion SHOULD be at least 300
seconds. All convergence times should be measured from the time
cable is physically re-inserted (Tf).
Since the test execution methodology is similar for all test cases we
will describe it here for both steady state and failure recovery
testing. Any deviation from this will be specified per test case in
section 8.
Multiple iterations of each test are required to determine maximum
value for certain set of variables. A single iteration will be
referred to as a "Test Case Instance".
6.1 Steady State Testing
The following test execution procedure MUST be used for all Test Case
Instances during steady state testing of each test case defined in
section 8 of this document:
1) Ensure the testbed is setup according to Test Setup instructions
of individual test case
2) All Tunnel interfaces MUST be operational and default MDTs
required by the test case MUST be built as expected.
Verification can be done by DUT internal tools.
3) All real and emulated PE devices required by test cases MUST
have all C-instance PIM neighborships (including over Multicast
Dry, et al. Expires April 11, 2007 [Page 10]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Tunnel Interfaces (MTIs)) operational in both directions.
Verification MUST be done by both external test apparatus and
DUT internal tools.
4) All destination test apparatus ports configured to receive
multicast traffic should join all configured multicast groups.
5) All source test apparatus ports configured to transmit multicast
traffic should start transmitting to all multicast groups.
6) All multicast traffic MUST be received at all expected
destination test ports without any packet drops. This MUST be
verified using external test apparatus. If this state can not be
reached within 10 minutes of execution of step 5 test case
instance is considered failed and next test case instance with
reduced value of scaled variable/s needs to be performed.
7) After state in previous step is reached wait 10 minutes and
start collecting data for this test instance required by
individual test case. This time instance is considered steady
state.
8) If any one of following conditions are reached the test case
instance is considered failed and next test case instance with
reduced value of scaled variable/s needs to be performed:
o 100% utilization of system resources (memory, processor,
etc.)
o Failing of any of test case specific criteria or criteria in
steps 1-6 above
The number of Test Case Instances per test case is left to
tester's discretion. However, it is DESIRABLE to have results for
at least 5 test case instances. Having a range of values will help
in variable's characterization. The characterization of a variable
cannot be achieved with only maximum scalability.
6.2 Failure Recovery Testing
The following test execution procedure MUST be used for all Test Case
Instances during failure recovery testing of each test case defined
in section 8 of this document:
1) Execute steps 1-6 from section 6.1
Dry, et al. Expires April 11, 2007 [Page 11]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
2) After steady state in previous step is reached wait 10 minutes
to initiate one of mandatory failure stimulus listed in section
6. Note the time of failure recovery action (Tf) as displayed by
the external test apparatus that is measuring the received
multicast traffic.
3) Using the external test apparatus note the time when THE FIRST
multicast packet has been received on at least ONE of expected
ports. Refer to this time instance as Tree for the first such
encapsulated packet and Trod for the first such decapsulated
packet.
4) Using the external test apparatus note the time when ALL
multicast traffic has been received on ALL expected ports, i.e.
it has returned to the same initial rate (in pps). Refer to this
time instance Trall. If this state can not be reached within 20
minutes of execution of step 2 test case instance is considered
failed and next test case instance with reduced value of scaled
variable/s needs to be performed.
5) After state in previous step is reached execute steps 2-3 from
6.1.
6) If all data verified in step 5 is the same as before failure
wait 10 minutes and start collecting data for this test instance
required by each individual test case
7) If any one of following conditions are reached the test case
instance is considered failed and next test case instance with
reduced value of scaled variable/s needs to be performed:
a. Value of MVPN metric in steady state reached after failure
stimuli (step 6 above) is not the same as in original
steady state.
b. Multicast latency [RFC2432] averaged over all C-instance
multicast flows in steady state after failure recovery
stimuli is more than 10% larger than in original steady
state
c. Failing of any of test case specific criteria or criteria
in steps 1-6 above
The number of Test Case Instances per test case is left to
tester's discretion. However, it is DESIRABLE to have results for
at least 5 test case instances. Having a range of values will help
Dry, et al. Expires April 11, 2007 [Page 12]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
in variable's characterization. The characterization of a variable
cannot be achieved with only maximum scalability.
7 Results Content and Reporting Format
7.1 Steady State Testing
For steady state portion of testing for each test case the following
results MUST be included in the test case report:
1. Maximum value achieved for variables requested to be varied in
individual test case
2. Values of MVPN Metric variables in the test instance in which item
1 of this report was achieved. The MVPN Metric as defined in
section 4 of this document MUST be used
3. Forwarding rate(in pps)[RFC2285] and packet sizes (in bytes) of all
flows in encapsulation direction at DUT
4. Forwarding rate(in pps)[RFC2285] and packet sizes (in bytes) of all
flows in decapsulation direction at DUT
5. Multicast Latency [RFC2432]averaged over all C-instance multicast
flows in encapsulation direction
6. Multicast Latency [RFC2432]averaged over all C-instance multicast
flows in decapsulation direction
7. Utilization of all processors in the system including the main
processor and line card processors where applicable. A description
of the way processor utilization is measured SHOULD be included in
the report.
8. Utilization of all relevant DUT memory components including the
main route processor memory and line cards where applicable.
9. Utilization of any relevant hardware components where applicable
10. Any deviations in DUT configuration from the configuration
defined in this document.
It is DESIRABLE to include in the report items 1-8 above for all
optional test case instances executed, where instead of maximum value
achieved one would report tested value for each test case instance.
Dry, et al. Expires April 11, 2007 [Page 13]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
7.2 Failure Recovery Testing
In addition to data included in steady state reports defined in the
previous section the following MUST be included in the result report
of each failure recovery test case:
1. The worst case end to end traffic convergence time (Trall-Tf)
2. The best case end to end traffic convergence time ((Tre-Tf) for
encapsulation and (Trd-Tf) for decapsulation)
Note: Determination of whether all multicast flows had recovered to
the original traffic rate MUST be made by external test tools and
not by any available tools internal to the DUT or other routers in
the test topology.
It is DESIRABLE to include:
1. A graph from all test tool ports showing transmitted and received
packet rate starting from 60 seconds prior to failure action to 60
seconds after all multicast flows had recovered to the traffic rate
they had prior to the failure.
2. The worst case VPN PIM neighborship convergence time: time interval
from instance Tf to instance when the first C-instance PIM neighbor
across one of MTIs comes up on both DUT and neighboring device
(i.e. "bi-directional" neighborships are established).
3. The worst case VPN PIM neighborship convergence time: time interval
from instance Tf to instance when all expected C-instance PIM
neighbors across one of MTIs comes up on both DUT and neighboring
device (i.e. "bi-directional" neighborships are established).
8 Test Cases - Steady State Testing
8.1 "Empty" MVPNs Scale
Test Objective:
To determine maximum number of MVPN instances that can be
configured and operational on the MVPN PE router. Note that we
refer here to mVPNs as "empty" as amount of PIM neighborships,
interfaces, VRF (Virtual Routing and Forwarding) multicast state
and data MDT's associated with given mVPN is negligible or not
present in this test case.
Dry, et al. Expires April 11, 2007 [Page 14]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT (Shortest Path Tree) threshold for Default MDT
groups: infinity
d. Data MDT's used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 1
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap
direction:1
e. Number of C-instance sources per group in encap
direction:1
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap
direction:1
h. Number of C-instance sources per group in decap
direction:1
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured
on DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data
MDTs:0
n. Number of C-instance (S,G) flows using received data
MDTs:0
Test Execution Procedure:
Dry, et al. Expires April 11, 2007 [Page 15]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Execute number of test case instances where in each test case
instance number of configured mVPNs is varied with the goal of
finding maximum number of mVPNs that can be configured and
operational on DUT. Configured mVPN will be considered operational
if it satisfies all of following:
o Tunnel interface associated with this mVPN is operational
o Default MDT associated with this mVPN is built correctly
according to PIM protocol rules.
o There is at least one PIM neighbor in C-instance associated with
this mVPN across MTI and at least one on respective DUT's L3VPN
interface.
o All traffic flows are being received on ALL expected ports
without any drops.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of number of mVPNs achieved. It is DESIRABLE
to include the same data for at least 5 different values of number
of mVPNs (i.e. for at least 5 test case instances).
8.2 PIM Enabled VPN C-Interfaces Scale
Test Objective:
To determine maximum number of PIM enabled VPN C-interfaces that
can be operational on the MVPN PE router for couple of fixed values
of number of mVPNs. Amount of all other MVPN Metric such as PIM
neighborships and VRF multicast state is minimized in this test
case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
Dry, et al. Expires April 11, 2007 [Page 16]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: varies
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap
direction:0
e. Number of C-instance sources per group in encap
direction:0
f. Number of C-instance OIFs per (S,G) in encap direction:0
g. Number of C-instance multicast groups in decap
direction:0
h. Number of C-instance sources per group in decap
direction:0
i. Number of C-instance OIFs per (S,G) in decap direction:0
j. Maximum allowed number of sourced data MDT's configured
on DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data
MDTs:0
n. Number of C-instance (S,G) flows using received data
MDTs:0
Test Execution Procedure:
Following are steps to execute this test case:
1. Configure 100 mVPNs on DUT and PE2. Execute number of test case
instances where in each test case instance number of PIM enabled
VPN C-interfaces per mVPN is varied with the goal of finding
maximum number of PIM enabled VPN C-interfaces that can be
configured and operational on DUT. Configured VPN C-interface will
Dry, et al. Expires April 11, 2007 [Page 17]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
be considered operational if there is at least one bidirectional
PIM neighbor in VPN C-instance on configured interface.
2. Repeat step 1 for 100*I mVPNs where "i=2
N" where N is integer
value for which either maximum number of PIM enabled VPN C-
interfaces per mVPN becomes smaller than one or maximum number of
mVPNs found in test case 8.1 is reached.
Note that in this test case there SHOULD NOT be any multicast C-
instance traffic sources or receivers thus one MUST modify test
execution procedure from 6.1 and 6.2. For each test case instance
perform steps 1-3,7 from section 6.1. and 1-2,5-7 from section 6.2
for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum achieved value of number of PIM enabled VPN C-
interfaces. It is DESIRED to include the same data for at least 5
different values of PIM enabled VPN C-interfaces (i.e. for at least
5 test case instances).
8.3 PIM Neighborships Scale
Test Objective:
To determine maximum number of PIM C-instance neighborships across
MTIs that PE router can create and maintain. Amount of most of
other MVPN Metric such as VRF multicast state is minimized in this
test case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
Dry, et al. Expires April 11, 2007 [Page 18]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 1
c. Number of remote PEs: varies
d. Number of C-instance multicast groups in encap direction:1
e. Number of C-instance sources per group in encap direction:1
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction:1
h. Number of C-instance sources per group in decap direction:1
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data MDTs:0
Test Execution Procedure:
Number of C-instance PIM neigborships across MTIs is proportional
to product of number of mVPNs DUT belongs to and average number of
PE's belonging to the same mVPNs. Depending on platform
implementation for maintaining PIM neighborships over multi-access
interfaces such as MTI, it is possible that total number of C-
instance PIM nieghborships across MDTs that platform can support
depends on distribution of number of PE routers over mVPNs. For
example, it is possible that 100 mVPNs with average of 100 PEs per
mVPN (which results in 10,000 PIM neighbors) doesn't consume same
DUT resources as 50 mVPNs with average of 200 PEs per mVPN (which
also results in 10,000 PIM neighbors). In order to identify whether
this is the case for given implementation, in this test case we
will vary both number of mVPNs per DUT as well as average number of
PE routers per mVPN.
Test will consist of finding maximum number of C-instance PIM
neighborships across MDTs by varying average number of PE's per
Dry, et al. Expires April 11, 2007 [Page 19]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
mVPN for set of fixed values of number of mVPNs. Procedure is as
follows:
Configure maximum number of mVPNs achieved in test case 8.1
1. Configure 100 mVPNs on DUT. Execute number of test case
instances where in each test case instance number of PE routers
belonging to each mVPN is varied until maximum number of such
PE's is found. All mVPNs should have same number of PE routers.
2. Repeat step 1 for 100*I mVPNs where "i=2
N" where N is integer
value for which either maximum number of PIM enabled VPN C-
interfaces per mVPN becomes smaller than one or maximum number of
mVPNs found in test case 8.1 is reached.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of average number of PE's for every tested
value of number of mVPNs per PE. It is DESIRED to include the same
data for at least 5 different values of number of PE's for each of
tested values of number of mVPNs per PE(i.e. for at least 5 test
case instances per each tested value of number of mVPNs).
8.4 Default MDT's Multicast State Scale
Test Objective:
To determine maximum number of mVPNs and PE routers per mVPN when
P-instance of PIM is using protocol that generates maximum amount
of PIM P-instance state. Amount of most of other MVPN Metric such
VRF multicast state is minimized in this test case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SSM
Dry, et al. Expires April 11, 2007 [Page 20]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
b. RP location for Default MDT groups: NA
c. SPT threshold for Default MDT groups: NA
d. Data MDT's used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 1
c. Number of remote PEs: varies
d. Number of C-instance multicast groups in encap direction:1
e. Number of C-instance sources per group in encap direction:1
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction:1
h. Number of C-instance sources per group in decap direction:1
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data MDTs:0
Test Execution Procedure:
Amount of PIM P-instance state on PE router created by default MDTs
depends in general on choice of PIM protocol variant, number of
mVPNs and average number of PE routers per mVPN. In order to assess
impact PIM P-instance state created by MVPN default MDTs has on
resources test case 8.4 will be repeated with changing PIM P-
instance protocol mode to SSM. Note that test cases 8.1-8.3 use
PIM-SM with SPT threshold of infinity in order to minimize impact
PIM P-instance state has on resources while focusing on
characterizing other variables described in test cases 8.1-8.3
Test Result Report:
Dry, et al. Expires April 11, 2007 [Page 21]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of average number of PEs for every tested
value of number of mVPNs per PE. It is DESIRED to include the same
data for at least 5 different values of number of PEs for each of
tested values of number of mVPNs per PE(i.e. for at least 5 test
case instances per each tested value of number of mVPNs).
8.5 VRF Multicast State Scale
Test Objective:
To determine maximum amount of PIM C-instance state that PE router
can create, maintain and forward on. Amount of most of other MVPN
Metric such as PIM neighborships and P-instance PIM state is
minimized in this test case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 1
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap direction:
varies
e. Number of C-instance sources per group in encap
direction:50
Dry, et al. Expires April 11, 2007 [Page 22]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction:
varies
h. Number of C-instance sources per group in decap
direction:50
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data MDTs:0
Test Execution Procedure:
Total number of C-instance PIM state is proportional to product of
number of mVPNs DUT belongs to and average number of C-instance PIM
state per mVPN. There are four distinct C-instance state types that
depending on implementation might be utilizing platform resources
in different way: (S,G) state with MDT Tunnel interface in OIL;
(*,G) state with MDT Tunnel interface in OIL; (S,G) state with MDT
Tunnel interface as IIF (Incoming Interface) and (*,G) state with
MDT Tunnel interface as IIF. We will refer to state with MDT Tunnel
in OIL as "encap state" and to one with MDT Tunnel as IIF as "decap
state".
In order to simplify testing we will assume fixed number of S per
each G and thus will not exploit impact ratio of (S,G) to (*,G)
state has on platform resources. However we will address couple of
scenarios with respect to ratio of encapsulation do decapsulation
C-instance state.
Note that size of OIL can have significant impact on platform
resources and will be addressed in separate test case: 8.6.
In addition depending on platform implementation it is possible
that total number of C-instance state that platform can support
depends on distribution of that state over number of mVPNs. For
example, it is possible that 100 mVPNs with average of 100 C-
instance routes per mVPN (which results in total of 10,000 C-
instance PIM state ) doesn't consume same DUT resources as 50 mVPNs
with average of 200 state per mVPN (which also results in total of
10,000 C-instance PIM state ). In order to identify whether this is
Dry, et al. Expires April 11, 2007 [Page 23]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
the case for given implementation, in this test case we will vary
both number of mVPNs per DUT as well as average number of PIM C-
instance state per mVPN.
Test will consist of finding maximum number of C-instance PIM state
by varying average number of C-instance PIM state per mVPN for set
of fixed values of number of mVPNs. Procedure is as follows:
1. On DUT and PE2 100 mVPNs achieved in test case 8.1. Setup
environment such that all PIM C-instance state is in encap
direction. Execute number of test case instances using steps 1-7
in section 6.1 where in each test case instance number of C-
instance PIM groups is varied until maximum number of C-instance
PIM state is found.
2. Repeat step 1 for 100*I mVPNs where "i=2
N" where N is integer
value for which either maximum number of PIM enabled VPN C-
interfaces per mVPN becomes smaller than one or maximum number of
mVPNs found in test case 8.1 is reached.
3. Repeat steps 1 and 2 for two more cases of ratios of encap:decap
C-instance state: 100% state is in decap direction;
10%encap+90%decap.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of average number of PIM C-instance state
for every tested value of number of mVPNs per PE. It is DESIRED to
include the same data for at least 5 different values of number of
PIM C-instance state per mVPN for each of tested values of number
of mVPNs per PE(i.e. for at least 5 test case instances per each
tested value of number of mVPNs).
8.6 VRF Multicast OIF Scale
Test Objective:
To determine maximum amount of PIM C-instance OIFs that PE router
can create and maintain. Amount of some of other MVPN Metric such
Dry, et al. Expires April 11, 2007 [Page 24]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
as PIM neighborships and P-instance PIM state is minimized in this
test case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology 1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDTs used: NO
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: 100 and maximum value
tested in 8.5
b. Number of PIM VPN C-interfaces: maximum found in 8.2
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap direction:0
e. Number of C-instance sources per group in encap direction:0
f. Number of C-instance OIFs per (S,G) in encap direction:0
g. Number of C-instance multicast groups in decap direction:
varies
h. Number of C-instance sources per group in decap
direction:50
i. Number of C-instance OIFs per (S,G) in decap direction:
varies
j. Maximum allowed number of sourced data MDTs configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data MDTs:0
Dry, et al. Expires April 11, 2007 [Page 25]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Test Execution Procedure:
Test will consist of finding maximum number of C-instance PIM OIFs
by varying average number OIFs per PIM C-instance state. Maximum
number will be found for couple of values of number of C-instance
PIM state. Test will be executed for two values of number of mVPNs:
1 and maximum value tested in 8.5.All C-instance PIM state will be
in decap direction. Procedure is as follows:
1. For the first iteration of test number of C-instance decap
groups should be set to 25% of maximum value achieved in test
case instance of 8.5 where all C-instance groups were in decap
direction and 100 mVPNs was used. Execute number of test case
instances using steps 1-8 in section 6.1 where in each test case
instance average number of C-instance OIFs per state is varied
in increments of 2 until maximum number of OIFs is reached.
2. Repeat step 1 for 50%,75% and 100% of C-instance decap groups
achieved in test case 8.5.
3. Repeat steps 1 and 2 where maximum value of achieved C-instance
groups from 8.5 is taken for maximum number of mVPNs tested in
8.5.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of OIFs per C-instance state for every
tested value of number of decapsulation groups per PE. It is
DESIRED to include the same data for at least 5 different values of
number of OIFs for each of tested values of number of decap
groups(i.e. for at least 5 test case instances per each tested
value of number of decap groups).
8.7 Joined Data MDT Scale
Test Objective:
To determine maximum number of data MDT's that PE can join. In
order to asses maximum number of data MDT's joined and minimize
resources taken by C-instance mroutes no data MDT reuse is utilized
in this test case.
Dry, et al. Expires April 11, 2007 [Page 26]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: maximum number of mVRFs
obtained in test case 8.5 (refer to it as Vmax)
b. Number of PIM VPN C-interfaces: max found in 8.2 for Vmax
mVPNs
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap direction:0
e. Number of C-instance sources per group in encap direction:0
f. Number of C-instance OIFs per (S,G) in encap direction:0
g. Number of C-instance multicast groups in decap direction:
:[Smax/4] where Smax is maximum number of C-instance groups
obtained in test case 8.5 for Vmax number of mVRFs and case
where 100% of state is in decap direction.
h. Number of C-instance sources per group in decap direction:2
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT: see test
procedure
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data
MDTs:varies
Test Execution Procedure:
Dry, et al. Expires April 11, 2007 [Page 27]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Test will consist of varying number of data MDT's for flows that
have receivers behind SUT (refer to those data MDT's as "received
data MDT's"). During all test case instances total number of C-
instance PIM state MUST remain constant and will be [Smax/4]
rounded to the first lower integer. We will vary total number of
received data MDT's by varying number of mVRFs configured to use
data MDT's at the remote PE that has sources behind it, while
number of data MDTs per mVPNs will be same for all mVPNs that use
them. If given mVPN is using data MDT's in particular test case
instance number of them should be Dvrf=[Smax/(4*Vmax)] rounded to
first lower value that can be represented as 2^I where I is an
integer. Note that number of data MDT's configured and sourced by
DUT MUST be zero in this test case. Procedure is as follows:
1. Configure one mVRF on the remote PE and all flows so that this
mVRF has Dvrf data MDT's utilized and all are sourced at the
remote PE and received at DUT. Execute steps 1-7 in section 6.1
2. Repeat step 1 where number of mVRFs that utilize data MDT's (in
the exact same way as 1 mVRF described in step1) takes following
values 25%,50%,75%,and 100% of Vmax.
3. If platform limit is not reached during execution of step 2,
increase number of data MDT's per mVRF and repeat steps 1 and 2.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
all test case instances executed.
8.8 Sourced Data MDT Scale
Test Objective:
To determine maximum number of data MDTs that PE can source. In
order to asses maximum number of data MDTs joined no data MDT reuse
is utilized in this test case.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
Dry, et al. Expires April 11, 2007 [Page 28]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
1. Topology: Reference Topology #2
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: maximum number of mVRFs
obtained in test case 8.5 (refer to it as Vmax)
b. Number of PIM VPN C-interfaces: max found in 8.2 for Vmax
mVPNs
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap direction:
[Smax/4] where Smax is maximum number of C-instance groups
obtained in test case 8.5 for Vmax number of mVRFs and case
where 100% of state is in encap direction.
e. Number of C-instance sources per group in encap direction:2
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction:0
h. Number of C-instance sources per group in decap direction:0
i. Number of C-instance OIFs per (S,G) in decap direction:0
j. Maximum allowed number of sourced data MDT's configured on
DUT: see test case procedure
k. Number of data MDTs sourced from DUT: set test case
procedure
l. Number of data MDTs with receivers behind DUT: 0
m. Number of C-instance (S,G) flows using sourced data
MDTs:varies
n. Number of C-instance (S,G) flows using received data MDTs:0
Test Execution Procedure:
Reverse role of DUT and remote PE from test case 8.7, where now DUT
is sourcing all data MDTs while remote PE is on the receiving end of
them. Repeat test case 8.7 for this reversed role scenario.
Dry, et al. Expires April 11, 2007 [Page 29]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
8.9 Data MDT Reuse
Test Objective:
To determine maximum number of C-instance flows that can utilize
data MDT's.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: maximum number of mVRFs
obtained in test case 8.5 (refer to it as Vmax)
b. Number of PIM VPN C-interfaces: max found in 8.2 for Vmax
mVPNs
c. Number of remote PEs: 1
d. Number of C-instance multicast groups in encap direction: :
50% of Semax where Semax is maximum number of C-instance
encap groups obtained in test case 8.5 for Vmax number of
mVRFs and case where 10% of state is in encap direction.
e. Number of C-instance sources per group in encap
direction:50
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction: :
50% of Sdmax where Sdmax is maximum number of C-instance
encap groups obtained in test case 8.5 for Vmax number of
mVRFs and case where 90% of state is in decap direction..
Dry, et al. Expires April 11, 2007 [Page 30]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
h. Number of C-instance sources per group in decap
direction:50
i. Number of C-instance OIFs per (S,G) in decap direction:1
j. Maximum allowed number of sourced data MDT's configured on
DUT: 2
k. Number of data MDTs sourced from DUT: 2
l. Number of data MDTs with receivers behind DUT: 8
m. Number of C-instance (S,G) flows using sourced data
MDTs:varies
n. Number of C-instance (S,G) flows using received data
MDTs:varies
Test Execution Procedure:
Test will consist of varying number of C-instance flows that will
utilize data MDT's, while keeping number of C-instance mroutes and
data MDT's constant. By doing this one ca asses data MDT reuse
capabilities of the implementation.
Procedure is as follows:
1. Configure test apparatus such that number of flows using data
MDT's is the same as number of data MDT's, i.e. there is no
data MDT reuse by multiple traffic flows. Execute steps 1-7 in
section 6.1 and 1-8 in section 6.2
2. Repeat step 1 for 100*I where "i=2
N" where N is integer value
for which either maximum number of flows mapped to data MDT is
reached or number of flows becomes equal to number of (S,G) C-
instance mroutes.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
all test case instances executed.
8.10 PIM J/P Suppression Effectiveness
Test Objective:
Dry, et al. Expires April 11, 2007 [Page 31]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
In VRF context MVPN appears to PE routers as multi-access network.
Depending on distribution of VPN sources, RP's and receivers in MVPN
network capability to perform J/P Suppression can have great impact
on overall scale capabilities of PE devices. In particular largest
impact is on scale capabilities of PE router whose attached customers
source large number of multicast flows or host large number of RP's
(refer to such PE as "source" PE)in the network with large number of
PE routers with receivers for those flows. However function of PIM
J/P Suppression is performed by PE devices that have receivers behind
them (refer to such PE as "receiving" PE). Goal of this test case is
to asses capability of "receiving" PE to perform J/P suppression for
large amount of VRF state.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology:
________ _________ ________
/ \ / \ / \
| |R1 | (DUT) |D2 | (RR) |
| Rx1 |====| PE1 |====| P |
| | D1| | B1| |
\________/ \_________/ \________/
||
|| ____________ _______
|| / \ / \
|| | (Emulated) |B3 | |
++===========| PE2 |====| Src |
|| B2| | B4| |
|| \____________/ \_______/
||
|| ____________ _______
|| / \ / \
|| | (Emulated) |B6 | |
++===========| PE3 |====| Rx2 |
B5| | B7| |
\____________/ \_______/
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: NO
Dry, et al. Expires April 11, 2007 [Page 32]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
e. Protocol for Data MDT groups: NA
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 1
c. Number of remote PEs: 2
d. Number of C-instance multicast groups in encap direction:
varies
e. Number of C-instance sources per group in encap
direction:50
f. Number of C-instance OIFs per (S,G) in encap direction:1
g. Number of C-instance multicast groups in decap direction:0
h. Number of C-instance sources per group in decap direction:0
i. Number of C-instance OIFs per (S,G) in decap direction:0
j. Maximum allowed number of sourced data MDT's configured on
DUT: 0
k. Number of data MDTs sourced from DUT:0
l. Number of data MDTs with receivers behind DUT:0
m. Number of C-instance (S,G) flows using sourced data MDTs:0
n. Number of C-instance (S,G) flows using received data MDTs:0
Test Execution Procedure:
For maximum number of VRF state obtained in test case 8.5 for 100
mVRFs and 100% state in decap direction perform following:
1)Establish all PIM session required to emulated defined
topology
2)Perform all C-instance PIM joins from test apparatus port
"Rx2" (B5 in topology diagram)
3)Start all traffic from test apparatus port "Src" (R1 in
topology diagram) and wait until steady state is achieved.
4)On VPN PIM session of test apparatus port "Src" (B2) measure
number of J/P PDUs received in 10 minute (J1) interval and
calculate rate of J/P PDUs as JR1=J1/(60*10)
Dry, et al. Expires April 11, 2007 [Page 33]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
5)Perform all C-instance PIM joins from test apparatus port
"Rx1" (R1) and wait until steady state is achieved on DUT.
6)On VPN PIM session of test apparatus port "Src" (B2) measure
number of J/P PDUs received in 10 minute (J2) interval and
calculate rate of J/P PDUs as JR2=J2/(60*10)
7)If JR2 < 1.2*JR1 we can conclude that DUT is suppressing J/P
messages successfully.
For maximum number of VRF state obtained in test case 8.5 for maximum
number of mVRF and 100% state in decap direction repeat steps 1-7.
Note that no failure recovery testing is required in this test case.
Test Result Report:
Data listed in 7.1 MUST be reported in tabular format for all test
case instances. In addition rates JR1 and JR2 MUST be reported.
Optionally one can report absolute numbers or rates of number of
PIM J/P PDUs transmitted by DUT and PE3 (test apparatus port B5).
8.11 Additional Tests for Devices Lacking "Efficient" Join/Prune
Suppression
Repeat test case 8.5 where for all groups in encapsulation direction
J/P messages are sent from more than one remote PE router, i.e.
number of remote PE routers with receivers becomes additional
variable. One MUST execute test for at least 3 values of number of
remote PE routers with receivers. It is suggested to chose values
such that product of number of PE routers with receivers and number
of mVPNs is 50% of maximum number of C-instance PIM neighbors over
MDTs achieved in test 8.3.
8.12 Scale of mVPNs spanning large number of PEs
Test Objective:
As we noted mVPN scale is multidimensional and depends on number of
variables. While test cases 8.1-8.11 focused on only one or two
variables at the time while minimizing impact of all others, they
don't give good representation of platform capabilities in more
realistic deployment scenarios where none of variables are
Dry, et al. Expires April 11, 2007 [Page 34]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
minimized. Objective of this test case and test cases 8.12 and 8.13
is to asses capabilities of platform in more realistic deployment
scenario. In particular this test case will focus on finding
maximum number of mVPN instances that span large number of PE
routers while they have values for other MVPN variables chosen to
be on the order of magnitude used by MVPN deployments at the time
this draft was written. Specific values are defined in Test Setup
section.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDT's used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 2
c. Number of remote PEs: 500
d. Number of C-instance multicast groups in encap direction: 1
e. Number of C-instance sources per group in encap direction:2
f. Number of C-instance OIFs per (S,G) in encap direction:2
g. Number of C-instance multicast groups in decap direction:9
h. Number of C-instance sources per group in decap direction:2
i. Number of C-instance OIFs per (S,G) in decap direction:2
j. Maximum allowed number of sourced data MDT's configured on
DUT: 2
k. Number of data MDTs sourced from DUT:2
l. Number of data MDTs with receivers behind DUT:8
m. Number of C-instance (S,G) flows using sourced data MDTs:2
n. Number of C-instance (S,G) flows using received data
MDTs:18
Dry, et al. Expires April 11, 2007 [Page 35]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Test Execution Procedure:
Execute number of test case instances where in each test case
instance number of configured mVPNs is varied with the goal of
finding maximum number of mVPNs that platform can support. mVPN
instance here includes C-instance state, OIFs, PIM neighborships
and data MDT's as specified by Test Setup.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Note that, if DUT was determined in test case 8.9 that it
efficiently implements Join/Prune Suppression, then test apparatus
SHOULD be configured such that only one remote PE is sending J/P
message for any given C-instance encapsulation group. On contrary
if 8.9 revealed that DUT platform doesn't perform J/P suppression
for any given encapsulation multicast group J/P MUST be sent from
all of emulated PE routers.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of number of mVPNs achieved. It is DESIRED
to include the same data for at least 5 different values of number
of mVPNs (i.e. for at least 5 test case instances).
8.13 Scale of mVPNs with larger amount of state
Test Objective:
As we noted mVPN scale is multidimensional and depends on number of
variables. While test cases 8.1-8.11 focused on only one or two
variables at the time while minimizing impact of all others, they
don't give good representation of platform capabilities in more
realistic deployment scenarios where none of variables are
minimized. Objective of this test case just is to asses
capabilities of platform in more realistic deployment scenario. In
particular this test case will focus on finding maximum number of
mVPN instances that contain large number of C-instance PIM state
while they have values for other MVPN variables chosen to be on the
order of magnitude used by MVPN deployments at the time this draft
was written. Specific values are defined in Test Setup section.
Test Setup:
Dry, et al. Expires April 11, 2007 [Page 36]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDTs used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 2
c. Number of remote PEs: 50
d. Number of C-instance multicast groups in encap direction:5
e. Number of C-instance sources per group in encap direction:5
f. Number of C-instance OIFs per (S,G) in encap direction:2
g. Number of C-instance multicast groups in decap direction:45
h. Number of C-instance sources per group in decap direction:5
i. Number of C-instance OIFs per (S,G) in decap direction:2
j. Maximum allowed number of sourced data MDTs configured on
DUT: 2
k. Number of data MDTs sourced from DUT:2
l. Number of data MDTs with receivers behind DUT:8
m. Number of C-instance (S,G) flows using sourced data MDTs:25
n. Number of C-instance (S,G) flows using received data
MDTs:225
Test Execution Procedure:
Execute number of test case instances where in each test case
instance number of configured mVPNs is varied with the goal of
finding maximum number of mVPNs that platform can support. mVPN
instance here includes C-instance state, OIFs, PIM neighborships
and data MDTs as specified by Test Setup.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Dry, et al. Expires April 11, 2007 [Page 37]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Note that, if DUT was determined in test case 8.9 that it
efficiently implements Join/Prune Suppression, then test apparatus
SHOULD be configured such that only one remote PE is sending J/P
message for any given C-instance encapsulation group. On contrary
if 8.9 revealed that DUT platform doesn't perform J/P suppression
for any given encapsulation multicast group J/P MUST be sent from
all of emulated PE routers.
Test Result Report:
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of number of mVPNs achieved. It is DESIRED
to include the same data for at least 5 different values of number
of mVPNs (i.e. for at least 5 test case instances).
8.14 Scale of "average" size mVPNs
Test Objective:
As we noted mVPN scale is multidimensional and depends on number of
variables. While test cases 8.1-8.11 focused on only one or two
variables at the time while minimizing impact of all others, they
don't give good representation of platform capabilities in more
realistic deployment scenarios where none of variables are
minimized. While test cases 8.12 and 8.13 assesses two more extreme
cases with respect to number of PE routers and mVPN routes,
objective of this test case is to asses number of mVPNs for the
case where each mVPN represents average size mVPN customer.
Specific values are defined in Test Setup section.
Test Setup:
Following test setup MUST be performed prior to executing this test
case
1. Topology: Reference Topology #1
2. Multicast Configuration:
a. Protocol for Default MDT groups: PIM-SM
b. RP location for Default MDT groups: P router
c. SPT threshold for Default MDT groups: infinity
d. Data MDTs used: YES
e. Protocol for Data MDT groups: SSM
f. Protocol for PIM C-instances: PIM-SM
Dry, et al. Expires April 11, 2007 [Page 38]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
g. RP Location for PIM C-instances: Test apparatus port
closest to the source.
h. SPT threshold for PIM C-instances: zero
3. Multicast Control Plane Profile (all per mVPN except a.; all
from DUT's perspective):
a. Number of MVPNs configured on DUT: varies
b. Number of PIM VPN C-interfaces: 2
c. Number of remote PEs: 100
d. Number of C-instance multicast groups in encap direction:2
e. Number of C-instance sources per group in encap direction:4
f. Number of C-instance OIFs per (S,G) in encap direction:2
g. Number of C-instance multicast groups in decap direction:18
h. Number of C-instance sources per group in decap direction:4
i. Number of C-instance OIFs per (S,G) in decap direction:2
j. Maximum allowed number of sourced data MDTs configured on
DUT: 2
k. Number of data MDTs sourced from DUT:2
l. Number of data MDTs with receivers behind DUT:8
m. Number of C-instance (S,G) flows using sourced data MDTs:8
n. Number of C-instance (S,G) flows using received data
MDTs:72
Test Execution Procedure:
Execute number of test case instances where in each test case
instance number of configured mVPNs is varied with the goal of
finding maximum number of mVPNs that platform can support. mVPN
instance here includes C-instance state, OIFs, PIM neighborships
and data MDTs as specified by Test Setup.
For each test case instance perform steps 1-8 from section 6.1. and
1-7 from section 6.2 for all mandatory stimuli in section 6.
Note that, if DUT was determined in test case 8.9 that it
efficiently implements Join/Prune Suppression, then test apparatus
SHOULD be configured such that only one remote PE is sending J/P
message for any given C-instance encapsulation group. On contrary
if 8.9 revealed that DUT platform doesn't perform J/P suppression
for any given encapsulation multicast group J/P MUST be sent from
all of emulated PE routers.
Test Result Report:
Dry, et al. Expires April 11, 2007 [Page 39]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Data listed in 7.1 and 7.2 MUST be reported in tabular format for
at least maximum value of number of mVPNs achieved. It is DESIRED
to include the same data for at least 5 different values of number
of mVPNs (i.e. for at least 5 test case instances).
9 Security Considerations
Documents of this type do not directly affect the security of
the Internet or of corporate networks as long as benchmarking
is not performed on devices or systems connected to operating
networks.
10 IANA Considerations
This document requires no IANA considerations.
Dry, et al. Expires April 11, 2007 [Page 40]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
11 Acknowledgments
Authors would like to thank Aamer Akhter, Arjen Boers, Yiqun Cai, Min Li,
Amal Maalouf, Mike McBride, Chip Popoviciu and Dan Williston for
their valuable input on content of this draft. We would like to thank
Nick Satsia for his support with test verification of this draft.
12 References
12.1 Normative References
[MVPN-REQ] T. Morin, Ed., "Requirements for Multicast in L3 Provider-
Provisioned VPNs", draft-ietf-l3vpn-ppvpn-mcast-reqts-09.txt
[L3VPN-MCAST] E. Rosen, R. Aggarwal, "Multicast in MPLS/BGP IP VPNs",
draft-ietf-l3vpn-2547bis-mcast-02.txt
[RFC4364] E.Rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private Networks
(VPNs)"
[RFC4601] B. Fenner, M. Handley, H. Holbrook, I. Kouvelas, "Protocol
Independent Multicast - Sparse Mode (PIM-SM):Protocol Specification"
12.2 Informative References
[ROSEN-8] E. Rosen, Y. Cai, I. Wijnands, "Multicast in MPLS/BGP IP
VPNs", draft-rosen-vpn-mcast-08.txt
[MVPN-BCP] Y. Cai, M. McBride, C. Hall, M. Napierala, "Multicast VPN
Deployment Recommendations", draft-ycai-mboned-mvpn-deploy-00.txt
Dry, et al. Expires April 11, 2007 [Page 41]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Author's Addresses
Silvija A. Dry
Cisco Systems, Inc.
sdry@cisco.com
Fernando Calabria
Cisco Systems, Inc.
fcalabria@cisco.com
Ian Yee Yan Fung
Cisco Systems, Inc.
ifung@cisco.com
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Dry, et al. Expires April 11, 2007 [Page 42]
Internet-Draft Multicast VPN Scalability Benchmarking October 2006
Copyright Statement
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Dry, et al. Expires April 11, 2007 [Page 43]