Benchmarking Methodology WG (bmwg) Thursday, July 13, 2006 1300-1500 Afternoon Session I (Room 519A) CHAIR(s): Al Morton AGENDA: 0. Agenda bashing (we may need to shuffle a few items) See https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=66 for Agenda updates and Slides 1. Working Group Status (Chair) Hash and Stuffing Draft (WGLC completed, cross-area review comments) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-hash-stuffing-05.txt Terms and Methods for Benchmarking IPsec Devices (WG review and feedback is needed! Please read and comment...) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-term-08.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-meth-01.txt IGP Data plane convergence benchmark I-Ds Update on changes from WG Chair Shepherding Review. 3 WGLC and Cross-Area Review complete. Request for publication. http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-meth-11.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-term-11.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-igp-dataplane-conv-app-11.txt Check the BMWG mail archive for comments: http://www.ietf.org/mail-archive/working-groups/bmwg/current/ 2. IPv6 Benchmarking Methodology - (Popoviciu, Dugatkin or other co-author) (Note: this is a New Work Proposal, moved up because of IETF session shuffling) 1) We would like to review the chages made based on the comments received from the working group. 2) Open the following topics up for further discussion: 2.1) IPv6 Terminology documentation 2.2) Media types to be considered in addition to the ones listed in RFC2544 2.3) Handling Hop-by-Hop Extension Headers 2.4) Evaluating performance with each EH type individually 2.5) IPv4/IPv6 traffic mix proposal 2.6) Relevancy of tunneling performance evaluation for this document 3) We would like to ask the WG to adopt this document as a work group item Related Draft: http://tools.ietf.org/wg/bmwg/draft-popoviciu-bmwg-ipv6benchmarking-01.txt 3. Techniques for Benchmarking Core Router Accelerated Stress Testing. The updated Methdology and Terminology have both been posted and the authors believe it is ready for the first WGLC. (Poretsky) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-term-09.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-acc-bench-meth-05.txt 4. Methodology for Benchmarking Network-layer Traffic Control Mechanisms Update on Methodology document changes from WG comments (Poretsky) http://tools.ietf.org/wg/bmwg/draft-ietf-bmwg-dsmmeth/draft-ietf-bmwg-dsmmeth-02.txt 5. Milestone Status (Chair) ******* New Work Proposals ********* 6. Sub-IP Protection Mechanisms - (Papneja) As per the minutes from the last IETF meeting, it was mentioned that "there was a significant amount of support in the meeting, and the interest in the work item as reached its peak". The next step was to present a proposal (done already), and then submit a merged IETF draft before the next IETF meeting (done). The draft has been posted under the IETF draft directory. This merged document is in response to the action items that the co-authors were asked to put together. See: http://www1.ietf.org/mail-archive/web/bmwg/current/msg01137.html In addition, large numbers of comments were received on the mailing list, and we have been able to incorporate all of the comments into the merged document. In this slot, we would like to cover the following items: 1. Present the highlights of the merged document 2. Summarize the comments received, and actions by us 3. Summarizing the history and commitment of the initial authors to expand the work to larger sectors of the industry (past 4 years) - Carriers, vendor and now even test vendors are supporting the work and implementing. 4. Requesting the working group to have a conclusion on the work item based on the minutes of last meeting, and consistent support on the mailing list. Related Drafts: draft-papneja-mpls-protection-meth-merge-00.txt draft-poretsky-protection-term-02.txt 7. LDP Convergence Benchmarking (Old Work Proposal) - (Papneja) Based on the wide acceptance of LDP as the core MPLS technology for deploying advanced services, it has become evident that following IGP convergence, LDP convergence is also a very important factor in the successful deployment of the LDP protocol. As per the results of survey conducted to highlight the operational experience with LDP for the MPLS working (published under - draft-ietf-mpls-ldp- experience-00.txt), it is important to realize that LDP convergence is a very important factor for wide scale adoption of this protocol. This was one of the primary causes of concern since LDP is very closely tied to IGP convergence. Since we already had submitted this work proposal earlier, we have updated the draft and are re-submitting to the BMWG group. Currently, we have revised the terminology document, and before the next IETF meeting we would like to propose the LDP convergence methodology and applicability document as well. Topics for the presentation would be: 1. Significance of this work, and motivation 2. Summarizing the updates from version 02 3. Requesting the feedback on this document from the operators Related Draft: draft-eriksson-ldp-convergence-03.txt 8. MPLS Benchmarking Methodology (Akhter) This allotment would be used to introduce the draft to BMWG. As it is still a work in progress I am hoping that the presentation might: * generate interest in the work * alert the authors of any obvious mistakes we may have made in methodology * alert the authors of any additional areas we may want to consider The existing draft is at -00. We are working on a -01 revision that should be forthcoming before the meeting. The draft hopes to set the methodology for benchmarking MPLS forwarding. Right now, we are only looking at the forwarding plane and not considering control plane (LDP, RSVP, BGP). Related Draft: http://tools.ietf.org/wg/bmwg/draft-akhter-bmwg-mpls-meth-00.txt 9. SIP Performance Benchmarking (Poretsky) Introduction to work item proposal, see: http://www1.ietf.org/mail-archive/web/bmwg/current/msg01142.html Review of Terminology (-01) Preview of Methodology (-00 to be posted after IETF meeting.) Discussion of collaboration with SIPPING WG (A meeting is to be scheduled at IETF for participants of both working groups to discuss this work item. We will also discuss collaboration with SIP and SPEC in the meeting, see postings to the BMWG mailing list). Related Draft: http://www.ietf.org/internet-drafts/draft-poretsky-sip-bench-term-01.txt 10. Extending the current methodologies to cover wireless LAN switches and wireless LAN meshes (Perser) Statement of Proposed Work -------------------------- Enterprise wireless LANs are now comprised of highly IP-centric devices. Considerable work is being done in various IETF WGs in this area. For example, CAPWAP is defining WLAN switch protocols, and MANET is defining IP over wireless mesh and ad-hoc networks. Modern WLAN switches are Layer 3/4 aware and include many traditional IETF defined functions such as ARP, DHCP and firewalling in combination with wireless functions such as mobility. We propose that BMWG should take up work in support of these technologies. Metrics are required for roaming and scalability performance (related to CAPWAP work) and multi-hop performance and recovery times (related to MANET work), as well as general WLAN switch data plane performance. The proposed work item is as follows: - extend existing LAN switch benchmarking terminology and methodology (RFC 1242, RFC 2285 etc.) to wireless LAN switching devices - create new wireless-specific terminology and methodology for mobility, scalability, and mesh networks The IEEE 802.11T Task Group is focused on terminology and metrics that relate to link layer functions; addressing Layer 3-7 functions is outside their charter. Further, IEEE 802.11 standards do not encompass WLAN switches, as they do not contain implementations of the 802.11 MAC protocol. (CAPWAP is standardizing these switch protocols.) Therefore, metrics for WLAN switches and IP over wireless meshes falls within the BMWG charter. The lack of such metrics today makes it very difficult for vendors to compare and improve the performance of their devices. Scope ----- The scope of the proposed work will cover benchmarking terminology and methodology for measuring the performance of wireless LAN switches and wireless LAN meshes, with particular focus on extending RFC 1242, RFC 2544, RFC 2285 and RFC 2889 to such systems. Benchmarking and performance measurement of 802.11 PHY or link layer functions is a non-objective. Status of existing drafts ------------------------- No drafts currently exist in support of this work. For background on related BMWG wireless discussions, see: http://www3.ietf.org/proceedings/04nov/bmwg.html#cmr Please help contribute to a successful meeting by reading the above I-D(s) and references *before* we meet. To offer comments on BMWG work in progress or the agenda itself, please send email to: bmwg@ietf.org Alternatively, to offer potential agenda items, please email: acmorton@att.com