Skip to main content

Minutes IETF115: bmwg

Meeting Minutes Benchmarking Methodology (bmwg) WG
Date and time 2022-11-10 09:30
Title Minutes IETF115: bmwg
State Active
Other versions plain text
Last updated 2022-11-10


BMWG Session (IETF-115)
Nov 10, Thursday Session I
0930 - 1130  Local Time (London, UTC-0000)
Other Time Zones: 0330 Chicago CST US
Room: Mezzanine 1-4          ops        bmwg

Agenda (Final) with Minutes

25 people attended

WG Status:

 - IESG Comment Resolution, WG Adoptions, and Proposals

WG Drafts:

 - Multiple Loss Ratio Search
   + There is an update, still work in progress.
   + The draft updated before the bmwg meeting (even though people probably
     did not have enough time to read it yet).
   + Maciek: Major re-write based on comments, using updated terms, removed
   direct references to the implementation/code, goals to minimize test
   duration, find multiple loss ratios, improve repeatability. PROBLEMS: Too
   Long test duration with 2544 binary search, DUT is nested in SUT, search
   must foster repeatability, THPT with non-zero loss may be more popular, but
   zero loss harder to measure (?) Using the term "conditional loss" for
   non-zero loss. + Vratko: this is not an algorithm, it's a methodology with
   typical inputs and a sequence of three phases. New term - FRFRMOL - very

- Benchmarking Methodology for Network Security Device Performance
   - ALL DISCUSS ballots have been resolved!
   - Approval announcement - should occur just after IETF-115 with one final
   review coming

- A YANG Data Model for Network Tester Management
   - Still work in progress, but no outstanding issue on the list. Running Code
   in good shape - single stream. Draft updates include added modifiers,
   removed ETH-specific fields, was not necessary - lock in to single
   technology. More work at the Hackathon: added support for single stream
   modifier. Finding additional areas/communities that interested in this
   testing work.

 + Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814
   Pseudorandom Port Numbers
   - comments exchanged on new results-reporting section.
   - now WG 01 version, consitency checks, scale for CPU cores, etc.  Method
   now works with any IP version, example of stateful NAT64. - measurements
   cinducted in preliminary phase to fill connection tracking table and
   responder table. List of conditions to support repeatable measurements. -
   Scaleability over number of Network flows: new recommendations. Scale over
   CPU cores: when GW is implemented in software, best is to use 1 Core with
   well-known CPU.  Also, made improved recommendations in
   reporting/summarizing function with percentiles. - Discussion topic:
   multiple network flows. does not work with BSD OS. Should we add requirement
   for multiple IP address requirement??? - Beyond network flows and CPU cores,
   what else should be scaled GPS and new sessions per second - - Eduard V 
   there might be interactions between these.  May not have simple  limts in
   software implementations, hardware is different. - Maciek - forward and
   return flows are different from IP hashing perspective, also splitting flow
   directions across cores (multiple)


 + Considerations for Benchmarking Network Performance in Containerized
 Infrastructures - Minh-Ngoc Tran
   - Sharing updates and progress at previous Hackathon
   - New focus on the Benchmarking considerations in the outline, new eBPF
   scenario with AFXDP - Maciek and Vratko will review, also seeking Anuket

 + Benchmarking Methodology for MPLS Segment Routing - Paolo Volpato
   Related Draft:
   - describe the latest updates to address the comments received so far
   - Start with RFC5695, impose a set of labels on the packets for PUSH, NEXT,
   CONTINUE operations. - Addressed comments on the list - equally applicable
   to IPv6 draft as well! - Any implementation experience? - work on this with
   lab tests ACTION for authors!

 + Benchmarking Methodology for IPv6 Segment Routing - Eduard Vasilenko
   Related Draft:
   - describe the latest updates to address the comments received so far
   - Challenge is that this combines IPv6 and MPLS techniques. Special
   procedure is needed, but limit coverage of draft. - Huge update - to logic
   of testing / needs re-review. - Also, agree to show test results in future

  + Problems and Requirements of Evaluation Methodology for
   Integrated Space and Terrestrial Networks
   - Presenter: Zeqi Lai and co-authors
   - updated draft with additional considerations, plus new draft
   - Can we define meaningful benchmarking that fits within BMWG's lab-only
   charter? - Key question: what limited aspect of the many-dimensional ISTN
   problem presented here could be pursued for benchmarking network DUT/SUT? 
   Need concrete test cases - Three steps to emulation benchmarking approach -
   1 Pulic info on the current networks with topology - 2 real data enables
   mimicked LEO behaviors - 3 sepcifiy DUT or SUT, like a virtual emulated node
   - Can evaluate new protocols and new metrics - Need further discussion and
   contribution on all phases - working on platforms for lab-level ISTN
   environment - focus on sic bencxhmarking - Pursue emulation - with peer
   reviewed papers, real experts could join us

 + AOB