Liaison statement
LS - Harmonization of IP Capacity and Latency Parameters: Revision of Draft Rec. Y.1540 on IP packet transfer performance parameters and New Annex A with Lab Evaluation Plan

As an update to our communications from the Q17/12 Interim meeting (Darmstadt,
16-17 October 2018) and the Geneva, 27 November - 6 December 2018 meeting with
the full SG12, we now share further results from the Q17/12 Interim meeting
(Berlin, 5-7 March 2019).

We have completed nearly all planned tests using the Phase 1 Laboratory test
bed that supports development our revised Rec. Y.1540 and new Annex A/Y.1540
specifications of IP Capacity and Latency methods of measurement. We have
compared several existing measurement methods based on TCP and UDP transport.
The Phase 2 Network tests are just starting now.

We continue to invite interested parties to join the testing efforts, and to
coordinate on the development of a new generation of harmonized specifications
of IP Capacity performance metrics and methods of measurement, and other key
performance parameters.

Our key results and current findings from the Berlin meeting are:

•       The text of the IP Capacity definition and Methods of measurement, and
the Phase 1 and Phase 2 sections of the test and evaluation plan were edited
and agreed during the meeting. In the agreed text, the Phase 2 study will be
carried-out in production networks, and lab networks if they are made available
in a timely fashion. This material constitutes revisions to Y.1540 sections and
the new ANNEX A/Y.1540. The meeting agreed to seek Consent on this material in
May, 2018. •       The meeting agreed to prepare the Preliminary Test Summary
(presented for the first time at this meeting) as a new informative APPENDIX to
Y.1540, to provide the supporting test data for the conclusions represented as
requirements in the body and Annex A. •       Even without correction factors,
the UDP measurements are considerably closer to the calibrated shaper rate than
TCP, despite using the most favourable circumstances for TCP (no added delay or
background traffic). With correction for headers that are viewed by the traffic
shaper, UDP measurements using iPerf 2 are within 200ppm of the configured
shaper rate. UDP-based measurements are the benchmark for capacity, accurately
assessing the “ground truth” of the traffic shaper rate under all tested
conditions. •       TCP measurements using iPerf2 underestimate the shaper rate
with or without correction factors. Typical round-trip delay and the presence
of competing/background traffic tend to make TCP-based estimates of available
capacity appreciably worse. •       Another contribution updated the overall
academic Survey, adding material to one section on WiFi performance and one new
section Encrypted stream Network QoS parameters and their relationship to QoE.
Both topics introduce a set of KPIs/metrics prioritized through machine
learning, and models of QoE.  The relevance to active testing includes the
possible additional metrics collected, or input for stream design. •       It
is clear from the surveys presented that Internet subscriber use of TCP
protocol is declining. Video and browser traffic has been shifting to UDP and
higher-layer reliability mechanisms for years, with the most significant growth
in 2018 due to CDN adoption. •       There was a demonstration of a new
prototype measurement tool that makes UDP-based Capacity, Latency, and Loss
measurements while searching for the peak Capacity that can be supported on the
tested path. The meeting provided useful feedback for the developer.

Q17/12 and collaborating SDOs have covered many new areas of investigation, and
are approaching the point where development of coordinated specifications can
proceed. All group members continue to seek additional SDO and individual
support, participation and/or constructive review, so that together, we can
proceed toward the next generation of IP performance metrics.

Attachments:
–       SG12-TD775 with revised Y.1540 clauses and new Y.1540 Annex A;
–       SG12-TD776 with new Y.1540 Appendix, Phase 1 Test Summary.