Liaison statement
LS/r on current status of the draft Recommendation ITU-T Q.3961 (reply to SG12-LS59)

This liaison answers SG12-LS59.

The joint meeting of ITU-T SG11 and ETSI TC INT took place in Geneva on 12
March 2019 and they discussed the incoming liaison statements submitted on this
subject (SG12-LS73, SG12-LS70, SG12-LS59, SG12-LS58, ETSI TC STQ-STQ(18)059029,
ETSI TC STQ-STQ(19)060067r2). ETSI TC INT and ITU-T SG11 would like to thank
ITU-T SG12 for the liaison statement (SG12-LS59), on Draft ITU-T Q.3961
“Testing methodologies of Internet related performance measurements including
e2e bit rate within the fixed and mobile operator’s networks” and inform the
group about the following: -       ETSI TC INT informed ITU-T SG11 that any
references to ETSI TS 103 222-1 have been deleted from ETSI TS 103 427. ETSI TC
INT informed ITU-T SG11 that they decided to endorse ITU-T Q.3960 and close
this work item; -       All revisions as well as the proposed new Annex to
Recommendation Y.1540, “Internet protocol data communication service – IP
packet transfer and availability performance parameters”, are out of the scope
of draft ITU-T Q.3961 because it describes mechanisms of functioning and
optimization of TCP/IP. It is not dealing with the end-to-end customer
performance. Moreover, the approach defined in Annex to Y.1540 can be used in
laboratory or controlled environment; -       With regard to provide quotes
“Coming to the disclaimer of OECD we can say as reported correctly by the
regulators page, that The locations and technical features of the broadband
connections have been provided by the service users themselves, who are
entirely and exclusively liable for their accuracy. The speed resulting from
the measurement as the final speed of the connection is also affected by a
series of factors, such as the quality of cables, connections and equipment or
electromagnetic interference”, this disclaimer is correct and reports
components that may heavily affect the user experience with possible impact on
churn. For this reason having a set of agreed test specifications that can
provide along with GPS coordinates the position of potential lack of customer
performance to be analysed.

Following discussion in July 2018 and the TSAG request to organize close
collaboration between ITU-T SG11 and ITU-T SG12 on this matter, in March 2019,
ITU-T SG11 organized a Workshop on “Benchmarking of emerging technologies and
applications. Internet related performance measurements” which was attended by
many ITU-T SG11 delegates and it was well received by the audience. ITU-T SG12
experts were invited to attend this event but, unfortunately, they were not
able to. Please note that among the outcomes (SG11-TD813/GEN) of the Workshop
on “Benchmarking of emerging technologies and applications. Internet related
performance measurements” (during ITU-T SG11 meeting) related to draft ITU-T
Q.3961 are: -       Benchmark is “evaluation of performance value/s of a
parameter or set of parameters for the purpose of establishing value/s as the
norm against which future performance achievements may be compared or assessed”
(ITU-T E.800); -       Benchmarking is “performance tests of a system based on
a suite of standardized performance tests. The main purpose of a performance
benchmark is to produce a metric that can be rated and compared with the metric
values produced by other systems using the same benchmark” (ITU-T Q.3930); -   
   The benchmarking approach can be used for performance assessment of
parameters of different systems, networks, services and applications (e.g. call
drops, call set-up delay, CPU load, latency, jitter, download transmission
speed, upload transmission speed, etc.); -       The approach highlighted in
ITU-T Q.3960 (2016) and the proposed draft ITU-T Q.3961 is compliant with Net
Neutrality regulation 2015/2120 from BEREC and OECD 2014 report, underlying
that TCP protocol is widely used by customer application.

So far, ITU-T SG11 does not see relevant contributions from ITU-T SG12 experts,
which can be integrated into draft ITU-T Q.3961 in order to define testing
methodologies of Internet related performance measurements from a customer
perspective. Moreover, following the results of the Workshop mentioned above,
it was noted that draft ITU-T Q.3961 is in line with BEREC “Net Neutrality
Regulatory Assessment Methodology” (2017), presented by the BEREC
representative.

Finally, ITU-T SG11 will continue the work on draft ITU-T Q.3961 considering
any modifications which may need to be made. The beneficial contributions from
ITU-T SG12 experts on supporting the approach specified in draft ITU-T Q.3961
are encouraged by ITU-T SG11.

Attachment: 1
SG11-TD813/GEN: Outcomes of the ITU Workshop benchmarking of emerging
technologies and applications. Internet related performance measurements, 11
March 2019, Geneva