Liaison statement
LS/r to ITU-T SG11 on “QoS-related work in ITU-T SG11” (SG11-LS 105) [from ITU-T SG12]

Study Group 12 thanks you for your liaison reply concerning the development of
Recommendations closely related to our mandate, as lead Study Group on Quality
of Service and Quality of Experience.

We recognize the revisions to the Scope of Q.3960 (the former Q.FW_Int_sp_test)
to change the emphasis to end users of an IP packet transfer service and
supporting infrastructure, and to “estimate the access speed to the Internet
and to the Internet resources” using measurements that “can be established at
the national or international level” (from the current scope). However, the
terms “Internet Access”, Internet Connection”, and “Internet Service” are not
currently defined in the ITU terms and definitions, so it is necessary to
de-scope aspects describing international applicability. National definitions
of these terms may exist, but with considerable variability, again putting the
applicability of this draft text into question.

In our prior liaison reply, we asked that you take advantage of in-force
Recommendation Y.1540, which covers many critical aspects of the performance of
IP-based Networks (beyond highlighting the requirements of  a single section
which we considered particularly relevant to the draft text you shared with
us). This Recommendation provides a wide range of fundamental and secondary
performance parameters – all defined at the IP layer. The IP-layer is the layer
with end-to-end significance to Internet service providers. Higher layers are
implemented in hosts beyond the control of service providers, and the headers
of higher layers are part of the payload octets conveyed in an IP packet
transfer service. Thus, the availability of IP packet transfer and the
performance of IP packet transfer attempts determine the quality of the
service, according to the metrics defined in Y.1540. Other metrics are under
study and should be removed from the scope from any ITU-T Recommendation
(Q.FW_Int_sp_test) until on-going study is complete.

Your Liaison also mentions a new Recommendation planned for development in
collaboration with ETSI INT, and which overlaps with the existing
Recommendations of Q13/12. The true nature of this Recommendation was not clear
from the description in the liaison, and was only clarified by your counsellor
in response to questions.  Again, this planned effort is apparently un-aware of
in-force Recommendations of the lead SG on the topics where you intend to
design tests. The literature of SG 12 Question 13 must be studied and
appreciated before you proceed:
        G.1010    End-user multimedia QoS categories
        G.1031    QoE factors in web-browsing
        G.1080    Quality of experience requirements for IPTV services
        G.1091    Quality of Experience requirements for telepresence services

In the brief interval allowed for us to prepare a liaison response, we list the
following additional issues with your proposed text under Consent in Q15/11:

•       SG 11 believes that Y.1540 and IPPM RFCs are primarily relevant to the
second phase of their Internet Speed development on test methodology. The SG 12
LR did not ask for references to be added to the list, it asked for the
development of text to take advantage of the in-force specifications. Although
Recommendation Y.1541 is an existing reference, there is no citation in the
text. The same is true for BBF Technical Report 304 (2015), which references
many IETF RFCs, and the TSB Editors could rightly remove these references from
the list (since they are not cited in any way). •       All Figures in the
Consented Internet Speed Test Framework (Q. 3960) are now designated as
examples. Examples or not, the Figures lack the detail to describe service
scope, and so all discussion of Service Level Agreements (SLAs) is
inappropriate. •       In a brief review of the text of Q. 3960, there was
strong opposition expressed to use of a “single Internationally recognized
entity” to provide Internet Speed test facilities. AT&T and DT representatives
expressed that the idea of a single entity having control of this testing was
completely unacceptable (and probably not feasible, due to the multiplicity of
peering arrangements required). Every Recommendation must allow for multiple
independent implementations. •       The measurement test definitions seek to
measure the “absolute value” of transmission speed between different
measurement points. However, there is no known approach which provides such a
value. In fact, the surveys of Internet performance on which this framework is
partly based have been informative as relative measures comparing different
technologies of several service providers, but do not pretend to quantify
anything in absolute terms. It must be made clear that any tests used to judge
the performance of IP network service providers necessarily exclude factors
beyond the service scope (e.g., the User-Network Interface, UNI, and
Network-Network Interface, NNI).

Other comments are embedded in the text of Q. 3960, attached.

We reiterate that descriptions of popular measurement studies do not constitute
a basis for an international standard, as they do not possess the necessary
specificity to guarantee equivalent results from multiple independent
implementations, or necessarily use performance parameters that exhibit
important attributes such as repeatability.

We urge you to continue study of the relevant Recommendations, IETF RFCs, and
other work-in-progress, and to suspend plans to seek approval for all related
activities in SG 11. We invite you to join SG12 at our meeting in June 2016,
where we will be able to share additional background on IP-based network
performance parameters and methods of measurement.

We are unable to send a representative to the Joint meeting you have arranged
with ETSI INT, and we apologize for our absence. If we are to coordinate and
collaborate on this topic, then the first step is to coordinate among the
calendars of key organizations and their interested participants.

Attachment: TD862 Rev.1 (Comments on the Last Call Text of Draft new
Recommendation ITU-T Q. 3960)