Large-Scale Broadband Measurement Use Cases
draft-ietf-lmap-use-cases-06

[Ballot comment]
Thank you for an excellent set of edits. This addresses my earlier concerns about the regulator use case text being too politically charged and focuses the document on the real engineering concerns that regulators need addressed. Well done.

[Ballot comment]
I tend to agree with Pete's concerns about marketing fluff in the document, but I think the core message of the document is worth stating.  It would be nice if some of the fluff Pete is complaining about could be removed, but I'm not going to insist on it.

[Ballot comment]

- I do think that this kind of document is useful in
this case and am fine that it becomes an RFC.

- I agree with the comments about the language in the
regulator section in particular.

- 3.5: I'd be interested in a reference to back up
that 80% figure if there's a good one.

- 3.5: The on-demand test that the call centre runs
sounds dangerous to me unless at least accompanied by
a s/w update for the CPE - how do you avoid leaving
bad holes open otherwise?

- section 5: I think the list at the end should
acknowledge that a measurement system might itself
include vulnerabilities that put the user's home
network at risk, e.g. if it required opening a port on
the CPE or if someone could spoof the server-side of
the measurement system and the home-side had a buffer
overrun.

- section 6: I'm wondering if it's really wise to
include both measurement and troubleshooting
capabilities in the same thing. I see the attraction
but the latter may have more onerous security
requirements that would tend not to be met by
implementations of the former.

- section 7, point 4: Would it be worth noting that
the pattern of times for measurements could leak
information about the user's presence/activity at
home? E.g. if measurements are normally taken mostly
at night but for two weeks in august happen precisely
every hour all day long, then I could conclude the
person at that location is on vacation.

- section 7: "informed consent" - hmm, I wonder what
that really means but I like that you recognise the
issue and particularly the secondary uses problem.

- section 7 (or section 6): I think it would be good
to say here that LMAP protocols really do need
signiicant analysis of privacy considerations and it's
likely that those ought end up being explicitly
documented in the relevant specifications.

[Ballot discuss]
In the end, I'm not going to stand in the way of this document if there is consensus on its content; it is just Informational. But on our agenda, we are specifically asked regarding Informational documents: "Is this document a reasonable contribution to the area of Internet engineering which it covers?" And I'd like to take a moment to DISCUSS that question.

There's a significant amount of the discussion of the ISP Use Case that contains a lot of fluff about business and operational models that I think probably doesn't belong in an IETF document, though it's nothing out of hand. But I'm having some real trouble with some of the discussion on the Regulator Use Case. Some examples:

2.2 Regulator Use Case

  Regulators in jurisdictions around the world are responding to
  consumers' adoption of Internet access services for traditional
  telecommunications and media services by promoting competition among
  providers of electronic communications, to ensure that users derive
  maximum benefit in terms of choice, price, and quality.

  Competition is more effective with better information, so some
  regulators have developed large-scale measurement programs. For
  example, programs such as the U.S. Federal Communications
  Commission's (FCC) Measuring Broadband America (MBA)...
 
4.1 Promoting competition through transparency

  Competition plays a vital role in regulation of the electronic
  communications markets. For competition to successfully discipline
  operators' behavior in the interests of their customers, end users
  must be fully aware of the characteristics of the ISPs' access
  offers. In some jurisdictions regulators mandate that transparent
  information imade available about service offers.
 
  [...]

4.2 Promoting broadband deployment

  Governments sometimes set strategic goals for high-speed broadband
  penetration as an important component of the economic, cultural and
  social development of the society. To evaluate the effect of the
  stimulated growth over time, broadband Internet access take-up and
  penetration of high-speed access can be monitored through measurement
  campaigns.

  An example of such an initiative is the "Digital Agenda for Europe"...

It seems completely inappropriate for an Internet *Engineering* Task Force Working Group to be making proclamations about economic competition and the merits (and no notable demerits!) thereof, talking about competitions's "vital role in the regulation of electronic communications markets", and touting programs for "economic, cultural and social development of the society." I think it's a bit unseemly, and at some level (and certainly for folks from different cultures) potentially offensive. (And I haven't mentioned the whole net neutrality discussion that Alissa points out.)

Is this really appropriate material for an IETF consensus document? I can see it as an IAB document, I can see it as an independent submission, but as an IETF WG document?

I'd really like to hear other IESG members' opinions on this. As I said, in the end I won't stand in the way, but before I ABSTAIN, I'd like to hear what others think.

[Ballot discuss]
The draft is well written and easy to read, thank you for that.

I just have one concerns I'd like to chat about:

Section 3.1, second to last paragraph
This discusses the probes nicely in terms of privacy concerns, but leaves out the security concerns that have resulted in any type of probe traffic being blocked in the past.  If you are going to leave the privacy concerns in this section, you should also have the security concerns with network mapping and the ability to use this information to assess open ports as well as path information that could be used in the first case to attack/compromise a host or for denial of service attacks for the first and second case.  Path information is mentioned in 3.3, so perhaps the DoS mention would be better there and the general security along with the privacy in section 3.1.

Later section 5 mentioned the use of an app to gather information from the end user as opposed to their gateway.  Security and privacy considerations should be mentioned here as well since they differ from the the probe scenario.  I would not recommend going in depth on this topic as this is just the use case document, but rather mentioning it.  It would be out-of-scope to get to deep into application security and protections from compromise or the ability to gather privacy information about the user patterns.  A high level statement of those concerns is enough (IMO) and can get addressed in the solution documents.  Apps and desktop tools can be directly associated with a user as opposed to a home and can lead to desktop compromise, so this is different than the previously mentioned privacy concerns in section 3.  I am aware that these types of services are available today via web pages and is up to the user to initiate.

Section 7
First bullet
I'd keep DoS attacks restricted to the degradation of service and have the points on privacy and business confidentiality listed separately as merging them confusing the points and attack vectors.  Both can potentially use information gathered from the probes or the actual management station itself, but in different ways, this isn't clear from the bullet.  The DoS could be launched directly from the management station or information about the network and path to hosts could be used to launch a targeted DoS attack.  The accuracy of the measurement system mentioned in the last part of the bullet makes sense combined with the Dos Attack.
For privacy, you could collect patterns of use to understand user behavior (already in bullet 2).  Additionally, one could use the information gathered from probes to determine open ports and potential vulnerabilities that could be used to compromise the end point (security concern). 
For business confidentiality, I'm not sure what you mean here.  Is this referring to an ability to gather endpoint information to monitor along paths for a business?  I would assume that the probe traffic will be designed to include innocuous information and will just be used for measurement data.

It looks as is the SecDir reviewer had similar concerns with the connections between points made in bullet one, so it would be good to fix this and I hope my above explanation is enough to help you adjust the text and add the additional considerations.  I'm happy to help with the text if needed.  The suggested text to the SecDir review is helpful, but would need to change to address the additional concerns I raised (thanks for your work on that with Hannes!).

http://trac.tools.ietf.org/area/ops/trac/wiki/mib-security

The SecDir review had a few other good points for bullet 6 that you have addressed with proposed text and your proposed text is included here for tracking purposes:
Part of Hannes' request:

* prevent unauthorized access to collected measurement data,
* give users the ability to view collected data,
* give users the ability to exert control over sharing, and
* enforce retention periods.

** Editor's proposed text on all points for bullet 6:
6. a measurement system that does not indicate who is responsible for the collection and processing of personal data and who is responsible for fulfilling the rights of users. The responsible party (often termed the "data controller") should, as good practice, consider issues such as defining:- the purpose for which the data is collected and used; how the data is stored, accessed, and processed; how long it is retained for; and how the end user can view, update, and even delete their personal data. If anonymised personal data is shared with a third party, the data controller should consider the possibility that the third party can de-anonymise it by combining it with other information.

[Ballot comment]
= Section 4.3 =

I would recommend removing or re-wording some bits of language from this section that aren't strictly necessary and could be contentious or turn the reader off. The following seems like it could be deleted (especially since some of the relevant bits of the R&O, relating to transparency, have not been overturned):

"such as the court action negating the FCC R&O"

Similarly, I would suggest the following:

OLD
  Net neutrality regulations do not necessarily require every packet to
  be treated equally. Typically they allow "reasonable" traffic
  management (for example if there is exceptional congestion) and allow
  "specialized services" in parallel to, but separate from, ordinary
  Internet access (for example for facilities-based IPTV). A regulator
  may want to monitor such practices as input to the regulatory
  evaluation. However, these concepts are evolving and differ across
  jurisdictions, so measurement results should be assessed with
  caution.

  A regulator could monitor departures from application agnosticism
  such as blocking or throttling of traffic from specific applications,
  and preferential treatment of specific applications. A measurement
  system could send, or passively monitor, application-specific traffic
  and then measure in detail the transfer of the different packets.
  Whilst it is relatively easy to measure port blocking, it is a
  research topic how to detect other types of differentiated treatment.
    The paper, "Glasnost: Enabling End Users to Detect Traffic
  Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
  [Glasnost] are examples of work in this area.

NEW
  A regulator may want to monitor traffic management practices or compare the performance of Internet service with services offered in parallel to but separate from Internet access (for example IPTV). A regulator could monitor departures from application agnosticism
  such as blocking or throttling of traffic from specific applications,
  and preferential treatment of specific applications. A measurement
  system could send, or passively monitor, application-specific traffic
  and then measure in detail the transfer of the different packets.
  Whilst it is relatively easy to measure port blocking, it is a
  research topic how to detect other types of differentiated treatment.
    The paper, "Glasnost: Enabling End Users to Detect Traffic
  Differentiation" [M-Labs NSDI 2010] and follow-on tool "Glasnost"
  [Glasnost] are examples of work in this area.

[Ballot comment]
Like my commentary on "requirements documents", I don't see the value in publishing use cases as standalone RFCs. I would rather see these maintained in a wiki or living I-D and either not published or published as an appendix to the
corresponding protocol specification.

That being said, I will not stand in the way of this document.

[Ballot comment]
The text in 3.1 about sample size seems poorly justified. should be genericised, I think and would hopefully never be interpreted literally.

  "The understanding can be gained through a
  "panel", i.e. measurement probes deployed to a few 100 or 1000
  customers. The panel needs to include a representative sample for
  each of the operator's technologies (fiber, Hybrid Fibre-coaxial
  (HFC), DSL...) and broadband speeds (80Mb/s, 20Mb/s, basic...). For
  reasonable statistical validity, approximately 100 probes are needed
  for each ISP product."

IESG/Authors/WG Chairs:

IANA has reviewed draft-ietf-lmap-use-cases-04, which is currently in Last Call, and has the following comments:

We understand that, upon approval of this document, there are no IANA Actions that need completion.

While it is helpful for the IANA Considerations section of the document to remain in place upon publication, if the authors prefer to remove it, IANA doesn't object.

If this assessment is not accurate, please respond as soon as possible.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Large-Scale Broadband Measurement Use Cases) to Informational RFC


The IESG has received a request from the Large-Scale Measurement of
Broadband Performance WG (lmap) to consider the following document:
- 'Large-Scale Broadband Measurement Use Cases'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2014-10-07. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  Measuring broadband performance on a large scale is important for
  network diagnostics by providers and users, as well as for public
  policy.  Understanding the various scenarios and users of measuring
  broadband performance is essential to development of the Large-scale
  Measurement of Broadband Performance (LMAP) framework, information
  model and protocol. This document details two use cases that can
  assist to developing that framework.  The details of the measurement
  metrics themselves are beyond the scope of this document.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-lmap-use-cases/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
Measuring broadband performance on a large scale is important for
network diagnostics by providers and users, as well as for public
policy. Understanding the various scenarios and users of measuring
broadband performance is essential to development of the framework,
information model and protocol. This document details two use cases
that can assist to developing that framework. The details of the
measurement metrics themselves are beyond the scope of this document.


Working Group Summary:

The Working Group debated the scope of the work and decided to limit the scope of work in the first phase of LMAP to the ISP and Regulator use cases. There was strong consensus in the favor of this approach.

Document Quality:

This is a use case document. There are various approaches implemented by vendors and providers for functions similar to these performed but LMAP, but no full solutions.

Personnel:

Dan Romascanu is the Document Shepherd. Benoit Claise is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

I reviewed this version as well as the previous ones. This version is IMO ready for publication.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

The document was carefully reviewed by many participants in the WG.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosure

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

Solid consensus

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

Idnits leads to the following, easy to fix in the future versions:



Checking nits according to http://www.ietf.org/id-info/checklist :

----------------------------------------------------------------------------



** There is 1 instance of too long lines in the document, the longest one

being 2 characters in excess of 72.



** There are 2 instances of lines with control characters in the document.



(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

N/A



(13) Have all references within this document been identified as either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

N/A

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

N/A

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

No IANA requests

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

N/A

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.
N/A