Testing Eyeball Happiness
draft-baker-bmwg-testing-eyeball-happiness-05

[Ballot comment]
In the timing requirements for each metric, it's confusing whether the requirement is 0.1 ms error over 60 seconds, or 1ppm, since both are mentioned.

[Ballot comment]
The Security Considerations section should mention the potential threat of overloading production networks if the conditions described in the last paragraph in Section 1 are not respected and advise care in using the test procedures in operational networks.

[Ballot comment]
(1) 2.2 says to do "at least N" runs when Bob has N addresses, but
isn't that a bit misleading when alice has M addresses as well given
that you're interested in the min and max over (I guess) all
combinations of the (M,N) pairs that can ever work, blocking all but
one each time? Maybe characterising the number of iterations like
that would be better?

(2) I wasn't clear if these metrics are also intended to be used
e.g. for a config with a middlebox between router1 and router2 that
enables one of Alice's IPv6 addresses setup a TCP session with one
Bob's IPv4 addresses.  I'm not asking for anything in particular, I
just wondered and didn't get an answer from the text.

Editorial nit-like stuff:

- Saying "installing" "Path MTU issues" is awkward at best and
possibly misleading - maybe put the word "installing" in each of the
bullets but the last, and say "creating path MTU issues" for the
last bullet and merge the next paragraph into that bullet?

- Section 2.2: the sentence "Different measurement trials revise..."
is hard to understand and I think could just be omitted.

- 3wHS is a bit cryptic in Figure 2 - if you just add the
abbreviation to the text describing figure 2, that'd be ok, as in
"three-way handshake (3wHS)", or maybe even make that paragraph be
the caption for figure 2.

- 2.3.1: Name == "Session Setup Interval" but then you say that
"Session setup time" is measured in ms. Better to use the
terminology consistently. Similarly the "units" text in 2.3.2 &
2.3.3 also seems wrong. Just saying "Units: milliseconds" would be
fine.

- Seems odd for the measurement point(s) text in 2.3.2 and 2.3.3 to
refer to 2.3.1 but to then repeat the text for Timing. Same thing
for 2.3.4 I guess.

- I guess I don't understand what a "Descriptive Metric" might be,
but I assume the intended readership would.  I also don't see how it
has "units" of ms if it requires text to be understood? If there's
an RFC that defines this term then referencing that would be
good.

State changed to In Last Call from Last Call Requested.

The following Last Call Announcement was sent out:

From: The IESG
To: IETF-Announce
Reply-To: ietf@ietf.org
Subject: Last Call:  (Testing Eyeball Happiness) to Informational RFC


The IESG has received a request from an individual submitter to consider
the following document:
- 'Testing Eyeball Happiness'
  as an Informational
RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2011-09-27. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  The amount of time it takes to establish a session using common
  transport APIs in dual stack networks and networks with filtering
  such as proposed in BCP 38 is a barrier to IPv6 deployment.  This
  note describes a test that can be used to determine whether an
  application can reliably establish sessions quickly in a complex
  environment such as dual stack (IPv4+IPv6) deployment or IPv6
  deployment with multiple prefixes and upstream ingress filtering.
  This test is not a test of a specific algorithm, but of the external
  behavior of the system as a black box.  Any algorithm that has the
  intended external behavior will be accepted by it.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-baker-bmwg-testing-eyeball-happiness/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-baker-bmwg-testing-eyeball-happiness/


No IPR declarations have been submitted directly on this I-D.

Shepherd's Write-up for
draft-baker-bmwg-testing-eyeball-happiness-05

This is not a BMWG document, but BMWG Participants have reviewed it and
now appear to be satisfied (since all requested changes have been adopted).
It usually takes a year or two to put a new work item on BMWG's charter
and adopt one or more drafts to address the item(s). From the outset,
the goal for this memo was to work on a much faster schedule.

(1.a) Who is the Document Shepherd for this document? Has the
Document Shepherd personally reviewed this version of the
document and, in particular, does he or she believe this
version is ready for forwarding to the IESG for publication?
Al Morton, yes and yes.

(1.b) Has the document had adequate review both from key WG members
and from key non-WG members? Does the Document Shepherd have
any concerns about the depth or breadth of the reviews that
have been performed?
This is not a BMWG document but BMWG Participants have reviewed it and
now appear to be satisfied (since all changes requested by key participants
have been adopted). The work has also been presented in v6ops, and
similar testing (but not this exact procedure) has been presented there.

(1.c) Does the Document Shepherd have concerns that the document
needs more review from a particular or broader perspective,
e.g., security, operational complexity, someone familiar with
AAA, internationalization or XML?
Absolutely not.

(1.d) Does the Document Shepherd have any specific concerns or
issues with this document that the Responsible Area Director
and/or the IESG should be aware of? For example, perhaps he
or she is uncomfortable with certain parts of the document, or
has concerns whether there really is a need for it. In any
event, if the WG has discussed those issues and has indicated
that it still wishes to advance the document, detail those
concerns here. Has an IPR disclosure related to this document
been filed? If so, please include a reference to the
disclosure and summarize the WG discussion and conclusion on
this issue.
No IPR has been disclosed.

(1.e) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with
others being silent, or does the WG as a whole understand and
agree with it?
All feedback from key individuals has been incorporated, both from e-mail
and BMWG session discussions - this has not been controversial.
Further, we have been able to apply the principles of the
"Guidelines for Considering New Performance Metric Development" memo
developed in the PMOL working group.


(1.f) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in
separate email messages to the Responsible Area Director. (It
should be in a separate email because this questionnaire is
entered into the ID Tracker.)
Absolutely not.

(1.g) Has the Document Shepherd personally verified that the
document satisfies all ID nits? (See the Internet-Drafts Checklist
and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
not enough; this check needs to be thorough. Has the document
met all formal review criteria it needs to, such as the MIB
Doctor, media type and URI type reviews?
Yes. The example address space used appears to be appropriate:
http://tools.ietf.org/html/rfc5737
http://tools.ietf.org/html/rfc3849


(1.h) Has the document split its references into normative and
informative? Are there normative references to documents that
are not ready for advancement or are otherwise in an unclear
state? If such normative references exist, what is the
strategy for their completion? Are there normative references
that are downward references, as described in [RFC3967]? If
so, list these downward references to support the Area
Director in the Last Call procedure for them [RFC3967].
Yes. One Normative reference is progressing in the v6ops WG.

(1.i) Has the Document Shepherd verified that the document IANA
consideration section exists and is consistent with the body
of the document? If the document specifies protocol
extensions, are reservations requested in appropriate IANA
registries? Are the IANA registries clearly identified? If
the document creates a new registry, does it define the
proposed initial contents of the registry and an allocation
procedure for future registrations? Does it suggest a
reasonable name for the new registry? See [RFC5226]. If the
document describes an Expert Review process has Shepherd
conferred with the Responsible Area Director so that the IESG
can appoint the needed Expert during the IESG Evaluation?
Yes. No IANA actions required.

(1.j) Has the Document Shepherd verified that sections of the
document that are written in a formal language, such as XML
code, BNF rules, MIB definitions, etc., validate correctly in
an automated checker?
NA


(1.k) The IESG approval announcement includes a Document
Announcement Write-Up. Please provide such a Document
Announcement Write-Up? Recent examples can be found in the
"Action" announcements for approved documents. The approval
announcement contains the following sections:

Technical Summary
Testing "eyeball happiness" is a euphemism for measuring user-perceived
session response time in the presence of a dual-stack implementation and
the possibility to communicate successfully using only one of the IP
address families. In essence, the use of multiple address familes during
a transition period should not be detrimental to user-perceived performance,
or there will be justified resistance to IPv6 adoption among users.
This memo describes the background, considerations, procedures and metrics
for measuring a close surrogate for session response time by
measuring the DNS query/response and transport layer handshaking,
because no application-layer
communication takes place without these steps and this generalizes the
procedure to be applicable to many user applications/implementations.

Working Group Summary
This is not a BMWG document but BMWG Participants have reviewed it and
now appear to be satisfied (since all changes requested by key participants
have been adopted). The work has also been presented in v6ops, and
similar testing (but not this exact procedure) has been presented there.

Document Quality
Although there are no exact implementations of the test procedures,
the specifications are clear and unambiguous, and should be straightforward
to follow.