Framework for TCP Throughput Testing
draft-ietf-ippm-tcp-throughput-tm-13

The Abstract says:

   In this framework, 
   TCP and IP parameters are specified and should be configured as 
   recommended. 

I think this could be refined a little to give the scope of the recommendation. For example:
   ... in order to perform the specific tests described.

1. I support the issues raised by Stewart and Ralph in their DISCUSS and COMMENT entries. 

2. This metrics definitions in this document would have benefitted from following the recommendations in Section 5.4 of http://www.ietf.org/id/draft-ietf-pmol-metrics-framework-08.txt. 

I really can't figure out why this document uses 2119 language at all, and without explanation I would really prefer to see it removed. However, even if it is to remain, I don't understand when it is and isn't being used. For instance, part of Ralph's comments contain this:

   At the end, a TCP Throughput Test Device (TCP TTD) should generate a
   report with the calculated BDP and a set of Window Size experiments.
   Window Size refers to the minimum of the Send Socket Buffer and TCP 
   RWND.  The report should include TCP Throughput results for each TCP 
   Window Size tested.

Why are those shoulds and not SHOULDs? (Again, I think lowercase is fine, but I don't see why other things are not all lowercase). The one instance I could find that sounds like it might be 2119-worthy is:

   A compliant TCP TTD SHOULD 
   provide a warning message when the expected test throughput will 
   exceed 10% of the network bandwidth capacity.

But that worries me a little. Is this supposed to be a compliance document for test devices? If so, why is it Informational?

I'm not holding up the document on this basis, but I think the WG and the AD need to answer these questions.

1. Terms given definitions and acronyms in the Terminology section are
sometimes unnecessarily expanded in the body of the document.  Why not
just use the acronyms throughout the document?

2. In section 3:

   - More important, the TCP test host MUST be capable to generate 
   and receive stateful TCP test traffic at the full link speed of the
   network under test.

Which link?  There are several links in the NUT as depicted in Figure
1.1.

3. Add "Bandwidth Delay Product" to the Terminology section.

4. Is "Send Buffer" equivalent to "Send Socket Buffer"?

5. In section 5.2:

   At the end, a TCP Throughput Test Device (TCP TTD) should generate a
   report with the calculated BDP and a set of Window Size experiments.
   Window Size refers to the minimum of the Send Socket Buffer and TCP 
   RWND.  The report should include TCP Throughput results for each TCP 
   Window Size tested.

Is there any advice for the range of window sizes that should be
tested?  Section 3.3.1 requires:

   The TCP TTD MUST allow the Send Buffer and Receive Window sizes to
   be set higher than the BDP, other wise TCP performance will be 
   limited.

Should the window sizes in the test always be greater than BDP?

I support Stephen's discuss position.