IMIX Genome: Specification of Variable Packet Sizes for Additional Testing
draft-ietf-bmwg-imix-genome-05

[Ballot comment]
section 1: s/this draft/this document/g

section 2: I don't see why the sequence length has to
be "not very long" - what's wrong with longer
sequences?

section 4: As a reader who'd never heard of this
before, I found this unclear but got it after a 2nd
reading. I'd suggest adding "This section describes
how to document an IMIX with custom packet sizes, e.g.
representing a 1020 byte packet size as ggg"
somewhere.

section 5: The run length encoding is also unclear.
Do you mean "IMIX - 20abcd40bcd" or something?

[Ballot comment]
In 1 Introduction:

The term IMIX is defined, but Genome isn't. I know what a Genome is, but having an explicit definition would be helpful.

[Ballot comment]
A question out of curiosity - did the authors consider using a pseudo-random sequence to generate indexes to their packet length table?

That would allow the genome for a long sequence to be compacted to polynomial, starting value and length.

[Ballot comment]
No objection to the publication of this document.
However, I have some remarks/questions. Please engage in the discussion.


-  z=MTU is seen as valuable, so MTU MUST be specified if used.
Where? by whom? The tester? Following Section 4 " The tester MUST complete the following table" example", you might need something such as:
    If the z (MTU) is used, the tester MUST specifiy the MTU value in the report

- While this approach allows some flexibility, there are also
  constraints.

  o  Non-RFC2544 packet sizes would need to be approximated by those
      available in the table.

  o  The Genome for very long sequences can become undecipherable by
      humans.

  o  z=MTU is seen as valuable, so MTU MUST be specified if used.

  o  "jumbo" sizes are included.

"jumbo" sizes are included: is this a constraint or an advantage? I thought it was an advantage.


-
OLD:

  The chosen configuration would be expressed the following general
  form:

NEW:
  The chosen configuration would be expressed in the following general
  form:

-
  +-----------------------+-------------------------+-----------------+
  | Source                | Destination            | Corresponding  |
  | Address/Port/Blade    | Address/Port/Blade      | IMIX            |
  +-----------------------+-------------------------+-----------------+
  | x.x.x.x Blade2        | y.y.y.y Blade3          | IMIX - aaafg    |
  +-----------------------+-------------------------+-----------------+

I don't see the port in the examples.
Maybe you meant Address/{Port|Blade} ?
Or maybe you meant Address/{Port AND/OR Blade} ?

- Section 4.
The custom IMIX can use the MTU size, by setting it up in the Genome. However, the MTU semantic is not conveyed.
Is this intentional? I was thinking that Z would be the MTU, with the constraint that the tester MUST specify the MTU value in the report?

- Section 4.
Isn't it an issue that only 26 discrete values are possible?
Don't we have test for which the packet size increases by 1 monotonically?

- Section 5
I guess that the sentence "When a sequence can be decomposed into a series of short repeating sequences, then a run-length encoding approach MAY be used as shown below:" can also apply to custom IMIX. The example doesn't show it. If this is the case, you should mention it.

Editorial
"Genome" versus "genome" throughout document

IESG/Authors/WG Chairs:

IANA has reviewed draft-ietf-bmwg-imix-genome-04, which is currently in
Last Call, and has the following comments:

We understand that, upon approval of this document, there are no IANA
Actions that need completion.

If this assessment is not accurate, please respond as soon as possible.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce:;
CC:
Bcc:
Reply-To: IETF Discussion List
Sender:
Subject: Last Call:  (IMIX Genome: Specification of variable packet sizes for additional testing) to Informational RFC


The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'IMIX Genome: Specification of variable packet sizes for additional
  testing'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2013-05-06. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  Benchmarking Methodologies have always relied on test conditions with
  constant packet sizes, with the goal of understanding what network
  device capability has been tested.  Tests with constant packet size
  reveal device capabilities but differ significantly from the
  conditions encountered in operational deployment, and so additional
  tests are sometimes conducted with a mixture of packet sizes, or
  "IMIX".  The mixture of sizes a networking device will encounter is
  highly variable and depends on many factors.  An IMIX suited for one
  networking device and deployment will not be appropriate for another.
  However, the mix of sizes may be known and the tester may be asked to
  augment the fixed size tests.  To address this need, and the
  perpetual goal of specifying repeatable test conditions, this draft
  defines a way to specify the exact repeating sequence of packet sizes
  from the usual set of fixed sizes, and other forms of mixed size
  specification.





The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-bmwg-imix-genome/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-bmwg-imix-genome/ballot/


No IPR declarations have been submitted directly on this I-D.

Note added 'I've reviewed the discussion on this draft through it's revisions back to IETF 80. I don't believe there are any criticisms remaining that would be considered blocking.

As noted (minutes 85) SOB states that that this method is not likely to be used to produce representation of the real world, the real world is not consistent. I think that we can be abundantly aware of the limitations and find utility in this representation.

Seeing no additional concerns during the WGLC. I'm prepared to call  this document done and ready to advance.
'