Terminology for Benchmarking IPsec Devices
draft-ietf-bmwg-ipsec-term-12

[Ballot comment]
- Section 3.1.2: "public key cryptography" should be "public key
encryption" (that's the term used in RFC 2409, and digital
signatures are also a form of public key cryptography).

[Ballot discuss]
I am picking up Pasi's DISCUSS on this document.  Some were moved to COMMENT status, but the authors should also consult Pasi's COMMENT positions (if any).

I have reviewed draft-ietf-bmwg-ipsec-term-12, and have couple of
concerns about the terminology. For any other document, these
would be just editorial nits, but this is a terminology document,
after all:

- The text in Section 3.1.1 and 7.4 seems to apply only to IPsec
(ESP/AH) SAs, but not IKE SAs?

- Section 3.1.2: I would strongly suggest avoiding the term
"nonrepudiation" -- it's so overloaded that it's hard to know what it
means (if anything). This part of the text doesn't seem very relevant
to benchmarking anyway, so it probably could be just deleted.

- Section 7.3.1: This text seems to confuse the inputs to IKE Phase 1
(information configured before Phase 1 is run), the actual Phase 1 of
the protocol, and the output of that phase (Phase 1 SA or ISAKMP/IKE
SA).

- Section 7.3.1: "SPI information" probably should be "IPsec SA
information" or something like that.

- Section 7.3.2: This text seems to confuse the Phase 2 of the IKE
protocol, and its output (IPsec SA(s)).

- Section 7.6.1/7.6.2/7.6.3/7.6.4: This text seems to conflate the
"Phase 1 initiator" and "phase 2 initiator". While it's common to have
an "IPsec client" that only acts as "Phase 1 Initiator", the client
will also act as "Phase 2 Responder", since either end can initiate
phase 2 exchanges.

- Section 7.6.1/7.7.1/7.7.3/7.13/10.4.4/10.5.1/10.5.3/and several
other sections: There's no such thing as "IKE Phase 2 SA"; these are
not IKE SAs, but IPsec SAs.

- Section 7.7.1: I found this term really confusing, especially
considering the rest of the document seems to use it also in cases
where more than one IPsec SA pair exists. For example, Section 7.13
talks about "IKE Phase 1 SA to IKE Phase 2 SA ratio" for the IPsec
tunnels -- but according to this definition, the ratio can't be
anything else than 1 (otherwise it's not an "IPsec tunnel")!  And
Section 10.1.2 talks about establishing several Tunnels with a single
IKE SA, which isn't consistent with this definition.

- Section 7.8.2: The figure here is really confusing: you can't use
transport mode SAs this way...? (unless you do something like RFC
3884)

- Section 7.9.1: MD5-HMAC should be spelled "HMAC-MD5"; likewise,
"HMAC-SHA1" instead of "HMAC-SHA"; and there's no such thing as
AES-HMAC.

- Section 7.9: The text should also cover combined mode algorithms,
like CCM and GCM (or explicitly state they're not covered by
this document).

- Section 10.2.2 and 10.2.3: I'm not quite sure what "The end points
(i.e. hosts, subnets) should NOT see any fragments at ANY time.  Only
on the IPsec link, fragments MAY occur." means.  Is this text trying
to say that when you're doing benchmarking, the devices under test
should be configured to do post-encryption fragmentation?  Or that the
endpoints generating the plaintext benchmarking fraffic should be
configured to use small enough packets that fragmentation see
fragments?

- Section 10.3.1: the terms "initiator" and "responder" don't seem
applicable here; perhaps sender/receiver?

- Section 10.4.1: the definition seems to cover only packet losses
before encryption or after decryption, but not between these two
points. Is this intentional? (it seems measuring this would be
challenging, since it the endpoints can only see how many frames were
dropped, but not where exactly)

- Section 10.8.2: it seems this should be "IPsec integrity check DoS
resiliency rate" or something -- it's not about IKE Phase 2.

- Section 10.8.3: likewise, this is not about Phase 2, so perhaps
"IPsec replay attack DoS resiliency rate"?

[Ballot comment]
The Gen-ART Review by Joel Halpern on 16-Oct-2009 raised some editorial
  concerns.

  The document is missing the IANA considerations placeholder.

  There seem to be a number of references that are missing.  RFC1962
  is an example of the problem.

  The "Issue" in section 7.5 looks like a reasonable issue, but it
  appears to have nothing to do with selectors, the subject of 7.5.

  The definitions in section 7.7.3 and 7.7.4 define the terms
  "Established Tunnel" and "Active Tunnel" and defines them as "An
  IPsec device ...".  The other tunnel definitions are in terms of the
  association.  It is quite jarring to see a device referred to as a
  tunnel.  Are these terms supposed to be Tunnel Endpoints?

  The definition of 7.10.1 describes the properties of AH, but does not
  actually say what it is.  Instead of starting "Provides", it probably
  should start something like "A protocol header which provides".  A
  similar comment applies to 7.10.2 on ESP.

[Ballot discuss]
The Gen-ART Review by Joel Halpern on 16-Oct-2009 raised two issues
  that need to be resolved.

  First, in section 3.1.1, where Security Association is introduced, the
  text reads:

    "An SA is a relationship between two or more entities ..."

  But, section 7.4 formally defines Security Association as:

    "It is a negotation agreement between two IPsec devices ..."

  Is it always two?  If so, please remove "or more" in section 3.1.1.
  Also, is it always a negotiation agreement?  Earlier text talked about
  manual provisioning of Security Associations.

  Second, in section 10.2, the throughput measurement is defined in terms
  of packets per second.  This is probably the dominant factor.  However,
  some implementations may well be limited by cryptographic processing,
  and therefore have different packet processing performance for different
  ranges of packet sizes.  Section 10.2 should say something about the
  assumptions or requirements relative to packet size.  This seems to be
  important enough that it should not be assumed that readers will
  understand it from the reference to RFC 1242.

[Ballot comment]
- Sections 3 and 7.1: It might be worth clarifying that other
protocols than IKE can be used for key exchange (although benchmarking
them is beyond the scope of this document) in IPsec; in IETF, we have
specified at least Photuris, KINK, and HIP (and perhaps others I don't
recall right now).

- Sections 3, 3.1.2, 7.3: I think the text about historical origins
of IKE (hybrid of ISAKMP, Oakley, SKEME) is very unlikely to be
helpful to the readers (and probably mostly confusing).

[Ballot discuss]
I have reviewed draft-ietf-bmwg-ipsec-term-12, and have couple of
concerns about the terminology. For any other document, these
would be just editorial nits, but this is a terminology document,
after all:

- The text in Section 3.1.1 and 7.4 seems to apply only to IPsec
(ESP/AH) SAs, but not IKE SAs?

- Section 3.1.2: "public key cryptography" should be "public key
encryption" (that's the term used in RFC 2409, and digital
signatures are also a form of public key cryptography).

- Section 3.1.2: I would strongly suggest avoiding the term
"nonrepudiation" -- it's so overloaded that it's hard to know what it
means (if anything). This part of the text doesn't seem very relevant
to benchmarking anyway, so it probably could be just deleted.

- Section 7.3.1: This text seems to confuse the inputs to IKE Phase 1
(information configured before Phase 1 is run), the actual Phase 1 of
the protocol, and the output of that phase (Phase 1 SA or ISAKMP/IKE
SA).

- Section 7.3.1: "SPI information" probably should be "IPsec SA
information" or something like that.

- Section 7.3.2: This text seems to confuse the Phase 2 of the IKE
protocol, and its output (IPsec SA(s)).

- Section 7.6.1/7.6.2/7.6.3/7.6.4: This text seems to conflate the
"Phase 1 initiator" and "phase 2 initiator". While it's common to have
an "IPsec client" that only acts as "Phase 1 Initiator", the client
will also act as "Phase 2 Responder", since either end can initiate
phase 2 exchanges.

- Section 7.6.1/7.7.1/7.7.3/7.13/10.4.4/10.5.1/10.5.3/and several
other sections: There's no such thing as "IKE Phase 2 SA"; these are
not IKE SAs, but IPsec SAs.

- Section 7.7.1: I found this term really confusing, especially
considering the rest of the document seems to use it also in cases
where more than one IPsec SA pair exists. For example, Section 7.13
talks about "IKE Phase 1 SA to IKE Phase 2 SA ratio" for the IPsec
tunnels -- but according to this definition, the ratio can't be
anything else than 1 (otherwise it's not an "IPsec tunnel")!  And
Section 10.1.2 talks about establishing several Tunnels with a single
IKE SA, which isn't consistent with this definition.

- Section 7.8.2: The figure here is really confusing: you can't use
transport mode SAs this way...? (unless you do something like RFC
3884)

- Section 7.9.1: MD5-HMAC should be spelled "HMAC-MD5"; likewise,
"HMAC-SHA1" instead of "HMAC-SHA"; and there's no such thing as
AES-HMAC.

- Section 7.9: The text should also cover combined mode algorithms,
like CCM and GCM (or explicitly state they're not covered by
this document).

- Section 10.2.2 and 10.2.3: I'm not quite sure what "The end points
(i.e. hosts, subnets) should NOT see any fragments at ANY time.  Only
on the IPsec link, fragments MAY occur." means.  Is this text trying
to say that when you're doing benchmarking, the devices under test
should be configured to do post-encryption fragmentation?  Or that the
endpoints generating the plaintext benchmarking fraffic should be
configured to use small enough packets that fragmentation see
fragments?

- Section 10.3.1: the terms "initiator" and "responder" don't seem
applicable here; perhaps sender/receiver?

- Section 10.4.1: the definition seems to cover only packet losses
before encryption or after decryption, but not between these two
points. Is this intentional? (it seems measuring this would be
challenging, since it the endpoints can only see how many frames were
dropped, but not where exactly)

- Section 10.8.2: it seems this should be "IPsec integrity check DoS
resiliency rate" or something -- it's not about IKE Phase 2.

- Section 10.8.3: likewise, this is not about Phase 2, so perhaps
"IPsec replay attack DoS resiliency rate"?

[Ballot comment]
I think the RFC Editor will require you to remove citations from your
Abstract.

---

In the Abstract...
  This document seeks to extend               
  these efforts specific to the IPsec paradigm.
Is there any chance you could be a little more positive?

---

Abstract
  The BMWG produces two
  major classes of documents: Benchmarking Terminology documents and
  Benchmarking Methodology documents.  The Terminology documents
  present the benchmarks and other related terms.  The Methodology
  documents define the procedures required to collect the benchmarks
  cited in the corresponding Terminology documents.

Very interesting, and completely true. But not material to include in
this Abstract

---

Section 2

  A seperate document will be written specifically to address
  testing using the updated IKEv2 specification.

Pedantic, but...
The IKEv2 specification was not updated. Perhaps just strike "updated"?

---

Section 4

  The definition format utilized by this document is described in
  [RFC1242], Section 2.

Well...
Why do you then redefine the format in this section? That seems counter-
productive. You should probably cut this section down to only the
sentence I quoted.

[Ballot discuss]
I think this is a very useful document with a lot of material. The volume of work is reflected in the number of (relatively small) discussion points that I have.

Section 1

  To appropriately define the parameters and scope of this document,
  this section will give a brief overview of the IPsec standard.

Is there text missing? Should you be pointing forward to Section 3?

---

Section 2

  The terminology
  specified in this document is constrained to meet the requirements of
  the Methodology for Benchmarking IPsec Devices documented test
  methodologies.

This looks like the name of a document. Is a reference missing?

The text in the subsequent paragraphs appears to have been lifted from,
and to refer to, this other document.

---

Section 5

The use of RFC 2119 language seems eratic and maybe out of place.
You should decide whether you really want to use it, and if so, make its
use consistent.
At the very least you need to fix "WILL NOT" in section 7.7.1 as this
is not part of the RFC 2119 vocabulary.

---

Section 7.1

Since this section "defines" IPsec, it must include references to the
RFCs that define the protocol suite.

But I am not sure how this definition is intended to differ from that
in section 7.10.

---

Sections 8.1 and 8.2 include...


  See Also:  [...] Layer2 Clear Framesize, Layer2 Encrypted Framesize.

But these terms are not defined.

---

Sections 9.1 and 9.3

You should clarify the definition of "tunnel setup"?

One of the major issues in this type of work is comparing apples with
apples! This definition will be necessary to ensure that the metric
described in 10.5 has meaning.

Note also that in section 9.3 you have

  Issues:  If the TAPS increases, the TPS usually decreases, due to
      burdening of the DUT with the DOS attack traffic.

This might be true of a node under some form of load, and so applies to
the maxima in section 10.5, but it does not apply to the raw definitions
in this section. Not least to note is that TAPS includes all of the
successful setup attempts so an increase in TAPS may be a direct match
for an increase in TPS.

---

Section 10.2 offers symmetry for encryption and decryption at the DUT,
but does not seem to match this for send and receive. Thus, section
10.2.1 is ambiguous about "throughput" (as is RFC 1242) with respect to
a bidirectional flow.

    (1.a) Who is the Document Shepherd for this document? Has the
          Document Shepherd personally reviewed this version of the
          document and, in particular, does he or she believe this
          version is ready for forwarding to the IESG for publication?
Al Morton, chair of BMWG, has personally reviewed the documents and will
be the document shepherd. Editorial and nit-compliance changes requested
by Al were implemented in these versions. The documents are ready for
publication (several minor nits have been fixed in these versions).

    (1.b) Has the document had adequate review both from key WG members
          and from key non-WG members? Does the Document Shepherd have
          any concerns about the depth or breadth of the reviews that
          have been performed?
Yes, these documents have developed more or less continuously over the last
6 years, with good working group and external reviewer comments offered
and addressed.  The recent WGLC went quietly, indicating that the BMWG is
now satisfied with the documents (term-11 and meth-04).

    (1.c) Does the Document Shepherd have concerns that the document
          needs more review from a particular or broader perspective,
          e.g., security, operational complexity, someone familiar with
          AAA, internationalization or XML?
No, except that IESG review will be a good final check, of course.
As a side note, XML formatting problems held-up progress on these drafts
for a while, but there are no specifications involving XML.

    (1.d) Does the Document Shepherd have any specific concerns or
          issues with this document that the Responsible Area Director
          and/or the IESG should be aware of? For example, perhaps he
          or she is uncomfortable with certain parts of the document, or
          has concerns whether there really is a need for it. In any
          event, if the WG has discussed those issues and has indicated
          that it still wishes to advance the document, detail those
          concerns here. Has an IPR disclosure related to this document
          been filed? If so, please include a reference to the
          disclosure and summarize the WG discussion and conclusion on
          this issue.
One of the more controversial issues was whether these drafts should be
expanded to cover IKEv2.  The WG reached consensus to retain the IKEv1
scope, and a follow-up effort would address IKEv2.  This decision was
supported by events - it was reported at the IETF-74 BMWG session that
all known testing of implementations involves IKEv1 only.

    (1.e) How solid is the WG consensus behind this document? Does it
          represent the strong concurrence of a few individuals, with
          others being silent, or does the WG as a whole understand and
          agree with it?
It's fair to say that WG consensus involves key individuals rather than the
entire working group, but that's a byproduct of BMWG membership make-up
with diverse work areas. I believe that the entire WG has contributed to
discussion on aspects of these drafts, such as the exclusion of "IMIX"
traffic patterns (because of limited relevance to benchmarks, and lack
of consensus on the specifics - despite widespread use of the concept).

    (1.f) Has anyone threatened an appeal or otherwise indicated extreme
          discontent? If so, please summarise the areas of conflict in
          separate email messages to the Responsible Area Director. (It
          should be in a separate email because this questionnaire is
          entered into the ID Tracker.)
No.

    (1.g) Has the Document Shepherd personally verified that the
          document satisfies all ID nits? (See
          http://www.ietf.org/ID-Checklist.html and
          http://tools.ietf.org/tools/idnits/). Boilerplate checks are
          not enough; this check needs to be thorough. Has the document
          met all formal review criteria it needs to, such as the MIB
          Doctor, media type and URI type reviews?
Yes and Yes (but see below).

    (1.h) Has the document split its references into normative and
          informative? Are there normative references to documents that
          are not ready for advancement or are otherwise in an unclear
          state? If such normative references exist, what is the
          strategy for their completion? Are there normative references
          that are downward references, as described in [RFC3967]? If
          so, list these downward references to support the Area
          Director in the Last Call procedure for them [RFC3967].

The reference sections are split in both drafts, and these are Informational
Status drafts - so down-refs are not possible.

The terminology has normative references to several Obsolete RFCs:

  ** Obsolete normative reference: RFC 2393 (Obsoleted by RFC 3173)

  ** Obsolete normative reference: RFC 2401 (Obsoleted by RFC 4301)

  ** Obsolete normative reference: RFC 2402 (Obsoleted by RFC 4302, RFC 4305)

  ** Obsolete normative reference: RFC 2406 (Obsoleted by RFC 4303, RFC 4305)

  ** Obsolete normative reference: RFC 2407 (Obsoleted by RFC 4306)

  ** Obsolete normative reference: RFC 2408 (Obsoleted by RFC 4306)

  ** Obsolete normative reference: RFC 2409 (Obsoleted by RFC 4306)

  ** Obsolete normative reference: RFC 2547 (Obsoleted by RFC 4364)


The Methodology has a normative reference to:
    [I-D.ietf-ipsec-properties]
              Krywaniuk, A., "Security Properties of the IPsec Protocol
              Suite", draft-ietf-ipsec-properties-02 (work in progress),
              July 2002.
this needs to be dealt with, possibly by making it informative, or by deletion?

    (1.i) Has the Document Shepherd verified that the document IANA
          consideration section exists and is consistent with the body
          of the document? If the document specifies protocol
          extensions, are reservations requested in appropriate IANA
          registries? Are the IANA registries clearly identified? If
          the document creates a new registry, does it define the
          proposed initial contents of the registry and an allocation
          procedure for future registrations? Does it suggest a
          reasonable name for the new registry? See [RFC5226]. If the
          document describes an Expert Review process has Shepherd
          conferred with the Responsible Area Director so that the IESG
          can appoint the needed Expert during the IESG Evaluation?
Neither draft has an IANA section.  However, this should be simple to add,
since neither document makes a request of IANA (as is typical of BMWG memos,
the request for IPv6 address space last year is an exception).

    (1.j) Has the Document Shepherd verified that sections of the
          document that are written in a formal language, such as XML
          code, BNF rules, MIB definitions, etc., validate correctly in
          an automated checker?
N/A

    (1.k) The IESG approval announcement includes a Document
          Announcement Write-Up. Please provide such a Document
          Announcement Write-Up? Recent examples can be found in the
          "Action" announcements for approved documents. The approval
          announcement contains the following sections:

          Technical Summary
    The BMWG produces two major classes of documents:
    Benchmarking Terminology documents and Methodology documents.
    The Terminology documents present the benchmarks and other related terms.
    The Methodology documents define the procedures required to collect the
    benchmarks cited in the corresponding Terminology documents.

    This purpose of these documents is to define terms and methods for
    benchmarking the performance of IPsec devices.  It builds upon the
    tenets set forth in [RFC1242], [RFC2544], [RFC2285] and other IETF
    Benchmarking Methodology Working Group (BMWG) documents (used for
    benchmarking routers and switches).  This document seeks to extend
    these efforts specific to the IPsec paradigm.

          Working Group Summary
    Although there were some controversial points during development, all
    were resolved, and these drafts represent the consensus of the BMWG.

          Document Quality
    The Acknowledgements sections list many of the experts who reviewed
    thses drafts.