Terminology for Benchmarking IPsec Devices
draft-ietf-bmwg-ipsec-term-12

Summary: Needs a YES.

(Pasi Eronen) Discuss

Discuss (2009-10-22)
I have reviewed draft-ietf-bmwg-ipsec-term-12, and have couple of
concerns about the terminology. For any other document, these
would be just editorial nits, but this is a terminology document,
after all:

- The text in Section 3.1.1 and 7.4 seems to apply only to IPsec
(ESP/AH) SAs, but not IKE SAs?

- Section 3.1.2: "public key cryptography" should be "public key
encryption" (that's the term used in RFC 2409, and digital
signatures are also a form of public key cryptography).

- Section 3.1.2: I would strongly suggest avoiding the term
"nonrepudiation" -- it's so overloaded that it's hard to know what it
means (if anything). This part of the text doesn't seem very relevant
to benchmarking anyway, so it probably could be just deleted.

- Section 7.3.1: This text seems to confuse the inputs to IKE Phase 1
(information configured before Phase 1 is run), the actual Phase 1 of
the protocol, and the output of that phase (Phase 1 SA or ISAKMP/IKE
SA).

- Section 7.3.1: "SPI information" probably should be "IPsec SA
information" or something like that.

- Section 7.3.2: This text seems to confuse the Phase 2 of the IKE
protocol, and its output (IPsec SA(s)).

- Section 7.6.1/7.6.2/7.6.3/7.6.4: This text seems to conflate the
"Phase 1 initiator" and "phase 2 initiator". While it's common to have
an "IPsec client" that only acts as "Phase 1 Initiator", the client
will also act as "Phase 2 Responder", since either end can initiate
phase 2 exchanges.

- Section 7.6.1/7.7.1/7.7.3/7.13/10.4.4/10.5.1/10.5.3/and several
other sections: There's no such thing as "IKE Phase 2 SA"; these are
not IKE SAs, but IPsec SAs.

- Section 7.7.1: I found this term really confusing, especially
considering the rest of the document seems to use it also in cases
where more than one IPsec SA pair exists. For example, Section 7.13
talks about "IKE Phase 1 SA to IKE Phase 2 SA ratio" for the IPsec
tunnels -- but according to this definition, the ratio can't be
anything else than 1 (otherwise it's not an "IPsec tunnel")!  And
Section 10.1.2 talks about establishing several Tunnels with a single
IKE SA, which isn't consistent with this definition.

- Section 7.8.2: The figure here is really confusing: you can't use
transport mode SAs this way...? (unless you do something like RFC
3884)

- Section 7.9.1: MD5-HMAC should be spelled "HMAC-MD5"; likewise,
"HMAC-SHA1" instead of "HMAC-SHA"; and there's no such thing as
AES-HMAC.

- Section 7.9: The text should also cover combined mode algorithms,
like CCM and GCM (or explicitly state they're not covered by
this document).

- Section 10.2.2 and 10.2.3: I'm not quite sure what "The end points
(i.e. hosts, subnets) should NOT see any fragments at ANY time.  Only
on the IPsec link, fragments MAY occur." means.  Is this text trying
to say that when you're doing benchmarking, the devices under test
should be configured to do post-encryption fragmentation?  Or that the
endpoints generating the plaintext benchmarking fraffic should be
configured to use small enough packets that fragmentation see
fragments?

- Section 10.3.1: the terms "initiator" and "responder" don't seem
applicable here; perhaps sender/receiver?

- Section 10.4.1: the definition seems to cover only packet losses
before encryption or after decryption, but not between these two
points. Is this intentional? (it seems measuring this would be
challenging, since it the endpoints can only see how many frames were
dropped, but not where exactly)

- Section 10.8.2: it seems this should be "IPsec integrity check DoS
resiliency rate" or something -- it's not about IKE Phase 2.

- Section 10.8.3: likewise, this is not about Phase 2, so perhaps
"IPsec replay attack DoS resiliency rate"?
Comment (2009-10-22)
No email
send info
- Sections 3 and 7.1: It might be worth clarifying that other
protocols than IKE can be used for key exchange (although benchmarking
them is beyond the scope of this document) in IPsec; in IETF, we have
specified at least Photuris, KINK, and HIP (and perhaps others I don't
recall right now).

- Sections 3, 3.1.2, 7.3: I think the text about historical origins 
of IKE (hybrid of ISAKMP, Oakley, SKEME) is very unlikely to be
helpful to the readers (and probably mostly confusing).

(Adrian Farrel) Discuss

Discuss (2009-10-21)
I think this is a very useful document with a lot of material. The volume of work is reflected in the number of (relatively small) discussion points that I have.

Section 1

   To appropriately define the parameters and scope of this document,
   this section will give a brief overview of the IPsec standard.

Is there text missing? Should you be pointing forward to Section 3?

---

Section 2

   The terminology
   specified in this document is constrained to meet the requirements of
   the Methodology for Benchmarking IPsec Devices documented test
   methodologies.

This looks like the name of a document. Is a reference missing?

The text in the subsequent paragraphs appears to have been lifted from,
and to refer to, this other document. 

---

Section 5

The use of RFC 2119 language seems eratic and maybe out of place.
You should decide whether you really want to use it, and if so, make its
use consistent.
At the very least you need to fix "WILL NOT" in section 7.7.1 as this
is not part of the RFC 2119 vocabulary.

---

Section 7.1

Since this section "defines" IPsec, it must include references to the 
RFCs that define the protocol suite.

But I am not sure how this definition is intended to differ from that 
in section 7.10.

---

Sections 8.1 and 8.2 include...


   See Also:  [...] Layer2 Clear Framesize, Layer2 Encrypted Framesize.

But these terms are not defined.

---

Sections 9.1 and 9.3

You should clarify the definition of "tunnel setup"?

One of the major issues in this type of work is comparing apples with
apples! This definition will be necessary to ensure that the metric
described in 10.5 has meaning.

Note also that in section 9.3 you have 

   Issues:  If the TAPS increases, the TPS usually decreases, due to
      burdening of the DUT with the DOS attack traffic.

This might be true of a node under some form of load, and so applies to
the maxima in section 10.5, but it does not apply to the raw definitions
in this section. Not least to note is that TAPS includes all of the
successful setup attempts so an increase in TAPS may be a direct match 
for an increase in TPS.

---

Section 10.2 offers symmetry for encryption and decryption at the DUT, 
but does not seem to match this for send and receive. Thus, section 
10.2.1 is ambiguous about "throughput" (as is RFC 1242) with respect to
a bidirectional flow.
Comment (2009-10-21)
No email
send info
I think the RFC Editor will require you to remove citations from your
Abstract.

---

In the Abstract...
   This document seeks to extend                
   these efforts specific to the IPsec paradigm.
Is there any chance you could be a little more positive?

---

Abstract
   The BMWG produces two
   major classes of documents: Benchmarking Terminology documents and
   Benchmarking Methodology documents.  The Terminology documents
   present the benchmarks and other related terms.  The Methodology
   documents define the procedures required to collect the benchmarks
   cited in the corresponding Terminology documents.

Very interesting, and completely true. But not material to include in
this Abstract

---

Section 2

   A seperate document will be written specifically to address
   testing using the updated IKEv2 specification.

Pedantic, but...
The IKEv2 specification was not updated. Perhaps just strike "updated"?

---

Section 4

   The definition format utilized by this document is described in
   [RFC1242], Section 2.

Well...
Why do you then redefine the format in this section? That seems counter-
productive. You should probably cut this section down to only the
sentence I quoted.

(Russ Housley) Discuss

Discuss (2009-11-16)
  The Gen-ART Review by Joel Halpern on 16-Oct-2009 raised two issues
  that need to be resolved.

  First, in section 3.1.1, where Security Association is introduced, the
  text reads:

    "An SA is a relationship between two or more entities ..."

  But, section 7.4 formally defines Security Association as:

    "It is a negotation agreement between two IPsec devices ..."

  Is it always two?  If so, please remove "or more" in section 3.1.1.
  Also, is it always a negotiation agreement?  Earlier text talked about
  manual provisioning of Security Associations.

  Second, in section 10.2, the throughput measurement is defined in terms
  of packets per second.  This is probably the dominant factor.  However,
  some implementations may well be limited by cryptographic processing,
  and therefore have different packet processing performance for different
  ranges of packet sizes.  Section 10.2 should say something about the
  assumptions or requirements relative to packet size.  This seems to be
  important enough that it should not be assumed that readers will
  understand it from the reference to RFC 1242.
Comment (2009-11-16)
No email
send info
  The Gen-ART Review by Joel Halpern on 16-Oct-2009 raised some editorial
  concerns.

  The document is missing the IANA considerations placeholder.

  There seem to be a number of references that are missing.  RFC1962
  is an example of the problem.

  The "Issue" in section 7.5 looks like a reasonable issue, but it
  appears to have nothing to do with selectors, the subject of 7.5.

  The definitions in section 7.7.3 and 7.7.4 define the terms
  "Established Tunnel" and "Active Tunnel" and defines them as "An
  IPsec device ...".  The other tunnel definitions are in terms of the
  association.  It is quite jarring to see a device referred to as a
  tunnel.  Are these terms supposed to be Tunnel Endpoints?

  The definition of 7.10.1 describes the properties of AH, but does not
  actually say what it is.  Instead of starting "Provides", it probably
  should start something like "A protocol header which provides".  A
  similar comment applies to 7.10.2 on ESP.

(Sean Turner) Discuss

Discuss (2010-03-31)
I am picking up Pasi's DISCUSS on this document.  Some were moved to COMMENT status, but the authors should also consult Pasi's COMMENT positions (if any).

I have reviewed draft-ietf-bmwg-ipsec-term-12, and have couple of
concerns about the terminology. For any other document, these
would be just editorial nits, but this is a terminology document,
after all:

- The text in Section 3.1.1 and 7.4 seems to apply only to IPsec
(ESP/AH) SAs, but not IKE SAs?

- Section 3.1.2: I would strongly suggest avoiding the term
"nonrepudiation" -- it's so overloaded that it's hard to know what it
means (if anything). This part of the text doesn't seem very relevant
to benchmarking anyway, so it probably could be just deleted.

- Section 7.3.1: This text seems to confuse the inputs to IKE Phase 1
(information configured before Phase 1 is run), the actual Phase 1 of
the protocol, and the output of that phase (Phase 1 SA or ISAKMP/IKE
SA).

- Section 7.3.1: "SPI information" probably should be "IPsec SA
information" or something like that.

- Section 7.3.2: This text seems to confuse the Phase 2 of the IKE
protocol, and its output (IPsec SA(s)).

- Section 7.6.1/7.6.2/7.6.3/7.6.4: This text seems to conflate the
"Phase 1 initiator" and "phase 2 initiator". While it's common to have
an "IPsec client" that only acts as "Phase 1 Initiator", the client
will also act as "Phase 2 Responder", since either end can initiate
phase 2 exchanges.

- Section 7.6.1/7.7.1/7.7.3/7.13/10.4.4/10.5.1/10.5.3/and several
other sections: There's no such thing as "IKE Phase 2 SA"; these are
not IKE SAs, but IPsec SAs.

- Section 7.7.1: I found this term really confusing, especially
considering the rest of the document seems to use it also in cases
where more than one IPsec SA pair exists. For example, Section 7.13
talks about "IKE Phase 1 SA to IKE Phase 2 SA ratio" for the IPsec
tunnels -- but according to this definition, the ratio can't be
anything else than 1 (otherwise it's not an "IPsec tunnel")!  And
Section 10.1.2 talks about establishing several Tunnels with a single
IKE SA, which isn't consistent with this definition.

- Section 7.8.2: The figure here is really confusing: you can't use
transport mode SAs this way...? (unless you do something like RFC
3884)

- Section 7.9.1: MD5-HMAC should be spelled "HMAC-MD5"; likewise,
"HMAC-SHA1" instead of "HMAC-SHA"; and there's no such thing as
AES-HMAC.

- Section 7.9: The text should also cover combined mode algorithms,
like CCM and GCM (or explicitly state they're not covered by
this document).

- Section 10.2.2 and 10.2.3: I'm not quite sure what "The end points
(i.e. hosts, subnets) should NOT see any fragments at ANY time.  Only
on the IPsec link, fragments MAY occur." means.  Is this text trying
to say that when you're doing benchmarking, the devices under test
should be configured to do post-encryption fragmentation?  Or that the
endpoints generating the plaintext benchmarking fraffic should be
configured to use small enough packets that fragmentation see
fragments?

- Section 10.3.1: the terms "initiator" and "responder" don't seem
applicable here; perhaps sender/receiver?

- Section 10.4.1: the definition seems to cover only packet losses
before encryption or after decryption, but not between these two
points. Is this intentional? (it seems measuring this would be
challenging, since it the endpoints can only see how many frames were
dropped, but not where exactly)

- Section 10.8.2: it seems this should be "IPsec integrity check DoS
resiliency rate" or something -- it's not about IKE Phase 2.

- Section 10.8.3: likewise, this is not about Phase 2, so perhaps
"IPsec replay attack DoS resiliency rate"?
Comment (2010-03-31)
No email
send info
- Section 3.1.2: "public key cryptography" should be "public key
encryption" (that's the term used in RFC 2409, and digital
signatures are also a form of public key cryptography).

(Ron Bonica) Yes

(Ross Callon) No Objection

(Ralph Droms) No Objection

(Dan Romascanu) No Objection

Magnus Westerlund No Objection

Deborah Brungard No Record

Alissa Cooper No Record

Roman Danyliw No Record

Martin Duke No Record

(Cullen Jennings) (was Abstain) No Record

Comment (2009-11-18)
No email
send info
see comments on draft-ietf-bmwg-ipsec-meth

Benjamin Kaduk No Record

Erik Kline No Record

Murray Kucherawy No Record

Warren Kumari No Record

Barry Leiba No Record

Alvaro Retana No Record

Martin Vigoureux No Record

Éric Vyncke No Record

Robert Wilton No Record