Benchmarking Methodology for In-Service Software Upgrade (ISSU)
draft-ietf-bmwg-issu-meth-02

Section 3:
OLD
Note that, a given vendor implementation may or may not permit the abortion of
   the in-progress ISSU at particular stages.

NEW
Note that, a given vendor implementation may or may not permit halting the in-progress ISSU at particular stages.

OLD
the test
   plan document should reflect these and other relevant details and
   SHOULD be written with close attention

NEW
the test
   plan document should reflect these and other relevant details and
   should be written with close attention

- Some discrepancies regarding MUST, SHOULD, MAY

One one side: The hardware configuration of the DUT SHOULD be identical to the one expected to be or currently deployed in production
One the other side: the feature, protocol timing and other relevant configurations should be matched to the expected production environment.

Note: potentially some other discrepancie with lower/upper case MUST/SHOULD/MAY

- expand "NSR"


nits:
- point missing in the abstract
- section 1 title is wrongly formatted

In section 3, when "other checks" are described, I'd like to see an explicit mention of a check to ensure the device is authorized to install the software/firmware.  There have been numerous attacks against just about every vendor where counterfeit hardware is deployed and runs the firmware, OS, etc. issued.  Fraud can be a big issue and it isn't always talked about, but this should definitely be a security consideration.  I do think it fits in the subsections of section 3.  The statement added can be as simple as, "check performed to ensure the device is authorized to install the firmware".

- I was a bit surprised there was no reference to software
signing here (only checksums are mentioned).  The
difference is that signature verification can also
sometimes require e.g. an OCSP lookup to check for
revocation and that could I guess impact on benchmarking.
(The secdir reviewer also made this point.)

- Kathleen makes a good point in her comment too.