Clustered Alternate-Marking Method
draft-ietf-ippm-rfc8889bis-04

[Ballot comment]
Thanks for the work on this.  It was pretty easy to follow.

In Section 5, the document suddenly starts using the term "arc".  What's an arc?

Two problems in Section 9.  The first:

      One flag: packet loss measurement MUST be done as described in
      Section 6 by applying the network clustering partition described
      in Section 5.  While delay measurement MUST be done according to
      the Mean delay calculation representative of the multipoint path,
      as described in Section 7.1.1.  Single-marking method based on the
      first/last packet of the interval cannot be applied, as mentioned
      in Section 7.2.1.

The "While delay ..." sentence seems to be incomplete.  Should it be attached (via comma) to the sentence before it, or is there something missing here?

The same problem occurs in the next ("Two flags:") paragraph.

The second problem is the SHOULD in that same paragraph.  SHOULD presents the implementer with a choice, and I suggest adding some prose here to explain why one might legitimately decide to do something other than what the SHOULD says.

Lastly, some nits:

In Section 5:

  In addition, it is also possible to leverage [...]

You don't need both "In addition" and "also".

In Section 7.2.1:

      Double marking or multiplexed marking work but only through
      statistical means.  [...]

s/work/works/

  If it is performed a delay measurement for more than one [...]

Suggest "If a delay measurement is performed for more than one ...".

In Section 9:

  ... there is different kind of information that can be derived.

Missing "a", I believe?

      For example, to get measurements on a multipoint-paths
      basis, one flag can be used.  While, to get measurements on a
      single-packet basis, two flags are preferred.

Suggest removing "While".

[Ballot comment]
Thanks for the work done in this document.

Please find below some non-blocking COMMENT points; a reply will be appreciated.

Regards

-éric

## COMMENTS

### I-D Name

The name of the I-D is quite confusing "multipoint-to-multipoint"... I would have really preferred to use "cluster" in the title and abstract.

### Cluster

About "cluster", I second John's original comment about the "fuzziness" of the cluster definition.

### Section 5

"multipoint flows" is also weird as usually a flow is between 2 entities. Suggest to use "flow aggregate"

### Cluster and NAT

A cluster is only defined as packets in == packets out. But, if 'flow' is defined per IP addresses or 5-tuple, those properties must also be invariant, i.e., neither IPv4 NAPT nor IPv6 NPT nor encapsulation/decapsulation. This seems obvious, but this should be mentioned.

### Section 9

The title is about 'recommendations', but the text contains 'MUST' and not 'IS RECOMMENDED'. Suggest to change either the section title or the text itself.

[Ballot comment]
Thanks for addressing my DISCUSS, I've cleared it and retained the text below for posterity.

Previous DISCUSS:

Thanks for this document. As you may have noticed I had considerable difficulty with the definition of "cluster". Once I completed an end-to-end read-through, this was resolved (good!) but because RFCs are often consumed piecemeal (e.g. someone may just dip into a portion of a document rather than settling down with a nice cup of tea to read it end-to-end), I think it's important to fix this problem, on the assumption I'm not the only person who might be thrown off.

I'll leave the details in the COMMENT, but I will repeat one observation from my comment #3, which is that I count at least four separate (re-)definitions of "cluster" in the document. With so many, it's no wonder that they're inconsistent, and quite possibly the simplest solution would involve cutting the number of definitions down to as close to 1 as possible.

COMMENT:

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."

```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

7. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

8. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot comment]
Thanks for working on this specification.

I am supporting Roman, John, Alvaro's discuss.

I have following comments which I believe would improve the document quality, if addressed.

## Section 2: I really didn't get the definition of the flow. May be it is my lack of expertise in English language or the technology used here but when the definition of flow is defined by flow ( is says- a flow can be multipoint-to-multipoint flow) it becomes very difficult to understand. Are we defining flow or identification of flow? It would be great to get a better description of the definition of  "flow". I also agree with John's comment that if a flow does not have traffic, is this also a flow? May be we merely defining the link between network nodes as flow, then I would say the term flow is overloaded.

Actually section 3 has a much better definition of flow. why don't we use that?

      "a flow can be defined by a set of selection rules used to
  match a subset of the packets processed by the network device.  These
  rules specify a set of Layer 3 and Layer 4 header fields
  (identification fields) and the relative values that must be found in
  matching packets"

## Section 2: according to the definition of the cluster, any single node can be a cluster and measure of the that cluster will be same as point to point measure defined in rfc8321bis (like node packet loss). is that correct interpretation? if yes, rfc8321bis becomes a subset of this specification, is that the intention?

## Section 2.1 : as group to group segments are not used to define cluster in this specification, I would suggest following change -

      OLD text: the spatial metrics, used for measuring the performance of
      segments of a source to destination path, are applied here to
      group-to-group segments (called clusters).

      NEW text : the spatial metrics, used for measuring the performance of
      segments of a source to destination path, are applied here to
      clusters.

## Section 4.2 : What is "Non-initial fragments"?

## I think it is confusing when terms defined in the terminology section get redefined in the later sections in a different way than how it is defined in the terminology section. the term "cluster" is an example of such case. can we stick to one single definition of cluster and other terms used in this document?

[Ballot discuss]
# GEN AD review of draft-ietf-ippm-rfc8889bis-02

CC @larseggert

Thanks to Russ Housley for the General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/OTAnYgNGwDIRaQu_9IDo1QN1KHM).

## Discuss

### Section 1, paragraph 11
```
    Note that the fragmented packets case can be managed with the
    Alternate-Marking methodology.  The same considerations of
    [I-D.ietf-ippm-rfc8321bis] apply also in the case of Multipoint
    Alternate Marking.  As defined in [I-D.ietf-ippm-rfc8321bis] the
    marking node MUST mark all the fragments except in the case of
    fragmentation within the network domain, in that event it is
    suggested to mark only the first fragment.
```
"MUST mark ... except" is not clear enough. In the case where there is
fragmentation, what is the defined behavior? "Suggest to mark" leaves a lot
open to interpretation.

In general, the use of RFC2119 language in this document should be checked. See comments below.

[Ballot comment]
## Comments

### Section 3, paragraph 15
```
    The case of unicast flow is considered in Figure 1.  The anycast flow
    is also in scope, because there is no replication and only a single
    node from the anycast group receives the traffic, so it can be viewed
    as a special case of unicast flow.
```
Anycast is only a special case of a unicast flow is routing is stable throughout
the measurement period.

### Section 4.2, paragraph 0
```
  4.2.  Network Packet Loss
```
The definitions in this section seem to assume that routing is stable during the
measurement period (because otherwise the loss definitions don't work in case of
route changes or loops). That assumption should probably be made more explicit.

### Section 5, paragraph 5
```
    In a completely monitored unidirectional network (a network where
    every network interface is monitored), each network device
    corresponds to a cluster, and each physical link corresponds to two
    clusters (one for each device).
```
What is a "unidirectional network"?

### Section 5.1, paragraph 0
```
  5.1.  Algorithm for Clusters Partition
```
It would be helpful if this algorithm was also specified as pseudo code, rather
than just by example.

### Section 7.1.1, paragraph 2
```
    The average latency can be measured as the difference between the
    weighted averages of the mean timestamps of the sets of output and
    input nodes.  This means that, in the calculation, it is possible to
    weigh the timestamps by considering the number of packets for each
    endpoints.
```
Wouldn't you need to require that the weight is exactly the number of packets to
make the math work, and not just a weight that "considers" the number of
packets, i.e., is an unspecified function of it??

### Section 7.2.2, paragraph 2
```
    The hash-based selection methodologies for delay measurement can work
    in a multipoint-to-multipoint path and MAY be used either coupled to
    mean delay or stand-alone.
```
I think the MAY needs to be a MUST, because otherwise unspecified alternatives
are permitted?

### Section 7.2.2, paragraph 4
```
    In a multipoint environment, the hashing selection MAY be the
    solution for performing delay measurements on specific packets and
    overcoming the single- and double-marking limitations.
```
I don't understand why this uses an RFC2119 MAY?

### Section 9, paragraph 2
```
    Either one or two flag bits might be available for marking in
    different deployments:
```
The use of SHOULD and MAY in the three sections following this is leaving things
underspecified. What happens if SHOULDs and MAYs are not followed?

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Boilerplate

Document still refers to the "Simplified BSD License", which was corrected in
the TLP on September 21, 2021. It should instead refer to the "Revised BSD
License".

### Grammar/style

#### Section 1, paragraph 6
```
of problems (packet loss is measured or the delay is too high), the filterin
                                    ^^^
```
Use a comma before "or" if it connects two independent clauses (unless they are
closely connected and short).

#### Section 1, paragraph 8
```
DISCUSS: Note that the fragmented packets case can be managed with the Alter
                                  ^^^^^^^
```
An apostrophe may be missing.

#### Section 4.2, paragraph 3
```
algorithm was also specified as pseudo code, rather than just by example. A
                                ^^^^^^^^^^^
```
This word is normally spelled as one.

#### Section 5.1, paragraph 13
```
defined in Section 4.2, valid for the an entire monitored flow, can easily
                                  ^^^^^^
```
Two determiners in a row. Choose either "the" or "an".

#### Section 5.1, paragraph 25
```
ng the number of packets for each endpoints. Wouldn't you need to require th
                                  ^^^^^^^^^
```
The noun should probably be in the singular form.

#### Section 5.1, paragraph 32
```
can do the calculation. If we would perform a delay measurement for more th
                              ^^^^^^^^^^^^^
```
Consider removing "would". (Usually, "would" does not occur in a conditional
clause, unless to make a request or give a polite order.).

Also, the document uses first person plural ("we") in several places, which is
unusual in an RFC.

#### Section 7, paragraph 1
```
with the hashing selection allows to choose a simplified hash function since
                                  ^^^^^^^^^
```
Did you mean "choosing"? Or maybe you should add a pronoun? In active voice,
"allow" + "to" takes an object, usually a pronoun.

#### Section 7, paragraph 3
```
along the path. For example, in case an hashed sample is lost, it is confined
                                    ^^
```
Use "a" instead of "an" if the following word doesn't start with a vowel sound,
e.g. "a sentence", "a university".

#### Section 8, paragraph 4
```
of necessity (packet loss is measured or the delay is too high), the filterin
                                    ^^^
```
Use a comma before "or" if it connects two independent clauses (unless they are
closely connected and short).

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."

```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

7. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

8. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."

```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

6. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

7. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."
```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

6. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

7. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."

```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

6. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

7. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have multiple definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."
```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

6. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

7. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot discuss]
Thanks for this document. As you may have noticed I had considerable difficulty with the definition of "cluster". Once I completed an end-to-end read-through, this was resolved (good!) but because RFCs are often consumed piecemeal (e.g. someone may just dip into a portion of a document rather than settling down with a nice cup of tea to read it end-to-end), I think it's important to fix this problem, on the assumption I'm not the only person who might be thrown off.

I'll leave the details in the COMMENT, but I will repeat one observation from my comment #3, which is that I count at least four separate (re-)definitions of "cluster" in the document. With so many, it's no wonder that they're inconsistent, and quite possibly the simplest solution would involve cutting the number of definitions down to as close to 1 as possible.

[Ballot discuss]
Thanks for this document. As you may have noticed I had considerable difficulty with the definition of "cluster". Once I completed an end-to-end read-through, this was resolved (good!) but because RFCs are often consumed piecemeal (e.g. someone may just dip into a portion of a document rather than settling down with a nice cup of tea to read it end-to-end), I think it's important to fix this problem, on the assumption I'm not the only person who might be thrown off.

I'll leave the details in the COMMENT, but I will repeat one observation from my comment #3, which is that I count at least four separate (re-)definitions of "cluster" in the document. With so many, it's no wonder that they're inconsistent, and quite likely the cleanest solution would involve cutting the number of definitions down to as close to 1 as possible.

[Ballot comment]
I support Roman's DISCUSS.

1. In §1, this

                                  While this document and its Clustered
  Alternate-Marking method is valid for multipoint-to-multipoint
  unicast flows, anycast, and ECMP flows.
 
This is a sentence fragment (a naked subordinate clause). I'd suggest a rewrite but I can't make out what you are trying to say.

2. You have two different definitions of "cluster" in the document, it seems. In §2 you have,

      Cluster: Smallest identifiable subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm can be applied to split the monitoring
      network into clusters.

As we have discussed in the past, according to this definition, a cluster is always a single router. I think in an earlier discussion you suggested that it could be corrected by saying something like "smallest subnetwork larger than a singleton router", which I think would work, "smallest non-trivial" would too (although it's less explicit). But then in §5 you have,

  In addition, it is also possible to leverage the data provided by the
  other counters in the network to converge on the smallest
  identifiable subnetworks where the losses occur.  These subnetworks
  are named "clusters".

As written, this makes nonzero losses a definitional attribute of a cluster. Now that I've taken in the entire document, I guess this is the incorrect definition. Possibly a fix would be to just drop the final sentence, which (incorrectly?) implies that losses are an essential element of the definition.

For what it's worth, the algorithm of Section 5.1 is perfectly clear. I see the desire to have a prose description of what a cluster is trying to do, but I wonder if in the end it would be best to make the algorithm the canonical definition, referenced from the prose definition as in

      Cluster: Smallest identifiable non-trivial
      subnetwork of the entire monitoring
      network graph that still satisfies the condition that the number
      of packets that go in is the same as the number that go out.  A
      cluster partition algorithm, such as that found in Section 5.1,
      can be applied to split the monitoring
      network into clusters.

3. In §5, you have

  A cluster graph is a subnetwork of the entire monitoring network
  graph that still satisfies the packet loss equation (introduced in
  the previous section), where PL in this case is the number of packets
  lost in the cluster.

I'd previously pointed out that this is problematic since the PL equation you mention is in the nature of a definition; since you don't supply a condition there's no way of saying whether it's "satisfied" or not. In response you said (https://mailarchive.ietf.org/arch/msg/ippm/iXSBuXrQaETl6MyeDH8DBaPiJ9Q/),

```
We can modify the wording in Section 5 accordingly:

"A cluster graph is a subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out, if no packet loss occurs."
```

If indeed that is the right definition (see my earlier point, so probably inserting "non-trivial" or "with more than a single router") then I do think you should make that change, or in any case you should correct the current text somehow.

By the way, do you mean something subtly different by "cluster graph" than you do by "cluster"? Or do you just mean "the graph that represents the cluster"? I think my confusion over just what a "cluster" is would have been mitigated by fewer re-definitions of it. If "cluster graph" is indeed just another way of saying "cluster" it's the third of four definitions! (The two I mention in my point #2, this one, and the algorithm in Section 5.1.) With four definitions of the same thing, in different words, no wonder some inconsistency crept in!

4. In §5.1,

  In summary, once a flow is defined, the algorithm to build the
  clusters partition is based on topological information; therefore, it
  considers all the possible links and nodes crossed by the given flow,
  even if there is no traffic.
 
If a flow has no traffic, can we call it a flow? It seems counter to the normal English meaning of the word. Perhaps you mean possible links and nodes that could potentially be crossed by the given flow?

5. Thanks for reporting the outcome of the earlier experiment, in §9. As with my review of rfc8321bis, I think the RFC 2119 keywords need to be in a "Deployment Considerations" section or similar, which could be achieved by retitling this section or by separating the material into reporting the experiment outcome (this section) and a new subsection for deployment advice.

6. Again similarly to my review of the companion document, I think §9's "the Multipoint Alternate Marking Method is RECOMMENDED only for controlled domains" needs to be fixed. Please bring it into line with whatever language you choose for rfc8321bis.




Nits:

6. While the reference tags are in principle arbitrary strings, I wondered if it was a typo that you used "PNPM" to tag a paper titled "AM-PM"?

7. In §7.1.1,

                This means that, in the calculation, it is possible to
  weigh the timestamps by considering the number of packets for each
  endpoints.
 
s/endpoints/endpoint/

[Ballot discuss]
§9 ("Results of the Multipoint Alternate Marking Experiment") makes several
recommendations about the use of one or two flag bits:

    One flag: packet loss measurement SHOULD be done as described in
      Section 6 by applying the network clustering partition described
      in Section 5.  While delay measurement MAY be done according to
      the Mean delay calculation representative of the multipoint path,
      as described in Section 7.1.1.  Single-marking method based on the
      first/last packet of the interval cannot be applied, as mentioned
      in Section 7.2.1.

      Two flags: packet loss measurement SHOULD be done as described in
      Section 6 by applying the network clustering partition described
      in Section 5.  While delay measurement SHOULD be done on a single
      packet basis according to double-marking method Section 7.2.1.  In
      this case the Mean delay calculation (Section 7.1.1) MAY also be
      used as a representative value of a multipoint path.

      One flag and hash-based selection: packet loss measurement SHOULD
      be done as described in Section 6 by applying the network
      clustering partition described in Section 5.  Hash-based selection
      methodologies, introduced in Section 7.2.2, MAY be used for delay
      measurement.

These recommendations are good, as they are the result of experimentation. 
However, they don't provide any deployment or operational guidelines of when
it is ok to follow them and when it isn't.  For example, for the one flag case,
when it is ok to not measure packet loss as described in §6?  Why is the use
of that mechanism only recommended and not required?

I have the same questions for all the recommendations and optional indications
in the text above.  To clear this DISCUSS I expect deployment or operational
recommendations that can be used as implementation/deployment guidance.

[Ballot comment]
I support Roman's DISCUSS position and have a related question about this text in §9:

  The Multipoint Alternate Marking Method is RECOMMENDED only for
  controlled domains, as per [I-D.ietf-ippm-rfc8321bis].

When is it ok to use the Multipoint Alternate Marking Method in any other deployment?  IOW, given the definition of a controlled domain in rfc8321bis, why is its use only recommended and not required?

Note that if this consideration depends entirely on rfc8321bis, you may be able to refer to it and eliminate the Normative language.

[I did not include this point as a DISCUSS because I expect it to be solved when Roman's concern is addressed.]

[Ballot discuss]
Please clarify the expected deployment model of this approach.
(a) Section 9.
  The Multipoint Alternate Marking Method is RECOMMENDED only for
  controlled domains, as per [I-D.ietf-ippm-rfc8321bis].

(b) Section 10
  This document specifies a method of performing measurements that does
  not directly affect Internet security or applications that run on the
  Internet.

The text in (a) suggests that deployment can occur on the Internet (although it isn’t recommended).  However, (b) suggests that OAM meta-data would not be used on the Internet.

[Ballot comment]
** Section 7.2.2.  Is [I-D.mizrahi-ippm-marking] the expected mechanism to combine hashing with Alternative Marking?  If so, it should be a normative reference.

** Section 9.  Is there a citation that can be provided for the experiments that informed this design?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-ippm-rfc8889bis-02, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

Michelle Thangtamsatid
IANA Services Specialist

The following Last Call announcement was sent out (ends 2022-06-21):

From: The IESG
To: IETF-Announce
CC: draft-ietf-ippm-rfc8889bis@ietf.org, ippm-chairs@ietf.org, ippm@ietf.org, martin.h.duke@gmail.com, tpauly@apple.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Multipoint Alternate-Marking Clustered Method) to Proposed Standard


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Multipoint Alternate-Marking Clustered
Method'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2022-06-21. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document generalizes and expands Alternate-Marking methodology
  to measure any kind of unicast flow whose packets can follow several
  different paths in the network -- in wider terms, a multipoint-to-
  multipoint network.  For this reason, the technique here described is
  called "Multipoint Alternate Marking".  This document obsoletes
  RFC8889.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-rfc8889bis/


The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/5650/

# Document Shepherd Writeup

*This version is dated 8 April 2022.*

Thank you for your service as a document shepherd. Among the responsibilities is
answering the questions in this writeup to give helpful context to Last Call and
Internet Engineering Steering Group ([IESG][1]) reviewers, and your diligence in
completing it, is appreciated. The full role of the shepherd is further
described in [RFC 4858][2], and informally. You will need the cooperation of
authors to complete these checks.

Note that some numbered items contain multiple related questions; please be sure
to answer all of them.

## Document History

1. Does the working group (WG) consensus represent the strong concurrence of a
  few individuals, with others being silent, or did it reach broad agreement?

This is small bis update we took through the WG quickly. It had clear consensus, with 
about 1/3-1/2 of the active WG chiming in. Both RFC8321bis and RFC8889bis are
being taken through the process together.

2. Was there controversy about particular points, or were there decisions where
  the consensus was particularly rough?

There was no controversy in these updates within the WG.

3. Has anyone threatened an appeal or otherwise indicated extreme discontent? If
  so, please summarize the areas of conflict in separate email messages to the
  responsible Area Director. (It should be in a separate email because this
  questionnaire is publicly available.)

No appeals threatened.

4. For protocol documents, are there existing implementations of the contents of
  the document? Have a significant number of potential implementers indicated
  plans to implement? Are any existing implementations reported somewhere,
  either in the document itself (as [RFC 7942][3] recommends) or elsewhere
  (where)?

Our adoption call of the -bis document raised the question of implementations,
and several implementations were discussed on the mailing list.

### Additional Reviews

5. Does this document need review from other IETF working groups or external
  organizations? Have those reviews occurred?

This -bis effort was spurred on by draft-ietf-6man-ipv6-alt-mark, which normatively
references the work. Authors are participants of both WGs, and that WG is aware of this work.

6. Describe how the document meets any required formal expert review criteria,
  such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

No formal review needed.

7. If the document contains a YANG module, has the final version of the module
  been checked with any of the [recommended validation tools][4] for syntax and
  formatting validation? If there are any resulting errors or warnings, what is
  the justification for not fixing them at this time? Does the YANG module
  comply with the Network Management Datastore Architecture (NMDA) as specified
  in [RFC 8342][5]?

No YANG model.

8. Describe reviews and automated checks performed to validate sections of the
  final version of the document written in a formal language, such as XML code,
  BNF rules, MIB definitions, CBOR's CDDL, etc.

Not applicable.

### Document Shepherd Checks

9. Based on the shepherd's review of the document, is it their opinion that this
  document is needed, clearly written, complete, correctly designed, and ready
  to be handed off to the responsible Area Director?

Yes, I think these documents are ready to progress.

10. Several IETF Areas have assembled [lists of common issues that their
    reviewers encounter][6]. Do any such issues remain that would merit specific
    attention from subsequent reviews?

I do not believe this document requires further area reviews.

11. What type of RFC publication is being requested on the IETF stream (Best
    Current Practice, Proposed Standard, Internet Standard, Informational,
    Experimental, or Historic)? Why is this the proper type of RFC? Do all
    Datatracker state attributes correctly reflect this intent?

Proposed Standard. The point of this bis effort is to move from Experimental
to Proposed Standard.

12. Has the interested community confirmed that any and all appropriate IPR
    disclosures required by [BCP 78][7] and [BCP 79][8] have been filed? If not,
    explain why. If yes, summarize any discussion and conclusion regarding the
    intellectual property rights (IPR) disclosures, including links to relevant
    emails.

Yes, IPR is filed for both RFC8321bis and RFC8889bis, which is just a carry-over
of the existing IPR on the original RFC8321 and RFC8889 documents.

13. Has each Author or Contributor confirmed their willingness to be listed as
    such? If the number of Authors/Editors on the front page is greater than 5,
    please provide a justification.

Both documents have 5 authors each, and all authors were involved with the
original RFCs.

14. Identify any remaining I-D nits in this document. (See [the idnits tool][9]
    and the checkbox items found in Guidelines to Authors of Internet-Drafts).
    Simply running the idnits tool is not enough; please review the entire
    guidelines document.

Nits for RFC8889bis:

  ** The abstract seems to contain references ([RFC8889]), which it
    shouldn't.  Please replace those with straight textual mentions of the
    documents in question.

  ** Downref: Normative reference to an Informational RFC: RFC 5474

  == Outdated reference: draft-ietf-ippm-route has been published as RFC 9198

These items should be addressed by making the reference to RFC8889 in the
abstract textual as suggesting, making the reference to RFC5474 informative,
and referring to RFC9198.

15. Should any informative references be normative or vice-versa?

For RFC8889bis: the reference to RFC5474 should become informative.

16. List any normative references that are not freely available to anyone. Did
    the community have sufficient access to review any such normative
    references?

All references are available.

17. Are there any normative downward references (see [RFC 3967][10],
    [BCP 97][11])? If so, list them.

For RFC8889bis: Currently RFC5474 is such, but I believe that should change.

18. Are there normative references to documents that are not ready for
    advancement or are otherwise in an unclear state? If they exist, what is the
    plan for their completion?

No.

19. Will publication of this document change the status of any existing RFCs? If
    so, does the Datatracker metadata correctly reflect this and are those RFCs
    listed on the title page, in the abstract, and discussed in the
    introduction? If not, explain why and point to the part of the document
    where the relationship of this document to these other RFCs is discussed.

Yes, RFC8889bis obsoletes RFC8889, as described in the abstract.

20. Describe the document shepherd's review of the IANA considerations section,
    especially with regard to its consistency with the body of the document.
    Confirm that all aspects of the document requiring IANA assignments are
    associated with the appropriate reservations in IANA registries. Confirm
    that any referenced IANA registries have been clearly identified. Confirm
    that each newly created IANA registry specifies its initial contents,
    allocations procedures, and a reasonable name (see [RFC 8126][12]).

This document does not have IANA actions.

21. List any new IANA registries that require Designated Expert Review for
    future allocations. Are the instructions to the Designated Expert clear?
    Please include suggestions of designated experts, if appropriate.

This document does not have IANA actions.

[1]: https://www.ietf.org/about/groups/iesg/
[2]: https://www.rfc-editor.org/rfc/rfc4858.html
[3]: https://www.rfc-editor.org/rfc/rfc7942.html
[4]: https://trac.ietf.org/trac/ops/wiki/yang-review-tools
[5]: https://www.rfc-editor.org/rfc/rfc8342.html
[6]: https://trac.ietf.org/trac/iesg/wiki/ExpertTopics
[7]: https://www.rfc-editor.org/info/bcp78
[8]: https://www.rfc-editor.org/info/bcp79
[9]: https://www.ietf.org/tools/idnits/
[10]: https://www.rfc-editor.org/rfc/rfc3967.html
[11]: https://www.rfc-editor.org/info/bcp97
[12]: https://www.rfc-editor.org/rfc/rfc8126.html

# Document Shepherd Writeup

*This version is dated 8 April 2022.*

Thank you for your service as a document shepherd. Among the responsibilities is
answering the questions in this writeup to give helpful context to Last Call and
Internet Engineering Steering Group ([IESG][1]) reviewers, and your diligence in
completing it, is appreciated. The full role of the shepherd is further
described in [RFC 4858][2], and informally. You will need the cooperation of
authors to complete these checks.

Note that some numbered items contain multiple related questions; please be sure
to answer all of them.

## Document History

1. Does the working group (WG) consensus represent the strong concurrence of a
  few individuals, with others being silent, or did it reach broad agreement?

This is small bis update we took through the WG quickly. It had clear consensus, with 
about 1/3-1/2 of the active WG chiming in. Both RFC8321bis and RFC8889bis are
being taken through the process together.

2. Was there controversy about particular points, or were there decisions where
  the consensus was particularly rough?

There was no controversy in these updates within the WG.

3. Has anyone threatened an appeal or otherwise indicated extreme discontent? If
  so, please summarize the areas of conflict in separate email messages to the
  responsible Area Director. (It should be in a separate email because this
  questionnaire is publicly available.)

No appeals threatened.

4. For protocol documents, are there existing implementations of the contents of
  the document? Have a significant number of potential implementers indicated
  plans to implement? Are any existing implementations reported somewhere,
  either in the document itself (as [RFC 7942][3] recommends) or elsewhere
  (where)?

Our adoption call of the -bis document raised the question of implementations,
and several implementations were discussed on the mailing list.

### Additional Reviews

5. Does this document need review from other IETF working groups or external
  organizations? Have those reviews occurred?

This -bis effort was spurred on by draft-ietf-6man-ipv6-alt-mark, which normatively
references the work. Authors are participants of both WGs, and that WG is aware of this work.

6. Describe how the document meets any required formal expert review criteria,
  such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

No formal review needed.

7. If the document contains a YANG module, has the final version of the module
  been checked with any of the [recommended validation tools][4] for syntax and
  formatting validation? If there are any resulting errors or warnings, what is
  the justification for not fixing them at this time? Does the YANG module
  comply with the Network Management Datastore Architecture (NMDA) as specified
  in [RFC 8342][5]?

No YANG model.

8. Describe reviews and automated checks performed to validate sections of the
  final version of the document written in a formal language, such as XML code,
  BNF rules, MIB definitions, CBOR's CDDL, etc.

Not applicable.

### Document Shepherd Checks

9. Based on the shepherd's review of the document, is it their opinion that this
  document is needed, clearly written, complete, correctly designed, and ready
  to be handed off to the responsible Area Director?

Yes, I think these documents are ready to progress.

10. Several IETF Areas have assembled [lists of common issues that their
    reviewers encounter][6]. Do any such issues remain that would merit specific
    attention from subsequent reviews?

I do not believe this document requires further area reviews.

11. What type of RFC publication is being requested on the IETF stream (Best
    Current Practice, Proposed Standard, Internet Standard, Informational,
    Experimental, or Historic)? Why is this the proper type of RFC? Do all
    Datatracker state attributes correctly reflect this intent?

Proposed Standard. The point of this bis effort is to move from Experimental
to Proposed Standard.

12. Has the interested community confirmed that any and all appropriate IPR
    disclosures required by [BCP 78][7] and [BCP 79][8] have been filed? If not,
    explain why. If yes, summarize any discussion and conclusion regarding the
    intellectual property rights (IPR) disclosures, including links to relevant
    emails.

Yes, IPR is filed for both RFC8321bis and RFC8889bis, which is just a carry-over
of the existing IPR on the original RFC8321 and RFC8889 documents.

13. Has each Author or Contributor confirmed their willingness to be listed as
    such? If the number of Authors/Editors on the front page is greater than 5,
    please provide a justification.

Both documents have 5 authors each, and all authors were involved with the
original RFCs.

14. Identify any remaining I-D nits in this document. (See [the idnits tool][9]
    and the checkbox items found in Guidelines to Authors of Internet-Drafts).
    Simply running the idnits tool is not enough; please review the entire
    guidelines document.

Nits for RFC8889bis:

  ** The abstract seems to contain references ([RFC8889]), which it
    shouldn't.  Please replace those with straight textual mentions of the
    documents in question.

  ** Downref: Normative reference to an Informational RFC: RFC 5474

  == Outdated reference: draft-ietf-ippm-route has been published as RFC 9198

These items should be addressed by making the reference to RFC8889 in the
abstract textual as suggesting, making the reference to RFC5474 informative,
and referring to RFC9198.

15. Should any informative references be normative or vice-versa?

For RFC8889bis: the reference to RFC5474 should become informative.

16. List any normative references that are not freely available to anyone. Did
    the community have sufficient access to review any such normative
    references?

All references are available.

17. Are there any normative downward references (see [RFC 3967][10],
    [BCP 97][11])? If so, list them.

For RFC8889bis: Currently RFC5474 is such, but I believe that should change.

18. Are there normative references to documents that are not ready for
    advancement or are otherwise in an unclear state? If they exist, what is the
    plan for their completion?

No.

19. Will publication of this document change the status of any existing RFCs? If
    so, does the Datatracker metadata correctly reflect this and are those RFCs
    listed on the title page, in the abstract, and discussed in the
    introduction? If not, explain why and point to the part of the document
    where the relationship of this document to these other RFCs is discussed.

Yes, RFC8889bis obsoletes RFC8889, as described in the abstract.

20. Describe the document shepherd's review of the IANA considerations section,
    especially with regard to its consistency with the body of the document.
    Confirm that all aspects of the document requiring IANA assignments are
    associated with the appropriate reservations in IANA registries. Confirm
    that any referenced IANA registries have been clearly identified. Confirm
    that each newly created IANA registry specifies its initial contents,
    allocations procedures, and a reasonable name (see [RFC 8126][12]).

This document does not have IANA actions.

21. List any new IANA registries that require Designated Expert Review for
    future allocations. Are the instructions to the Designated Expert clear?
    Please include suggestions of designated experts, if appropriate.

This document does not have IANA actions.

[1]: https://www.ietf.org/about/groups/iesg/
[2]: https://www.rfc-editor.org/rfc/rfc4858.html
[3]: https://www.rfc-editor.org/rfc/rfc7942.html
[4]: https://trac.ietf.org/trac/ops/wiki/yang-review-tools
[5]: https://www.rfc-editor.org/rfc/rfc8342.html
[6]: https://trac.ietf.org/trac/iesg/wiki/ExpertTopics
[7]: https://www.rfc-editor.org/info/bcp78
[8]: https://www.rfc-editor.org/info/bcp79
[9]: https://www.ietf.org/tools/idnits/
[10]: https://www.rfc-editor.org/rfc/rfc3967.html
[11]: https://www.rfc-editor.org/info/bcp97
[12]: https://www.rfc-editor.org/rfc/rfc8126.html