Multipoint Alternate-Marking Method for Passive and Hybrid Performance Monitoring
draft-ietf-ippm-multipoint-alt-mark-09

[Ballot comment]
I support Ben Kaduk’s DISCUSS position.

** Section 1.  Per “There are some applications of the alternate marking method where there are a lot of monitored flows and nodes.”
-- Editorial.  “alternative marking methods” is lower case but “Alternative Marking methodology” is upper case in other places

-- “a lot of monitored” seems colloquial and imprecise and begged for me, “what’s a lot”? (subsequent text says “n and m are high values”, but I also don’t have a sense for what that might be)

** Section 1.  Per “Without network clustering, it is possible to apply alternate marking only for all the network or per single flow.  Instead, with network    clustering, it is possible to use the network clusters partition …”, the phrase “network clusters partition” doesn’t parse for me.

** Section 4.  Per “By Using the "traditional" alternate marking method only point-to-point paths can be monitored.”, perhaps say alternative marking per RFC8321 instead of “traditional …”.

** Section 4.1.  “In general there are different options: the monitoring network can be obtained by considering all the possible paths for the traffic or also by checking the traffic sometimes and update the graph consequently.”

-- Editorial: s/In general there are different options: … by checking the traffic sometimes and update the graph consequently/In general, there are different options: by periodically checking the traffic and updating the graph as appropriate/

-- Is there any guidance on the how to frequently this evaluation of traffic should be?

** Section 5.  How generalizable is the flow description?  These statements seem inconsistent:

-- Section 4 says “Multipoint Alternate Marking enables the performance measurement for multipoint flows selected by identification fields without any constraints (even the entire network production traffic). 

-- Section 5 says “The flow definition is generalized here, indeed, as described before, a multipoint packet flow is considered and the identification fields of the 5-tuple can be selected without any constraints.

** Section 9.  I didn’t understand what this section was saying that is different than Section 10.  IMO, the “SDN Controller calibrating for performance measurements” and [I-D.song-opsawg-ifit-framework] descriptions seemed like “Examples of applications” covered in Section 10.  The title of this section “An Intelligent Performance Management approach” seemed a little “marketing like”.

[Ballot discuss]
Let's discuss whether [I-D.mizrahi-ippm-compact-alternate-marking] needs
to be a normative reference, to describe how the Hash selection method
works for multipoint.  This document alone does not even mention what is
used as input to the hash (though I think I have a good guess based on
the context).  Even if the intent is that RFC 5474 suffices (avoiding
the "dependency on individual document" issue), that is also listed only
as an informative reference.

Also, if the grouping procedure (section 6.1) does in fact require a
distinguished (but arbitrary?) choice of initial endpoint as I suspect
it does, that should be clarified.  (See COMMENT.)

[Ballot comment]
I echo the other reviewers' comments about clarity as to "multipoint"
vs. "multicast" and defining various terms.

I agree with Barry that there's a lot of editorial work to be done; I've
to note many instances in these comments even when the proper resolution
is unclear to me.

Section 1

  This approach fits very well with the Intelligent Network and
  Software Defined Network (SDN) paradigm where the SDN Orchestrator
  and the SDN Controllers are the brains of the network and can manage
  flow control to the switches and routers and, in the same way, can
  calibrate the performance measurements depending on the necessity.

nit: "necessity" doesn't sound right

  An SDN Controller Application can orchestrate how deep the network
  performance monitoring is setup by applying the Multipoint Alternate
  Marking as described in this document.

nit: "how deep the network" doesn't sound right

Section 3

  In this way the flows to be monitored are selected into the
  monitoring points using packet selection rules, that can also change
  the pattern of the monitored network.

nit: I'm not sure what "this way" refers to.  It also seems like this
may be two independent thoughts needlessly joined together with a comma.

Section 4.1

  [I-D.ietf-ippm-route]).  In general there are different options: the
  monitoring network can be obtained by considering all the possible
  paths for the traffic or also by checking the traffic sometimes and
  update the graph consequently.

"also by checking the traffic sometimes and update the graph
consequently" feels pretty informal and under-specified.

Section 5

  Since all the packets of the considered flow leaving the network have
  previously entered the network, the number of packets counted by all
  the input nodes is always greater or equal than the number of packets
  counted by all the output nodes.

[I think I could imagine some exotic cases where this does not hold, but
none of them really seem topical for this work.]

  It is possible to define the Network Packet Loss of one monitored
  flow for a single period: <>.  This is true for
  every packet flow in each marking period.

As another reviewer implies, this discounts the difference between the
number of packets "still in the network" at the start and end of the
measurement period.

      The flow definition is generalized here, indeed, as described
      before, a multipoint packet flow is considered and the
      identification fields of the 5-tuple can be selected without any
      constraints.

I don't think I understand what this means.  If identification is fully
general, how are we still limited to a 5-tuple?

Section 6
  In a completely monitored network (a network where every network
  interface is monitored), each network device corresponds to a Cluster
  and each physical link corresponds to two Clusters (one for each
  direction).

("what about unidirectional links?")

Section 6.1

I'm not following how the first step of this algorithm produces the
listed groups.  Are the links considered to be bidirectional?  Did we
have to pick some arbitrary starting node (R1), and decree that once a
link is in one group it cannot be in some other group?

  In this way the calculation of packet loss can be made on Cluster
  basis.  Note that CIR(Committed Information Rate) and EIR(Excess
  Information Rate) can also be deduced on Cluster basis.

Do we use CIR and/or EIR anywhere else?  (Are they references to some
other body of work?)

  In this way in a very large network there is no need to configure
  detailed filter criteria to inspect the traffic.  You can check
  multipoint network and only in case of problems you can go deep with
  a step-by-step cluster analysis, but only for the cluster or
  combination of clusters where the problem happens.

nits: there at least one missing article here ("a multipoint network")
and I think there was something else that feels not quite right.

  In summary, once defined a flow, the algorithm to build the Cluster
  Partition considers all the possible links and nodes crossed by the

What does "once defined a flow" mean?

  information.  So, if the flow do not enter or traverse all the nodes,
  the counters has a non-zero value for the involved nodes, while a

nit: singular/plural mismatch "the flow"/"do not", "counters"/"has a
[...] value"

  The algorithm described above is an Iterative clustering algorithm,
  but it is also possible to apply a Recursive clustering algorithm by
  using the node-node adjacency matrix representation.

Is there a reference for this?  (Do I assume it's the
[IEEE-ACM-ToN-MPNPM] at the end of the next paragraph?)

Section 7
  Therefore, when we expand to multipoint-to-multipoint flows, we have
  to consider that all source nodes mark the traffic and this adds more
  complexity.

I'm not sure what chain of reasoning "therefore" is attempting to
indicate.

  But we should now consider an additional contribution.  Since all
  source nodes mark the traffic, the source measurement intervals can
  be of different lengths and with different offsets and this mismatch
  m can be added to d, as shown in figure.

Please define m and d.

  So the misalignment between the marking source routers gives an
  additional constraint and the value of m is added to d (that already
  includes clock error and network delay).

  Thus, three different possible constraints are considered: clock
  error between network devices, network delay between measurement
  points and the misalignment between the marking source routers.

Are these "constraints" or "contributions [to error/uncertainty]"?  RFC
8321 does not seem to use "constraint" in this fashion.

  In the end, the condition that must be satisfied to enable the method
  to function properly is that the available counting interval must be
  > 0, and that means: L - 2m - 2d > 0 for each measurement point on
  the multipoint path.  Therefore, the mismatch between measurement
  intervals must satisfy this condition.

Is it bad to just make L really large?

Section 8

  period and this is the time reference that can be used.  It is
  important to highlight that both delay and delay variation
  measurements make sense in a multipoint path.  The Delay Variation is
  calculated by considering the same packets selected for measuring the
  Delay.

Is the "variation" considered here just the variation across the
separate paths that the selected packets take, or over time, or
something else?

  o  Delay measurements on single packets basis means that you can use
      multipoint path just to easily couple packets between inputs and

nit: singular/plural mismatch "packets"/"basis" (also missing "a"?)
nit: "inpur" singular, I think.

Section 8.1.1

I don't understand what weights are used for the "weighted averages".

Section 8.2.1

  since they would not be representative of the entire flow.  The
  packets can follow different paths with various delays and in general
  it can be very difficult to recognize marked packets in a multipoint-
  to-multipoint path especially in case they are more than one per
  period.

nits: "the case when there is", comma before "and in general".

Section 8.2.2

  [I-D.mizrahi-ippm-compact-alternate-marking] introduces how to use
  the Hash method combined with alternate marking method for point-to-

Is it really appropriate for a WG document to depend on an individual
document to introduce a concept?

  In a multipoint environment the behaviour is similar to point-to
  point flow.  In particular, in the context of multipoint-to-
  multipoint flow, the dynamic hash could be the solution to perform

nits: both of these either need "flows" plural or the article "a".

  The management system receives the samples including the timestamps
  and the hash value from all the MPs, and this happens both for point-
  to-point and for multipoint-to-multipoint flow.  Then the longest

nit: "flows"

Also, this seems to be the first time we abbreviate "measurement
points"; it would be good to provide the abbreviation/expansion together
and consistently use the abbreviation.  (Interestingly
https://www.rfc-editor.org/materials/abbrev.expansion.txt only has
"multiprotocol" and "maintenance point".)

  to-point and for multipoint-to-multipoint flow.  Then the longest
  hash used by MPs is deduced and it is applied to couple timestamps of
  same packets of 2 MPs of a point-to-point path or of input and output
  MPs of a Cluster (or a Super Cluster or the entire network).  But

nit: there appear to be several missing words here.

  In summary, the basic hash is logically similar to the double marking
  method, and in case of point-to-point path double marking and basic

I don't really recall any specific discussion of similarities between
basic hash and double marking, so calling this a "summary" is perhaps a
stretch.

Section 9

  An SDN Controller or a Network Management System (NMS) can calibrate
  Performance Measurements since it is aware of the network topology.
  It can start without examining in depth.  In case of necessity
  (packet loss is measured or the delay is too high), the filtering
  criteria could be immediately specified more in order to perform a
  partition of the network by using Clusters and/or different
  combinations of Clusters.  In this way the problem can be localized

nit: "specified more" seems incomplete; perhaps something about detail?

Section 11

I don't see much in RFC 8321 to note that "traffic observed by
measurement points may contain private information if not protected by
transport-layer security protocols, so measurement infrastructure should
be as equally protected/secured as routing hardware".  That said, it is
hopefully obvious, and not specific to this work, so I don't feel a
particular need to have it mentioned here.

[Ballot comment]
(1) Some of the terminology in this document can be confusing.  For example, I can't help but thinking about multicast and replication when reading about a point-to-multipoint flow.  Digging into the document, the concept is eventually explained (and the fact that multicast is not supported).  I think that there would be great benefit in adding a Terminology section to include a short definition of the flow types and other terms (production network, monitoring network, etc.).


(2) This document is classified as Experimental.  What is the experiment? 

Given that the document describes a methodology and not an interoperable implementation, should the experiment be, for example, to consider the applicability of the methodology to different applications?  It can obviously be something else...but I think it is important to indicate what it is.

FWIW, given that this is an "extension of RFC 8321", I think that Experimental is the right status.

[Ballot comment]
I found this to be a somewhat difficult read, with some awkward wording.  Because of time constraints I’ll mostly not comment specifically, but leave it to the RPC.  My comments below on Section 1 are examples.

— Abstract —

  This document aims to generalize and expand this
  methodology

“This document generalizes and expands this methodology”

— Section 1 —
The “; so” in the first sentence doesn’t follow from what comes before.  It seems that you’ve just glued together two unrelated sentences, and I would unglue them: “point-to-point path.  The extension proposed”

How does the extension explain anything?  Isn’t it the document that does the explaining, but then the extension that does the enabling?

I can’t parse the first sentence of the second paragraph, and I can’t figure out what you’re trying to say.  Can you try re-writing it?

Please check to make sure you are consistent about capitalizing “Alternate Marking” and “Multipoint Alternate Marking” through the document.  It looks like you have it about half and half.

[Ballot comment]
Thank you for the work put into this document. Beside some of my COMMENT about the wording, the flow and the explanations make the document easy to read.

Please find below some non-blocking COMMENTs and NITs. An answer will be appreciated.

I hope that this helps to improve the document,

Regards,

-éric

== COMMENTS ==

The wording "Multipoint to multipoint" raised confusion in my mind to be honest. It took reading further in the document to understand what it was about. What about: "multi-party multiple flows" or "aggregated alternate marking" or "clustered alternate marking" ?

-- Section 1 --
In the 5th paragraph, suggest to replace "Multipoint alternate marking" by "this document" (if my assumption is correct).

" Intelligent Network " looks more like a marketing name than an IETF defined one.

-- Section 3 --
What's in a name... but using "flow" to convey the semantics of multiple flows is weird. Why not using "aggregated flows" ?

"headers fields" is it limited to layer-3 only? Probably worth specifying so early in the text by "layer-3 and layer-4 headers fields"

Using "TCP 5-tuple" looks simple but what about fragmented packets ?

I really have problems with sentences such as "ECMP flow is in scope by definition, since it is a point-to-multipoint unicast flow", esp "point-to-multipoint unicast flow" that looks like an oxymoron to me. Perhaps, it is only me having this parsing problem ?

-- Section 4 --
Again about IPv4 fragmentation, if fragmentation happens on the path, then there could be more IP packets received than sent. Or are non-initial fragments ignored? If so, I suggest to clarify the text.


== NITS ==

RFC 7322 suggests "The full expansion of the text should be followed by the abbreviation itself in parentheses" ;-) Not always applied through this document: e.g., CIR, OTT

-- Section 1 --
s/ECMP (Equal-Cost Multi-Path) paths/ECMP (Equal-Cost MultiPath)/ to avoid repeating "path".

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-ippm-multipoint-alt-mark-05, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2020-03-06):

From: The IESG
To: IETF-Announce
CC: ippm-chairs@ietf.org, ippm@ietf.org, draft-ietf-ippm-multipoint-alt-mark@ietf.org, Tal Mizrahi , tal.mizrahi.phd@gmail.com, ietf@kuehlewind.net
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Multipoint Alternate Marking method for passive and hybrid performance monitoring) to Experimental RFC


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Multipoint Alternate Marking method
for passive and hybrid performance
  monitoring'
  as Experimental RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2020-03-06. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  The Alternate Marking method, as presented in RFC 8321 [RFC8321], can
  be applied only to point-to-point flows because it assumes that all
  the packets of the flow measured on one node are measured again by a
  single second node.  This document aims to generalize and expand this
  methodology to measure any kind of unicast flows, whose packets can
  follow several different paths in the network, in wider terms a
  multipoint-to-multipoint network.  For this reason the technique here
  described is called Multipoint Alternate Marking.  Some definitions
  here introduced extend the scope of RFC 5644 [RFC5644] in the context
  of alternate marking schema.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-multipoint-alt-mark/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-ippm-multipoint-alt-mark/ballot/

The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/4010/
  https://datatracker.ietf.org/ipr/3035/
  https://datatracker.ietf.org/ipr/3110/

This shepherd writeup follows the Essay Style Document Writeup template (https://www.ietf.org/chairs/document-writeups/essay-style-document-writeup/).


1. Summary

The document shepherd is Tal Mizrahi, and the responsible area director is Mirja Kühlewind.

This document describes Multipoint Alternate Marking, which is a method for measuring loss and delay in a multipoint-to-multipoint network, using an alternate marking field. The document extends the alternate marking method that was previously introduced in RFC 8321 to mp-to-mp scenarios. Note that RFC 8321 is also a document that was published by the IPPM working group.

The intended status of this document is Experimental, as it presents a measurement methodology without defining a specific protocol.


2. Review and Consensus

The draft was first submitted in June 2017, has been reviewed by a fair number of people in the IPPM working group, has had a fair number of supporters, and no objections from the working group. The main comments included clarification questions regarding the description of the measurement procedure and about some of the figures in the document. The draft was adopted by the working group in November 2018.

During working group last call the draft had a fair number of supporting WG members, with relatively minor comments, and these comments were addressed by the authors.

The authors of this draft have implemented a prototype of the methodology using Open vSwitch in the Mininet emulation environment. The implementation is publicly available, and some experimental evaluation has been published by the authors ("Multipoint Passive Monitoring in Packet Networks", IEEE/ACM Transactions on Networking).

The current version of the draft is clear, seems to have resolved all the issues, and has the consensus of the working group.


3. Intellectual Property

Three IPRs have been disclosed regarding this draft:
3035 Telecom Italia SpA's Statement about IPR related to draft-fioccola-ippm-multipoint-alt-mark
3110 Telecom Italia SpA's Statement about IPR related to draft-fioccola-ippm-multipoint-alt-mark
4010 Telecom Italia SpA's Statement about IPR related to draft-ietf-ippm-multipoint-alt-mark

The disclosures are available at https://datatracker.ietf.org/ipr/search/?submit=draft&id=draft-ietf-ippm-multipoint-alt-mark

An IPR poll was performed for this draft on the IPPM mailing list. The authors have confirmed on the mailing list that they are not aware of any related IPRs beyond the IPRs above.


4. Other Points

The draft does not include any requests from IANA.

This shepherd writeup follows the Essay Style Document Writeup template (https://www.ietf.org/chairs/document-writeups/essay-style-document-writeup/).


1. Summary

The document shepherd is Tal Mizrahi, and the responsible area director is Mirja Kühlewind.

This document describes Multipoint Alternate Marking, which is a method for measuring loss and delay in a multipoint-to-multipoint network, using an alternate marking field. The document extends the alternate marking method that was previously introduced in RFC 8321 to mp-to-mp scenarios. Note that RFC 8321 is also a document that was published by the IPPM working group.

The intended status of this document is Experimental, as it presents a measurement methodology without defining a specific protocol.


2. Review and Consensus

The draft was first submitted in June 2017, has been reviewed by a fair number of people in the IPPM working group, has had a fair number of supporters, and no objections from the working group. The main comments included clarification questions regarding the description of the measurement procedure and about some of the figures in the document. The draft was adopted by the working group in November 2018.

During working group last call the draft had a fair number of supporting WG members, with relatively minor comments, and these comments were addressed by the authors.

The authors of this draft have implemented a prototype of the methodology using Open vSwitch in the Mininet emulation environment. The implementation is publicly available, and some experimental evaluation has been published by the authors ("Multipoint Passive Monitoring in Packet Networks", IEEE/ACM Transactions on Networking).

The current version of the draft is clear, seems to have resolved all the issues, and has the consensus of the working group.


3. Intellectual Property

Three IPRs have been disclosed regarding this draft:
3035 Telecom Italia SpA's Statement about IPR related to draft-fioccola-ippm-multipoint-alt-mark
3110 Telecom Italia SpA's Statement about IPR related to draft-fioccola-ippm-multipoint-alt-mark
4010 Telecom Italia SpA's Statement about IPR related to draft-ietf-ippm-multipoint-alt-mark

The disclosures are available at https://datatracker.ietf.org/ipr/search/?submit=draft&id=draft-ietf-ippm-multipoint-alt-mark

An IPR poll was performed for this draft on the IPPM mailing list. The authors have confirmed on the mailing list that they are not aware of any related IPRs beyond the IPRs above.


4. Other Points

The draft does not include any requests from IANA.