Network Telemetry Framework
draft-ietf-opsawg-ntf-13

[S3.1; question]

* Figure 2: Is PCAP a possible Data Encoding for the Forwarding Plane?

* Figure 2: Should "plain text" be replaced by something like
  "proprietary"?

  I suspect the intent was to communicate that custom/bespoke encodings
  may still be quite common.

Thanks to Alexey Melnikov for the SECDIR review.

Thanks for addressing my DISCUSS point and some of my COMMENTs.

(Ballot note on -12: I wanted to quickly update my ballot before the telechat to reflect that -12 resolved my discuss.  I need to more carefully review the responses to the comments.  Where resolution could be quickly assessed from the diff, I have already updated my ballot accordingly.  -12 may in fact address more of the comments still noted below.)

(Ballot on -11):
I'm a bit of confusion on the framing of this document.  It seems to me to be suggesting that “OAM” is a tied to a series of static technologies and practices, and a set of new practices called “network telemetry” are needed.  I don’t disagree with the idea that network management practices need to evolve, and that the “networks of the future” will look different than today.  Relying on BCP 161 (RFC 6291), I took OAM to mean an evolving set of practices and technology.  Using Section 3 of BCP 161, O + A + M seemed like a contextual set of operations that would be done now and still required in networks of the future.  The document acknowledges that there is some ambiguity in “network telemetry”.  I think it needs to equally acknowledge that the same is true of OAM, and that RFC7276 is not OAM.  In the aggregate, I don’t think the text realizes the clarity that it set out to provide by defining “key characteristics of network telemetry which set a clear distinction from the conventional network OAM and show that some conventional OAM technologies can be considered a subset of the network telemetry technologies.”.  To be clear, I’m not raising an objection to many of the properties linked to network telemetry.  Instead, I think the clarity of message is getting diluted because a very particular distinction is trying to be made (OAM vs. network telemetry) and it isn’t clear.  See below for a specifics.

** Section 1
   Network telemetry extends beyond the historical network Operations,
   Administration, and Management (OAM) techniques and expects to
   support better flexibility, scalability, accuracy, coverage, and
   performance.

This seems hypothetical depending on the definition on which technologies are considered in scope of network telemetry and OAM.

** Section 2.

Today one can access advanced big data analytics capability through a
   plethora of commercial and open source platforms (e.g., Apache
   Hadoop), tools (e.g., Apache Spark), and techniques (e.g., machine
   learning).  Thanks to the advance of computing and storage
   technologies, network big data analytics gives network operators an
   opportunity to gain network insights and move towards network
   autonomy.  
In trying to contextual this observation, where is this capability relative to Figure 1?  In general, I would recommend that this reference architecture when assessing the ecosystem. 

** Section 2.

However, while the data processing capability is improved and
   applications are hungry for more data ...

What does it mean and what applications are “hungry for more data”.  Is a reference possible here?

** Section 2.3
   For a long time, network operators have relied upon SNMP [RFC3416],
   Command-Line Interface (CLI), or Syslog to monitor the network.  Some
   other OAM techniques as described in [RFC7276] are also used to
   facilitate network troubleshooting.
...
   These challenges were addressed by newer standards and techniques
   (e.g., IPFIX/Netflow, PSAMP, IOAM, and YANG-Push) and more are
   emerging.  These standards and techniques need to be recognized and
   accommodated in a new framework.

This section is an exemplar of the disconnect I noted in the definitions of OAM.  The first paragraph presents a narrow view of currently used (albeit older) network monitoring technologies (SNMP, CLI Syslog).  However, in the closing paragraph, the text names more modern technologies I would also consider OAM, and these technologies could meet some of the challenges mentioned in this section.  Furthermore, some of these “newer standards” are framed as things that need to be “recognized”.  This is puzzling because my understanding was that technologies like IPFIX/Netflow have been very widely deployed for quite some time now.  What’s the new framework needed?

** Section 2.4
Network telemetry covers the conventional network OAM and
   has a wider scope.  

Can the text be more specific in what way network telemetry is wider.  I thought OAM was rather ambiguous.

** Section 2.4
Hence, the network telemetry can directly
   trigger the automated network operation, while in contrast some
   conventional OAM tools are designed and used to help human operators
   to monitor and diagnose the networks and guide manual network
   operations.

I’m not sure if this is a fair generalization.  Even “older technologies” like SNMP currently trigger automated responses based on the values they return.

** Section 2.4.  Per “data fusion,” which part of the Figure 1 is this happening?

** Section 2.5.

Network data analytics and machine-learning technologies are applied
   for network operation automation, relying on abundant and coherent
   data from networks.  

What is coherent data?

** Section 2.5
All the use cases and
      applications are better to be supported uniformly and coherently
      under a single intelligent agent

-- Editorial.  There is a missing word which leads to this sentence not parsing.

-- What’s the basis for asserting that a “single intelligent agent” is the  best approach?

-- Maybe the issue is of semantics, what is an “intelligent agent” in this context?

** Section 2.5.  

Network visibility presents multiple viewpoints

and

Efficient data fusion is critical for applications to reduce the
      overall quantity of data and improve the accuracy of analysis.

Are these generalizations expected to be true across the broad use cases?

** Figure 2.  For the management plane, the data model module has MIB and syslog listed, but the data encodings as GPB, JSON and XML.  These data models and encodings don’t line up (i.e., MIBs and syslog typically don’t rely on GPB, JSON or XML).  

** Section 3.1.  Where do network security applications such as WAFs, IDS/IPS/ NGF, DLP, web-proxies, and pDNS fit into this taxonomy?

** Section 3.1.* These sections inconsistently describe properties/requirements for an architectural element and their challenges (but no solutions or requirements for) a given elements.  As a result, I had trouble understanding what an implementer should understand these components.  It would have been clearer is the different modules had common and module specific requirements.

** Section 3.1.1.  Per the requirements of “Convenient Data Subscription”, “Structured Data”, etc. why wouldn’t those be desirable requirements for all four of the modules?

** Section 3.1.3.  Providing “timely data” and “structured data”, seem like the restatements of Section 4.1.1’s “structure data” and “high speed transport”.  Is this a common requirement?

** Section 3.1.3.  Why wouldn’t it be desirable for all of the modules to support incremental deployment note here?

Firstly, thank you for writing this, and thanks to Sheng Jiang for the OpsDir review.

 I personally think that these sort of "overview" / framework documents are really useful; I've often had to get spun up on some new technology and reading an RFC which specifies the packet format without a good overview document (like this one ) is not helpful...

Anyway, with that soapbox rant over, I have 2 substantive comments and some nits to help make the document better / more readable. There is no need to reply to each comment (or at all) - these are non-blocking comments, but fixing them will help make the RFC Editor's job easier and help ensure that none sneak through... 

1:S1 - Introduction
"All the modules are internally structured in the same way, including components that allow to configure data sources in ..."
P: "that allow for the configuration" or "that allow the <subject - operator? system?> to configure what data to..."

"The framework can also simplify the tasks for designing, maintaining, and understanding a network telemetry system."
P:"the task of designing" (or just "simplify the design, maintenance and understanding of ..."

"At last, we outline the evolution stages of the network telemetry system and discuss the potential security concerns."
P: "Finally, ..." or "Lastly, ...". Actually, I think "In addition...." would be even better.

S 2.2 Use Cases
"Intent, as defined in [I-D.irtf-nmrg-ibn-concepts-definitions], is a set of operational goal that a network ..."
s/goal/goals/

"SLA Compliance: A Service-Level Agreement (SLA) defines the level of service a user expects from a network operator,"
I disagree with this -- an SLA is what a network operator has agreed to provide, usually with a contract and penalty to missing it. As a user I *expect* my network operator to always exceed the SLA - if the SLA specifies 95% uptime, I don't really expect 36 hours of downtime each month :-P. Also, in many cases I (sadly) *expect* much worse service than the SLA actually specifies... The Wikipedia https://en.wikipedia.org/wiki/Service-level_agreement page has some good text you could use...

"Root Cause Analysis: Any network failure can be the effect of a sequence of chained events."
The use of "Any" here seems odd / wrong -- perhaps "Network failures are often ..." or "Many network failures..." or just drop the first sentence and start with "Troubleshooting and recovery require ..."

S 2.3
"The poll-based low-frequency data collection is ill-suited..."
Drop the "The", or s/The/This/

"Subscription-based streaming data directly pushed from the data source (e.g., the forwarding chip) is preferred to provide enough data quantity and precision at scale."
Suggest s/enough/sufficient/ (just for flow)

"Comprehensive data is needed from packet processing engine to traffic manager, from line cards to main control board,..."
s/engine/engines/, or, better, "from the packet processing engine to the traffic manager..."

S 3.1. Top Level Modules
"Therefore, we categorize the network telemetry into four distinct modules with each having its own interface to Network Operation Applications."
It would be really helpful to list the "four distinct modules" here -- I spent quite a while looking at the figure (with 5 boxes :-)) trying to infer what you meant. Just adding them in a parenthetical would work well.

"For example, the forwarding chip has high throughput but limited capacity for processing complex data and maintaining states, while ..."
s/states/state/ (Yes, I get that the forwarding engine manages many different sets of state, but "maintaining states" still seems weird...)

Again, thank you for writing this, and also for considering / addressing these nits...

Thanks for working in the documents and thanks Michael Scharf for the TSVART review. I am happy with that fact that the table with selected technology is now lists examples.

Thank you for the work put into this document. All in all, this is a very good introduction document to telemetry but I am unsure whether it is a 'framework' (and this is a real concern of mine). Due to the amount of documents on the next IESG telecast agenda, I did not review the appendixes.

Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education), and some nits.

Special thanks to Alexander Clemm for the shepherd's write-up about the WG consensus.

Thanks also to Jean-Michel Combes for his Internet directorate review at:
https://datatracker.ietf.org/doc/review-ietf-opsawg-ntf-10-intdir-telechat-combes-2021-11-24/

I believe that Jean-Michel spotted a serious issue in the security section but I trust the Security Area Directors on this specific topic and will not raise a block DISCUSS on this point especially as Haoyu Song has taken some actions.

I hope that this helps to improve the document,

Regards,

-éric

As a non-English speaker, the document title is a little ambiguous at first sight: is it about telemetry about network data or about using the network to get some telemetry ? Unsure whether this ambiguity can be removed.

BTW, I like the sections on use cases and challenges but I am ambivalent whether they are useful in this document.

A lot of the concepts in this framework could be used for other applications beyond network layer telemetry. Was this extension considered by the authors ?

There are several informative references to IRTF & IETF drafts, which is OK of course but those drafts may never be published. Perhaps, is it premature to publish this document ?

-- Section 2 --
Suggest to rely on https://www.rfc-editor.org/materials/abbrev.expansion.txt to avoid redefining well-known terms as JSON or MIB.

-- Section 3 --
No need to reply but I am unsure whether the paragraph about SDN, AN, ... brings any value to this document.

-- Section 3.1 --
Should there be some words about whether actual packet content is part of telemetry ?

-- Section 3.2 --
Should there be a mention of "big data" again when enumerating the 4 Vs ?

-- Section 3.3 --
Is it the forwarding chip that generate, package, and send the telemetry as hinted in the first bullet ? I would guess that the chip indeed collects the data but it is up to another component to do rest.

Please add a definition/reference for "sFlow".

-- Section 3.4 --
Should there also be a mention of common naming/ID if data needs to be merged among different sources ? I.e., having common keys with the same format or at least having some equivalence.

"In-network customisation" seems partly contradictory with unified models or did I misread this part ?

For an uneducated reader (like myself), I wonder whether actual packet content is included in "The data originated from the data plane forwarding chips"

-- Section 3.5 --
In "Efficient data fusion is critical for applications to reduce the overall quantity of data", is it about "fusion" or "aggregation" ?

-- Section 4.1 --
In figure 2, what is 'template' ? This does not seem a well-defined term, suggest to remove it.

Also in figure 2, suggest replace "HTTP" by "HTTPS"

In figure 2, what is the meaning of "plain" for data encoding ?

-- Section 4.1.3 --
Could the "quality" in the first paragraph be described as it is rather vague?

Later in this section "The data should be structured and labeled", what is meant by 'labeled' ? And later in the same §, I am unsure how to understand "data types"... (as I read "data types" as float vs. integer) or is it simply "data"

Suggest to make the taxonomy part a subsection.

Is there a reason why "AM" is not expanded in "AM [RFC8321]" ?

In "Big Data sources that analyse streams of information, such as Twitter messages", I am afraid that I fail to understand how sources can analyse data.

-- Section 4.2 --
I fail to understand why formatting is part of "Data Generation and Processing" and not of "Data Encoding and Export".

-- Section 4.4 --
Please excuse my lack of knowledge in this domain, but I have hard time to understand how MIB & YANG modules can help in data generation and processing in figure 5. Should there be some additional clarifications ? Again, possibly because I am not an expert in this field.

-- Section 5 --
It is unclear for me whether the 4 stages (BTW why starting at stage 0 rather than stage 1 ?) are about telemetry (as a means) or about use cases.



== NITS ==


-- Section 2 --
Missing ':' after YANG ECA

Found terminology that should be reviewed for inclusivity; see
https://www.rfc-editor.org/part2/#inclusive_language for background and more
guidance:

 * Term "his"; alternatives might be "they", "them", "their".

 * Term "traditional"; alternatives might be "classic", "classical",
   "common", "conventional", "customary", "fixed", "habitual", "historic",
   "long-established", "popular", "prescribed", "regular", "rooted",
   "time-honored", "universal", "widely used", "widespread".

Thanks to Gyan S. Mishra for their General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/ItkS25VOVfDg8eLogFDMCtpeoFM).

-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

Section 3.2. , paragraph 4, nit:
> ic manipulation. In some cases, micro-bursts need to be detected in a very sh
>                                 ^^^^^^^^^^^^
This word is normally spelled as one.

Section 3.2. , paragraph 6, nit:
> ant to be warned of issues in advance so proactive actions can be taken to av
>                                      ^^^
Use a comma before "so" if it connects two independent clauses (unless they are
closely connected and short).

Section 4.1. , paragraph 3, nit:
> lso be implemented over TCP and SCTP but that is not recommended for forward
>                                     ^^^^
Use a comma before "but" if it connects two independent clauses (unless they
are closely connected and short).

Section 10. , paragraph 19, nit:
> P/2 [RFC7540] based open-source micro-service communication framework. It pro
>                                 ^^^^^^^^^^^^^
This word is normally spelled as one.

"A.3.1. ", paragraph 2, nit:
> pected movement makes it fascinating and many people connect to sites that a
>                                     ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

"A.3.1. ", paragraph 3, nit:
> ent can be affected by such situation so their management solutions should be
>                                      ^^^
Use a comma before "so" if it connects two independent clauses (unless they are
closely connected and short).

Document references draft-ietf-ippm-multipoint-alt-mark, but that has been
published as RFC8889.

Document references draft-song-ippm-postcard-based-telemetry-10, but -11 is the
latest available revision.

Document references draft-ietf-grow-bmp-adj-rib-out, but that has been
published as RFC8671.

A.3.4 and A.3.5. Note that IOAM has a direct export mode in an adopted draft with similar properties to PBT. https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-direct-export/

The reference to draft-ippm-multipoint-alt-mark is now RFC8889.

Thanks to Michael Scharf for the TSVART review.