Service Assurance for Intent-Based Networking Architecture
draft-ietf-opsawg-service-assurance-architecture-13

[Ballot comment]
Thanks to Bron Gondwana for the ARTART review, and to the authors for responding.

The following DISCUSS was resolved during the IESG telechat:

A number of the shepherd writeup questions were hastily answered and the information we need is largely missing.  For instance:

--
> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.
--

There are three questions here, and only the last of them can be answered "yes".  The second one is the most interesting one.

--
> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.
--

I don't understand.

--
> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.
--

So something's been filed, or reasonable reminders were sent?  If the former, where's the discussion or what was its result?  According to the Last Call text, there was one filing.  Was it discussed?  Is it a source of concern?

[Ballot discuss]
We can probably sort this out during the IESG telechat, but I want to be sure it's flagged.

A number of the shepherd writeup questions were hastily answered and the information we need is largely missing.  For instance:

--
> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.
--

There are three questions here, and only the last of them can be answered "yes".  The second one is the most interesting one.

--
> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.
--

I don't understand.

--
> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.
--

So something's been filed, or reasonable reminders were sent?  If the former, where's the discussion or what was its result?  According to the Last Call text, there was one filing.  Was it discussed?  Is it a source of concern?

[Ballot discuss]
** Section 2. health score.  The definition of a health score defined here (0 to 100) is inconsistent with Section 3.2 and the YANG definition in draft-ietf-opsawg-service-assurance-yang which uses -1 to 100.  Given the shepherd’s write-up for the -yang document says that this (changing from 0..100 to -1..100) was a last-minute addition, it seems that perhaps this document hasn’t caught up.

** Section 4.

However, the SAIN agents must be
  secured: a compromised SAIN agent may be sending wrong root causes or
  symptoms to the management systems.  This can be partially achieved
  by correctly setting permissions of each node in the YANG model as
  described in the companion document
  [I-D.ietf-opsawg-service-assurance-yang]

-- Can a more specific section reference be provided into the draft-ietf-opsawg-service-assurance-yang on this configuration?

-- what does it mean to “secure” the SAIN agent and how is this related to a YANG configuration.  Isn’t the agent either a running process on the device or a system/process on another system harvesting information (metrics)?

[Ballot comment]
** Section 1.  Editorial.
  The architecture presented in this document is completed by a set of
  YANG modules …

Should this say s/is completed/is implemented/ ?

** Section 2.  Is the numerical value of the health score comparable across organizations? Network services?  Will the semantics of a score of say “75” be the same everywhere?  Is a linear scale implied?  My assumption is that the computation is not comparable and entirely dependent on the analytics used by organization.  However, I don’t see that stated.

** Section 4.
  Devices should be configured so that agents have their own
  credentials with write access only for the YANG nodes configuring the
  telemetry.

Is this saying that each agent should have distinct credentials, or that the devices should have credentials distinct from agenda?

** Section 4. 

If a closed loop system relies on this architecture then the well
  known issue of those systems also applies, i.e., a lying device or
  compromised agent could trigger partial reconfiguration of the
  service or network.  The SAIN architecture neither augments nor
  reduces this risk.

-- the consequences of a lying decide or compromised agent could be more than triggering a partial reconfiguration

-- if the SAIN architecture is being used for decision support for network management, and the veracity of the information is suspect due to compromise, then SAIN does increase the risk (because incorrect decision could be made based on it).  The SAIN architecture becomes an attack vector.

[Ballot comment]
I'm kinda annoyed -- this is a 27 page document, but I was unable to find any issues (other than those already noted by Eric and Lars) to comment on. There were some readability nits, but even these didn't rise to a level worth commenting...

It's disheartening to read this long a document find anything to complain about...

[Ballot comment]

# Éric Vyncke, INT AD, comments for draft-ietf-opsawg-service-assurance-architecture-12
CC @evyncke

Thank you for the work put into this document. Very interesting and very easy to read document: congratulations !

Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education).

Special thanks to Michael Richardson for the shepherd's detailed write-up including the WG consensus *but* it lacks the justification of the intended status.

Please note that Benno Overeinder is the Internet directorate reviewer (at my request) and you may want to consider this int-dir reviews as well when Benno will complete the review (no need to wait for it though):
https://datatracker.ietf.org/doc/draft-ietf-ippm-rfc8321bis/reviewrequest/16061/

I hope that this review helps to improve the document,

Regards,

-éric

PS: thanks for listing me as contributor.

## COMMENTS

### Section 1

Please explain what the "SAIN orchestrator" is before using this concept. It would also be nice to describe what the "assurance graph" is. As I had some background information on SAIN, I was able to follow the introduction but I am unsure whether other readers will be able to understand it without reading the next section. Suggest to either reduce the section 1 (as section 3 goes in more details) or move the terminology section before the description.

Unsure whether graphs have roots ;-) in "The root of the assurance graph".

### Section 3

Should there be text on how the SAIN orchestrator generates the assurance graph before decomposing it ? I.e., a forward reference to section 3.1

Figure 1, as the text is also about VIM and NFC, should "Network System" be replaced by "System" or any generic term?

### Section 3.1

What is meant by "peer1 tunnel interface" ? Is it the egress interface used by the encapsulated packets ? Or is it the set of ingress interfaces where the to-be-encapsulated packets are received from ? (of course the same reasoning when decapsulating packets).

### Section 3.1.1

Unsure whether DAG is really a requirement (rationale will be welcome in the text), but suggest to rename the 'top' box into 'middle' (or 'synthetic') as it is not really on the top of figure 3. This would also avoid the use of "empty" box in figure 7.

### Section 3.2

Are SLO used anywhere else in the document? Else, don't introduce the acronym.

### Section 3.3

Is `a subservice also defines its assurance`correct? I would have expected `a subservice is associated to its assurance`.

### Section 3.4

What is the purpose of "global computation graph" ?

### Section 3.9

```
  The assurance graph will change over time, because services and
  subservices come and go (changing the dependencies between
  subservices), or simply because a subservice is now under
  maintenance.
```
But section 3.6 seems to indicate that maintenance is just an event/metric and does not cause a change in the assurance graph itself.


## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues.

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments

[Ballot comment]
# GEN AD review of draft-ietf-opsawg-service-assurance-architecture-12

CC @larseggert

Thanks to Paul Kyzivat for the General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/-6ZNHph8My80VstOhYoI8-XOZ4o).

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Outdated references

Document references `draft-ietf-opsawg-service-assurance-yang-09`, but `-10` is
the latest available revision.

Document references `draft-irtf-nmrg-ibn-concepts-definitions`, but that has
been published as `RFC9315`.

### Grammar/style

#### Section 2, paragraph 2
```
ervices, the edges indicate a dependency relations. SAIN collector: A functio
                            ^^^^^^^^^^^^^^^^^^^^^^
```
The plural noun "relations" cannot be used with the article "a".

#### Section 3.1, paragraph 8
```
with the only parameter (the device id) set to "PE1" will appear only once
                                    ^^
```
This abbreviation for "identification" is spelled all-uppercase.

#### Section 3.1.1, paragraph 25
```
given IP address), however it's a not a good parameter to identify an interf
                                ^^^^^^^^^^^^
```
It seems like one article is redundant in this context.

#### Section 3.4, paragraph 1
```
ignored because their state changes is probably the consequence of the main
                                    ^^
```
The verb form "is" does not seem to match the subject "changes".

#### Section 3.6, paragraph 2
```
ice for a L3VPN. * A tunnel can considered as a subservice for an applicatio
                                ^^^^^^^^^^
```
The modal verb "can" requires the verb's base form.

#### Section 3.6, paragraph 5
```
document refer to physical components but this is not a constraint. Indeed, t
                                    ^^^^
```
Use a comma before "but" if it connects two independent clauses (unless they
are closely connected and short).

#### Section 3.7, paragraph 4
```
s to obtain the configuration from the the service orchestrator. The latter
                                  ^^^^^^^
```
Possible typo: you repeated a word.

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-opsawg-service-assurance-architecture-11, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

David Dong
IANA Services Specialist

The following Last Call announcement was sent out (ends 2022-11-20):

From: The IESG
To: IETF-Announce
CC: draft-ietf-opsawg-service-assurance-architecture@ietf.org, mcr@sandelman.ca, opsawg-chairs@ietf.org, opsawg@ietf.org, rwilton@cisco.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Service Assurance for Intent-based Networking Architecture) to Informational RFC


The IESG has received a request from the Operations and Management Area
Working Group WG (opsawg) to consider the following document: - 'Service
Assurance for Intent-based Networking Architecture'
  as Informational
  RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2022-11-20. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document describes an architecture that aims at assuring that
  service instances are running as expected.  As services rely upon
  multiple sub-services provided by a variety of elements including the
  underlying network devices and functions, getting the assurance of a
  healthy service is only possible with a holistic view of all involved
  elements.  This architecture not only helps to correlate the service
  degradation with symptoms of a specific network component but also to
  list the services impacted by the failure or degradation of a
  specific network component.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-opsawg-service-assurance-architecture/


The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/3858/

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

This document is architecture document.
  Huawei has a prototype implementation of this architecture and specifically of the YANG module.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

no reviews were needed for the architecture document

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

yes.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

One document an architecture document.
draft-ietf-opsawg-service-assurance-yang-02 is where the protocol and
implementations would be.  There seem to be enough parties interested, but
as yet, none have indicated that they have implemented.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes. I wonder if the two documents couldn't be published together as a single document.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

The DT does not say Informational for the architecture document (yet).

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section, but YANG module registrations.

Implementation Experience:
  Huawei has a prototype implementation of this architecture and specifically of the YANG module.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

One document an architecture document.
draft-ietf-opsawg-service-assurance-yang-02 is where the protocol and
implementations would be.  There seem to be enough parties interested, but
as yet, none have indicated that they have implemented.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes. I wonder if the two documents couldn't be published together as a single document.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

The DT does not say Informational for the architecture document (yet).

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section, but YANG module registrations.

Implementation Experience:
  Huawei has a prototype implementation of this architecture and specifically of the YANG module.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

One document an architecture document.
draft-ietf-opsawg-service-assurance-yang-02 is where the protocol and
implementations would be.  There seem to be enough parties interested, but
as yet, none have indicated that they have implemented.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes. I wonder if the two documents couldn't be published together as a single document.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

The DT does not say Informational for the architecture document (yet).

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section, but YANG module registrations.