A YANG Data Model for Service Assurance
draft-ietf-opsawg-service-assurance-yang-11

[Ballot comment]
Thank you for the -11 revision, it addresses all my previous blocking DISCUSS points [1] and most of my COMMENT ones.

Still wondering why under-maintenance.contact can be free form in addition to an URI. But, this is not a blocking point.

Please note that Tommy Pauly is the Internet directorate reviewer (at my request) and you may want to consider this int-dir reviews as well when Tommy will complete the review (no need to wait for it though):
https://datatracker.ietf.org/doc/draft-ietf-opsawg-service-assurance-yang/reviewrequest/16806/

As I like the work, I am even balloting a YES.

[1] See https://mailarchive.ietf.org/arch/msg/opsawg/lH6bv_qVwxO7r5kYYHodZbpxhXs/

[Ballot comment]
I support Eric's DISCUSS position.  In particular, please fix the BCP 14 boilerplate.  BCP 13 is MIME registration procedures.

We covered the terseness of the shepherd writeup during the telechat, so my DISCUSS is resolved.  Preserving the comment:

A number of the shepherd writeup questions were hastily answered and the information we need is largely missing.  For instance:

--
> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.
--

There are three questions here, and only the last of them can be answered "yes".  The second one is the most interesting one.

--
> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.
--

I don't understand.

--
> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.
--

So something's been filed, or reasonable reminders were sent?  The datatracker appears to imply this must mean the latter, but it would be great to be clear.

[Ballot discuss]
We can probably sort this out during the IESG telechat, but I want to be sure it's flagged.

A number of the shepherd writeup questions were hastily answered and the information we need is largely missing.  For instance:

--
> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.
--

There are three questions here, and only the last of them can be answered "yes".  The second one is the most interesting one.

--
> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.
--

I don't understand.

--
> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.
--

So something's been filed, or reasonable reminders were sent?  The datatracker appears to imply this must mean the latter, but it would be great to be clear.

[Ballot comment]
I have not much to add to my esteemed collegue's Éric::Vyncke/128 review, two nits:

  |    +--rw under-maintenance!
  |    |  +--rw contact    string

This could use a freeform field or duration field to hold information about expected maintenance time, as that would be the main question anyone could have for the contact person :)

  description
    "This module extends the ietf-service-assurance-device module to
    add specific support for devices of ACME Corporation. ";

I would somehow put in the Example word in there to avoid anyone thinking that this is a real definition. I know it is stated before, but implementers just scanning code might miss it :)

[Ballot comment]
Thanks for working on this document.

Boilerplate and downref issues have already been pointed out by other ADs. I have a similar thought or question about namespace as Eric V has, would like to see the response from the authors.

[Ballot comment]
** YANG.  under-maintenance/contact. 

It is suggested that this name contain one or more of the
  following: IP address, management station name,
  network manager's name, location, or phone number.

Section 1 of the documents states that this YANG module is intended to be machine readable.  This field appears to be a free form text field with a number of possible of values.  Should additional structure be provided?

[Ballot discuss]

# Éric Vyncke, INT AD, comments for draft-ietf-opsawg-service-assurance-yang-10
CC @evyncke

Thank you for the work put into this document. A very interesting piece of work and a well-written piece of text (even if I am balloting DISCUSS). The examples are also helping.

Please find below some DISCUSS points (+ suggestions), some non-blocking COMMENT points (but replies would be appreciated even if only for my own education).

Special thanks to Michael Richardson for the shepherd's detailed write-up including the WG consensus *but* it lacks the justification of the intended status. It would have been nice to list the implementations (even if I know one).

Please note that Tommy Pauly is the Internet directorate reviewer (at my request) and you may want to consider this int-dir reviews as well when Tommy will complete the review (no need to wait for it though):
https://datatracker.ietf.org/doc/draft-ietf-opsawg-service-assurance-yang/reviewrequest/16806/

Also, thanks to the WG chairs and the responsible AD to bundle this I-D and its companion to the same IESG telechat: it helps a lot!

I hope that this review helps to improve the document,

Regards,

-éric


## DISCUSS

As noted in https://www.ietf.org/blog/handling-iesg-ballot-positions/, a DISCUSS ballot is a request to have a discussion on the following topics:

### BCP14 template

As noted by https://author-tools.ietf.org/api/idnits?url=https://www.ietf.org/archive/id/draft-ietf-opsawg-service-assurance-yang-10.txt and Lars, the BCP14 template is not correct even if it is required for a proposed standard (it mentions BCP13 ;-) ).

As I have further DISCUSS issues below, I am raising the trivial BCP14 issue to a blocking DISCUSS.

### Section 3.3

To my SQL eyes, it hurts to use a -1 value for health-score when there is no value. There is no "mandatory true" statement for this leaf, i.e., it can be absent in the telemetry. Is there a semantic difference between the absence of health-score and the value of -1 ? Is the SAIN collector expected to process those 2 cases differently ?

Suggest to either remove the -1 sentinel value, or add "mandatory true" attribute, or be specific about the difference (if any).

### Section 4

It is unclear from this section whether it applies to IETF-specified YANG modules only? I.e., may a vendor augment this IETF YANG module in its own namespace ? I guess so, but worth writing it (or restrict this section to future IETF work only).

[Ballot comment]

## COMMENTS

### Downref

As noted by https://author-tools.ietf.org/api/idnits?url=https://www.ietf.org/archive/id/draft-ietf-opsawg-service-assurance-yang-10.txt and Lars, there is a downward reference that was not mentioned in the IETF LC.

### Lack of YANG doctors review ?

Even if one author is Mr. YANG ;-), is there a reason why there is neither a Last Call nor a Telechat review by the YANG doctors ?

### Section 1

```
[I-D.ietf-opsawg-service-assurance-architecture] specifies an
  architecture and a set of involved components for service assurance,
  called Service Assurance for Intent-Based Networking (SAIN).
```
The SAIN architecture draft is informational, so it cannot "specify". Please use "describes".

### Section 3.1

```
  The two subservices presented above need different sets of parameters
  to fully characterize one of their instance.  An instance of the
  device subservice is fully characterized by a single parameter
  allowing to identify the device to monitor.  For ip-connectivity
  subservice, at least the device and IP address for both ends of the
  link are needed to fully characterize an instance.
```
s/two subservices presented/two example subservices presented/

While I am not English native speaker, I wonder whether the plural form should be used for "IP address for both ends" ?

`The dependencies are modelled as an adjacency list,` or simply `The dependencies are modelled as a list,` ?

### Section 3.2

```
  The date of last change "assurance-graph-last-change" is read only.
  It must be updated each time the graph structure is changed by
  addition or deletion of subservices, dependencies or modification of
  their configurable attributes.
```
Is 'under-maintenance' a triggering change to update the last change time ? Perhaps worth mentionning.

### Section 3.3

Should the under-maintenance.contact be more specified? I.e., as a URI like phone:+12309182309 or mailto:jean@example.net ? One goal of this document (section 1) is to be 'machine readable' ;-)

leaf health-score-weight, the use of this leaf is rather under specified. Should it rather be a float between 0.0 and 1.0 ? I also wonder whether the last sentence of the description applies to this leaf ?

I will let to my fellow ART AD to check whether the waiver `Therefore, no mechanism for language tagging is needed.` is acceptable for lack of i18n support (for me: it is ok).

### Section 5

Is a device always a 'physical' device or can it be virtual as well ?

Should there be a link to the miscellaneous 'inventory models' in OPSAWG ? If not, should there be more information about the device, e.g., its location.

### Section 6

I am not familiar enough with YANG, but should there be some text or YANG statement to declare the 'device' leaf in this module as a foreign key to the device module ?

Should the interface model handle the sub-interface (e.g., a specific VLAN on a GigEthernet interface) ?

### Appendix C

Thanks for using an IPv6 example ;)

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues.

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments

[Ballot comment]
Thanks the the authors for writing this document, and also to Bo Wu for the OpsDir review - https://datatracker.ietf.org/doc/review-ietf-opsawg-service-assurance-yang-10-opsdir-telechat-wu-2022-12-12/ .


I'm not a YANG expert, and so much of my comments are just editorial / readability nits to try make the document even better. Feel free to address them, or not...

1: "An instance of the device subservice is representing a subpart of the network system, namely a specific device. An instance of the ip-connectivity subservice representing a feature of the network, namely the connectivity between two specific IP addresses on two devices."
P: s/subservice is representing a/subservice represents a/  (in both places)

2: "For ip-connectivity subservice, at " -> "For *the* ip-connectivity subservice, at "

3: "a health-status indicating how healthy the subservice is and when the subservice is not healthy, a list of symptoms explaining why the subservice is not healthy."
P: "a health-status indicating how healthy the subservice is, and when the subservice is not healthy, a list of symptoms explaining why the subservice is not healthy." (I had to reread this a few time to parse)

4: "However, such an instability might be a relevant symptom for diagnosing the root cause of a problem."
P: "However, such instability might be a relevant symptom for diagnosing the root cause of a problem."

5: "In this section, we propose some guidelines in order to build theses extensions."
s/theses/these/

6: "We define two subservice types in the next sections:"
P: "We define two subservice types in the following sections:"

[Ballot comment]
# GEN AD review of draft-ietf-opsawg-service-assurance-yang-10

CC @larseggert

Thanks to Dan Romascanu for the General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/PUK6ex0SQtj7-di4O2RM31mul2A).

## Comments

### Boilerplate

This document uses the RFC2119 keywords "SHALL NOT", "SHOULD NOT", "SHALL",
"MAY", "REQUIRED", "RECOMMENDED", "OPTIONAL", "SHOULD", "MUST NOT", "NOT
RECOMMENDED", and "MUST", but does not contain the recommended RFC8174
boilerplate. (It refers to BCP 13 and not BCP 14...)

### DOWNREFs

DOWNREF `[I-D.ietf-opsawg-service-assurance-architecture]` from this Proposed
Standard to Informational `draft-ietf-opsawg-service-assurance-architecture`.
(For IESG discussion. It seems this DOWNREF was not mentioned in the Last Call
and also seems to not appear in the DOWNREF registry.)

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Outdated references

Reference `[RFC7895]` to `RFC7895`, which was obsoleted by `RFC8525` (this may
be on purpose).

### Grammar/style

#### Section 3.2, paragraph 29
```
. The subservices hierarchically organised by dependencies constitute an ass
                                ^^^^^^^^^
```
Do not mix variants of the same word ("organise" and "organize") within a
single text.

#### Section 3.3, paragraph 16
```
ption "Date and time at which the symptoms history starts for this subservic
                                  ^^^^^^^^
```
An apostrophe may be missing.

#### Section 3.3, paragraph 18
```
iption "Container for the list of agents’s symptoms"; list agent { key "id";
                                  ^^^^^^^^
```
Did you mean "agent's" or "agents'"?

#### Section 3.3, paragraph 18
```
e some guidelines in order to build theses extensions. The mechanism to add a
                                    ^^^^^^
```
Did you mean "these"?

#### Section 3.3, paragraph 18
```
The parameters should be defined in an container named "parameters" augmenti
                                    ^^
```
Use "a" instead of "an" if the following word doesn't start with a vowel sound,
e.g. "a sentence", "a university".

#### Section 6.1, paragraph 3
```
be given access to monitor their services status (e.g. via model-driven tel
                                  ^^^^^^^^
```
An apostrophe may be missing.

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-opsawg-service-assurance-yang-09. If any part of this review is inaccurate, please let us know.

The IANA Functions Operator has a question about one of the actions requested in the IANA Considerations section of this document.

The IANA Functions Operator understands that, upon approval of this document, there are two actions which we must complete.

First, in the ns registry on the IETF XML Registry page located at:

https://www.iana.org/assignments/xml-registry/

three, new namespaces will be registered as follows:

ID: yang:ietf-service-assurance
URI: urn:ietf:params:xml:ns:yang:ietf-service-assurance
Filename: [ TBD-at-Registration ]
Reference: [ RFC-to-be ]

ID: yang:ietf-service-assurance-device
URI: urn:ietf:params:xml:ns:yang:ietf-service-assurance-device
Filename: [ TBD-at-Registration ]
Reference: [ RFC-to-be ]

ID: yang:ietf-service-assurance-interface
URI: urn:ietf:params:xml:ns:yang:ietf-service-assurance-interface
Filename: [ TBD-at-Registration ]
Reference: [ RFC-to-be ]

IANA Question --> Section 7.1 of the current draft says, "This document registers two URIs in the IETF XML registry [RFC3688]." Should IANA understand the word two as an error and that the number is three as listed in Section 7.1?

As this document requests registrations in an Expert Review or Specification Required (see RFC 8126) registry, we will initiate the required Expert Review via a separate request. Expert review will need to be completed before your document can be approved for publication as an RFC.

Second, in the YANG Module Names registry on the YANG Parameters registry page located at:

https://www.iana.org/assignments/yang-parameters/

three, new YANG modules will be registered as follows:

Name: ietf-service-assurance
File: [ TBD-at-Registration ]
Maintained by IANA? N
Namespace: urn:ietf:params:xml:ns:yang:ietf-service-assurance
Prefix: sain
Module:
Reference: [ RFC-to-be ]

Name: ietf-service-assurance-device
File: [ TBD-at-Registration ]
Maintained by IANA? N
Namespace: urn:ietf:params:xml:ns:yang:ietf-service-assurance-device
Prefix: sain-device
Module:
Reference: [ RFC-to-be ]

Name: ietf-service-assurance-interface
File: [ TBD-at-Registration ]
Maintained by IANA? N
Namespace: urn:ietf:params:xml:ns:yang:ietf-service-assurance-interface
Prefix: sain-interface
Module:
Reference: [ RFC-to-be ]

IANA Question --> While the YANG module name will be registered after the IESG approves the document, the YANG module file will be posted after the RFC Editor notifies us that the document has been published.

The IANA Functions Operator understands that these are the only actions required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

David Dong
IANA Services Specialist

The following Last Call announcement was sent out (ends 2022-11-22):

From: The IESG
To: IETF-Announce
CC: draft-ietf-opsawg-service-assurance-yang@ietf.org, mcr@sandelman.ca, opsawg-chairs@ietf.org, opsawg@ietf.org, rwilton@cisco.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (YANG Modules for Service Assurance) to Proposed Standard


The IESG has received a request from the Operations and Management Area
Working Group WG (opsawg) to consider the following document: - 'YANG Modules
for Service Assurance'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2022-11-22. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document specifies YANG modules for representing assurance
  graphs.  These graphs represent the assurance of a given service by
  decomposing it into atomic assurance elements called subservices.  A
  companion document, Service Assurance for Intent-based Networking
  Architecture, presents an architecture for implementing the assurance
  of such services.

  The YANG data models in this document conforms to the Network
  Management Datastore Architecture (NMDA) defined in RFC 8342.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-opsawg-service-assurance-yang/


The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/3859/

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

** A last minute review changed the document to have the health-score include
  the value -1, to indicate that the health score was not available.
  This allows the health-score to always be returned (mandatory), but to
  represent when values are missing for operational things where lack of
  data should not be confused with "zero" data.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

The draft-ietf-opsawg-service-assurance-yang-02 has some implementations.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The architectyre and YANG documents cross-reference each other, but are going
to the IESG at the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There are only YANG module registrations.

DISCUSSES: Rob Wilton.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

The draft-ietf-opsawg-service-assurance-yang-02 has some implementations.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

yes.

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The architectyre and YANG documents cross-reference each other, but are going
to the IESG at the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There are only YANG module registrations.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

One document an architecture document.
draft-ietf-opsawg-service-assurance-yang-02 is where the protocol and
implementations would be.  There seem to be enough parties interested, but
as yet, none have indicated that they have implemented.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes. I wonder if the two documents couldn't be published together as a single document.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

The DT does not say Informational for the architecture document (yet).

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section, but YANG module registrations.

Document Shepherd Write-Up for Group Documents:
  * draft-ietf-opsawg-service-assurance-architecture-08
  * draft-ietf-opsawg-service-assurance-yang-02

Document History
>Does the working group (WG) consensus represent the strong concurrence of a
>few individuals, with others being silent, or did it reach broad agreement?

The consensus representes the strong concurrence of a few individuals, with
most WG participants being silent.  Given the very niche expertise involved,
this is not surprising.

>Was there controversy about particular points, or were there decisions where
>the consensus was particularly rough?

There was very little controversy.

> Has anyone threatened an appeal or otherwise indicated extreme discontent? If
> so, please summarize the areas of conflict in separate email messages to the
> responsible Area Director. (It should be in a separate email because this
> questionnaire is publicly available.)

No.

> For protocol documents, are there existing implementations of the contents of
> the document? Have a significant number of potential implementers indicated
> plans to implement? Are any existing implementations reported somewhere,
> neither in the document itself (as RFC 7942 recommends) or elsewhere
> (where)?

One document an architecture document.
draft-ietf-opsawg-service-assurance-yang-02 is where the protocol and
implementations would be.  There seem to be enough parties interested, but
as yet, none have indicated that they have implemented.

>Additional Reviews
>Do the contents of this document closely interact with technologies in other
>IETF working groups or external organizations, and would it therefore benefit
>from their review? Have those reviews occurred? If yes, describe which
>reviews took place.

The document deals with YANG and services.  While the L2 services used as
examples are not strictly within the normal IETF waist, the provisioning
these services has been done in the IETF for some decades.

> Describe how the document meets any required formal expert review criteria,
> such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

A YANG Doctors review of draft-ietf-opsawg-service-assurance-yang-02 was done.

>If the document contains a YANG module, has the final version of the module
>been checked with any of the recommended validation tools for syntax and
>formatting validation? If there are any resulting errors or warnings, what is
>the justification for not fixing them at this time? Does the YANG module
>comply with the Network Management Datastore Architecture (NMDA) as specified
>in RFC 8342?

Yes: 0 errors, 0 warnings

>Describe reviews and automated checks performed to validate sections of the
>final version of the document written in a formal language, such as XML code,
>BNF rules, MIB definitions, CBOR's CDDL, etc.

>Document Shepherd Checks
>Based on the shepherd's review of the document, is it their opinion that this
>document is needed, clearly written, complete, correctly designed, and ready
>to be handed off to the responsible Area Director?

Yes. I wonder if the two documents couldn't be published together as a single document.

> Several IETF Areas have assembled lists of common issues that their
> reviewers encounter. For which areas have such issues been identified
> and addressed? For which does this still need to happen in subsequent
> reviews?

no.

> What type of RFC publication is being requested on the IETF stream (Best
> Current Practice, Proposed Standard, Internet Standard,
> Informational, Experimental or Historic)? Why is this the proper type
> of RFC? Do all Datatracker state attributes correctly reflect this intent?

The DT does not say Informational for the architecture document (yet).

> Have reasonable efforts been made to remind all authors of the intellectual
> property rights (IPR) disclosure obligations described in BCP 79? To
> the best of your knowledge, have all required disclosures been filed? If
> not, explain why. If yes, summarize any relevant discussion, including links
> to publicly-available messages when applicable.

yes.

> Has each author, editor, and contributor shown their willingness to be
> listed as such? If the total number of authors and editors on the front page
> is greater than five, please provide a justification.

Not a problem.

> Document any remaining I-D nits in this document. Simply running the idnits
> tool is not enough; please review the "Content Guidelines" on
> authors.ietf.org. (Also note that the current idnits tool generates
> some incorrect warnings; a rewrite is underway.)

There are no significant idnits.
}  -- Obsolete informational reference (is this intentional?): RFC 7895
}    (Obsoleted by RFC 8525)

> Should any informative references be normative or vice-versa? See the IESG
> Statement on Normative and Informative References.

The list seems correct.

> List any normative references that are not freely available to anyone. Did
> the community have sufficient access to review any such normative
> references?

None.

> Are there any normative downward references (see RFC 3967 and BCP
> 97) that are not already listed in the DOWNREF registry? If so,
> list them.

No.

> Are there normative references to documents that are not ready to be
> submitted to the IESG for publication or are otherwise in an unclear state?
> If so, what is the plan for their completion?

The two documents cross-reference each other, but are going to the IESG at
the same time.

> Will publication of this document change the status of any existing RFCs?

no.

> Describe the document shepherd's review of the IANA considerations section,

There isn't a significant section, but YANG module registrations.