Nominating Committee Eligibility
draft-ietf-elegy-rfc8989bis-05

[Ballot comment]
Thank you for the work on this document.

Many thanks to Scott Hollenbeck for his ART ART Review: https://mailarchive.ietf.org/arch/msg/art/wPl_szdPTb1WBf3Ov_IebyH4oUk/ and to the author for addressing Scott's comments.

[Ballot discuss]
Thanks for the solid work on this document.

One comment that I'd like to discuss, I believe the reference to ietf-shmoo-remote-fee should be normative, since this document makes reference to a long term commitment to free remote participation, and to my knowledge absent the shmoo-remote-fee document that isn't a commitment that is codified.

[Ballot comment]
"The two-per-organization limit in [RFC8713] complicates such an attack.  To circumvent it, an organization must either [...] (3) propose candidates with false affiliations."

It's not really clear to me what a "false affiliation" actually is -- in some other organizations, where voting is a thing, it is common for there to be lots of one or two person consulting companies. These consultants all have different affiliations, they just *happen* to have contracts with the same larger organization, and also just *happen* to all vote in the same way... Would these be false affiliations? Note that I don't really view this as major issue -- if we end up in a scenario where people are gaming this, then we've already lost.

As a nit: 'either' is generally used with 2 options.

Thanks to Dan Romascanu for the OpsDir review.

[Ballot comment]
Thank you to Vincent Roca for the SECDIR review.

** Section 1.  Editorial.

OLD
  The actual NomCom is selected at random
  from the pool of eligible volunteers.

NEW
Appointment to the NomCom is selected at random from the pool of eligible volunteers.

** Section 1.

(2) author or editor of an IETF Stream RFC in the past
  five years, including internet-drafts in the RFC Editor queue

Section 4 of RFC8989 actually says “the person has been a listed author or editor (on the front page) of at least two IETF Stream RFCs within the last 5 years prior to the day the call for NomCom volunteers is sent to the community.”  The text in this document suggests that a single RFC constitutes eligibility, but RFC8989 says it is at least two.
** Section 2.  Considering mentioning that attending an the IETF F2F meeting used to be considered an essential part of participation in the community
** Section 2.
  Two days of travel and an attendance fee is a relatively large
  expenditure of time and money

I don’t understand the basis of the “two days of travel” metric.
** Section 2.
  However, attitudes to business travel evolve, and remote meeting
  technology continues to improve, to the extent that many longstanding
  community members choose to participate remotely. 

Do we have conclusive data on _many_ “longstanding members now participating remotely”?  Can we cite this and define what is “many” and “longstanding”?
** Section 2.

  Further, the NomCom has
  completed two cycles using entirely online tools.

-- What is meant by “entirely online tools”?
-- Please cite the NomCom years in question.
** Section 2.
  Finally, overly restrictive criteria work against getting a broad
  talent pool.

Recommend against the phrase “talent pool”.  To me it suggestions some set of skill or qualifications, which is not the diversity which the current NomCom process is engineering.
** Section 3.
In-person
  attendance is as determined by the record keeping of the Secretariat.    Online attendance is based on being a registered person who logged in
  for at least one session of an IETF meeting.


-- Section 1 has already said the in-person criteria amounts to “... the volunteer picked up their registration badge at an in-person meeting”, why not explicitly say that?

-- Per the online attendance, is that also “determined by the records keeping of the Secretariat”, as in, they get the logs from Meetecho?
** Section 4.1.1.
  A sudden surge in the number of volunteers, particularly of people
  that no one recognizes as a part of the community, is an early-
  warning sign for the community, leadership and the IETF Secretariat
  to further investigate.

What is the recourse of this behavior is discovered?

Shepherd report for draft-ietf-elegy-rfc8989bis

Document History
Does the working group (WG) consensus represent the strong concurrence of a
few individuals, with others being silent, or did it reach broad agreement?
- Broad agreement.

Was there controversy about particular points, or were there decisions where
the consensus was particularly rough?
- No.

Has anyone threatened an appeal or otherwise indicated extreme discontent? If
so, please summarize the areas of conflict in separate email messages to the
responsible Area Director. (It should be in a separate email because this
questionnaire is publicly available.)
- No.

For protocol documents, are there existing implementations of the contents of
the document? Have a significant number of potential implementers indicated
plans to implement? Are any existing implementations reported somewhere,
either in the document itself (as RFC 7942 recommends) or elsewhere
(where)?
- n/a

Additional Reviews
Do the contents of this document closely interact with technologies in other
IETF working groups or external organizations, and would it therefore benefit
from their review? Have those reviews occurred? If yes, describe which
reviews took place.
- No.

Describe how the document meets any required formal expert review criteria,
such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
- n/a

If the document contains a YANG module, has the final version of the module
been checked with any of the recommended validation tools for syntax and
formatting validation? If there are any resulting errors or warnings, what is
the justification for not fixing them at this time? Does the YANG module
comply with the Network Management Datastore Architecture (NMDA) as specified in RFC 8342?
- n/a

Describe reviews and automated checks performed to validate sections of the
final version of the document written in a formal language, such as XML code,
BNF rules, MIB definitions, CBOR's CDDL, etc.
- n/a

Document Shepherd Checks
Based on the shepherd's review of the document, is it their opinion that this
document is needed, clearly written, complete, correctly designed, and ready
to be handed off to the responsible Area Director?
- Yes.

Several IETF Areas have assembled lists of common issues that their
reviewers encounter. For which areas have such issues been identified
and addressed? For which does this still need to happen in subsequent
reviews?
- n/a

What type of RFC publication is being requested on the IETF stream (Best
Current Practice, Proposed Standard, Internet Standard,
Informational, Experimental or Historic)? Why is this the proper type
of RFC? Do all Datatracker state attributes correctly reflect this intent?
- BCP.  This document updates RFC 8713 and will become part of BCP 10.

Have reasonable efforts been made to remind all authors of the intellectual
property rights (IPR) disclosure obligations described in BCP 79? To
the best of your knowledge, have all required disclosures been filed? If
not, explain why. If yes, summarize any relevant discussion, including links
to publicly-available messages when applicable.
- Yes, and the document complies with BCP 79 and there is no relation between this document any any known IPR.

Has each author, editor, and contributor shown their willingness to be
listed as such? If the total number of authors and editors on the front page
is greater than five, please provide a justification.
- Yes.

Document any remaining I-D nits in this document. Simply running the idnits
tool is not enough; please review the "Content Guidelines" on
authors.ietf.org. (Also note that the current idnits tool generates
some incorrect warnings; a rewrite is underway.)
- None.

Should any informative references be normative or vice-versa? See the IESG
Statement on Normative and Informative References.
- No.

List any normative references that are not freely available to anyone. Did
the community have sufficient access to review any such normative
references?
- n/a

Are there any normative downward references (see RFC 3967 and BCP
97) that are not already listed in the DOWNREF registry? If so,
list them.
- No.

Are there normative references to documents that are not ready to be
submitted to the IESG for publication or are otherwise in an unclear state?
If so, what is the plan for their completion?
- No.

Will publication of this document change the status of any existing RFCs? If
so, does the Datatracker metadata correctly reflect this and are those RFCs
listed on the title page, in the abstract, and discussed in the
introduction? If not, explain why and point to the part of the document
where the relationship of this document to these other RFCs is discussed.
- Yes.  See above.

Describe the document shepherd's review of the IANA considerations section,
especially with regard to its consistency with the body of the document.
Confirm that all aspects of the document requiring IANA assignments are
associated with the appropriate reservations in IANA registries. Confirm
that any referenced IANA registries have been clearly identified. Confirm
that each newly created IANA registry specifies its initial contents,
allocations procedures, and a reasonable name (see RFC 8126).
- What sort of “description” are you looking for?  There are no IANA actions required, and the document says so.

List any new IANA registries that require Designated Expert Review for
future allocations. Are the instructions to the Designated Expert clear?
Please include suggestions of designated experts, if appropriate.
- n/a

[Ballot comment]
Hi Martin,

Thanks for documenting this - it needed to be done.  One comment/concern inline below ...

(1) p 5, sec 4.1.3.  One Year of Participation

  Attendance at three meetings requires at least eight months of
  waiting.  Given the volume of volunteers necessary to capture the
  process, an attack requires a surge in attendees over the course of a
  year.  Such a surge might trigger a community challenge to the list
  of eligible volunteers, and/or a leadership investigation to detect
  suspicious behavior (e.g., logging in to a single session and then
  immediately logging out).  In the event of abuse of process, the
  leadership would then have months to adjust policy in response before
  the NomCom cycle begins.

For section 4, I think that it would be helpful to identify which leadership body is responsible for adjusting policy, and also have clear text that empowers the leadership body to act to mitigate any abuse of the NomCom selection process, if needed.  I'm presuming this the leadership body here would be the IESG, backed by the IAB (in that any IESG decision is appealable to the IAB).  The reason that I believe that it is helpful to indicate the the IESG is obliged to act is because it may be impossible for the IESG to adjust policy through the normal IETF consensus process (although they should try) within the required timeframe - a sustained attack on the IETF processes would likely slow, or make it impossible, for an AD to call consensus, and if under attack, any decision would likely be appealed regardless, further slowing the process.  In this scenario, I think that it might be necessary for the IESG to act in the best interests of the IETF, but outside of the formal consensus process (e.g., perhaps via an IESG statement), which of course also also be subject to the normal appeals process if needed.

Regards,
Rob

[Ballot comment]
Thank you Martin for the work done in the document.

I strongly support it, but Murray's comments should be taken into account.

Regards

-éric

[Ballot comment]
Thanks to Scott Hollenbeck for his ARTART review.

From Section 4.1.1:

"A sudden surge in the number of volunteers, particularly of people that no one recognizes as a part of the community, is an early-warning sign for the community, leadership and the IETF Secretariat to further investigate. The community should monitor and assess a sudden increase in the number of online registration fee waivers awarded [...]"

This seems vague to me.  It's curious that this is non-specific about what is meant by "leadership", for example.  Is the IESG tasked with this monitoring?  Is it ISOC, who appoints the NomCom Chair?  The tools team?  Is the community at large supposed to remember to do it?  What are the limits of such an investigation, or of the corrections it may impose?

From Section 4.1.3:

"In the event of abuse of process, the leadership would then have months to adjust policy in response before the NomCom cycle begins."

That's presuming it notices in time.  How late is too late?

I wonder if it should be made more explicit about exactly who should be checking for what, and when.

[Ballot comment]
Thanks to Scott Hollenbeck for his ARTART review.

From Section 4.1.1:

"A sudden surge in the number of volunteers, particularly of people that no one recognizes as a part of the community, is an early-warning sign for the community, leadership and the IETF Secretariat to further investigate. The community should monitor and assess a sudden increase in the number of online registration fee waivers awarded [...]"

This seems vague to me.  It's curious that this is non-specific about what is meant by "leadership" or even "the community".  Is the IESG tasked with this monitoring?  The tools team?  Is the community at large supposed to remember to do it?  What are the limits of such an investigation, or of the corrections it may impose?

From Section 4.1.3:

"In the event of abuse of process, the leadership would then have months to adjust policy in response before the NomCom cycle begins."

That's presuming it notices in time.  How late is too late?

I wonder if it should be made more explicit exactly who should be checking for what, and when.

[Ballot comment]
From Section 4.1.1:

"A sudden surge in the number of volunteers, particularly of people that no one recognizes as a part of the community, is an early-warning sign for the community, leadership and the IETF Secretariat to further investigate. The community should monitor and assess a sudden increase in the number of online registration fee waivers awarded [...]"

Suppose such an unusual condition occurs.  Which body has the authority to pause the NomCom's process in order to investigate, and for how long, and what remedies are they empowered to enact?  I recognize the temptation to leave this unspecified to allow for flexibility, but that feels like it might be a double-edged sword.

From Section 4.1.3:

"In the event of abuse of process, the leadership would then have months to adjust policy in response before the NomCom cycle begins."

Only if it notices in time.  I wonder if it should be made more explicit who should be checking for what, exactly, and when.

[Ballot comment]
Section 4.1.1

        A sudden surge in the number of volunteers, particularly of
        people that no one recognizes as a part of the community, is an
        early-warning sign for the community, leadership and the IETF
        Secretariat to further investigate.

This 4.1 subsection is the only one that doesn't list the counter-move.
Maybe add to say something like:

        to further investigate and where needed take action as defined in
        RFC 8713 Section 3.7.3 to invalidate such candidates.

Section 4.1.3

Why is the counter-move here "to adjust policy in response" instead of simply
rejecting those candidates as per RFC 8713 Section 3.7.3 ?

[Ballot comment]
The Shepherd writeup indicates that this document will become part of BCP10.  Good.

What about rfc8788?  It only applied to the 2020-2021 nomcom and is also part of BCP 10.  Therefore, this document should also Obsolete rfc8788 "to make it clear that that document has been superseded" (the same reason that it Obsoletes RFC8989).

[Ballot comment]
# Internet AD comments for draft-ietf-elegy-rfc8989bis-04
CC @ekline

## Nits

### S4.2

* "Note that the counting remote participation" ->
  "Note that the counting of remote participation" or perhaps
  "Note that counting remote participation"

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-elegy-rfc8989bis-03, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

David Dong
IANA Services Specialist

Request for Last Call review by TSVART Completed: On the Right Track. Reviewer: Bernard Aboba. Sent review to list. Submission of review completed at an earlier date.

The following Last Call announcement was sent out (ends 2023-01-23):

From: The IESG
To: IETF-Announce
CC: barryleiba@computer.org, draft-ietf-elegy-rfc8989bis@ietf.org, elegy-chairs@ietf.org, eligibility-discuss@ietf.org, lars@eggert.org
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Nominating Committee Eligibility) to Best Current Practice


The IESG has received a request from the NomCom Eligibility Update WG (elegy)
to consider the following document: - 'Nominating Committee Eligibility'
  as Best Current Practice

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2023-01-23. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  The IETF Nominating Committee (NomCom) appoints candidates to several
  IETF leadership committees.  RFC8713 provides criteria for NomCom
  membership that attempt to ensure that NomCom volunteers are members
  of the loosely defined IETF community, by requiring in-person
  attendance in three of the past five in- person meetings.  In 2020
  and 2021, the IETF had six consecutive fully online plenary meetings
  that drove rapid advancement in remote meeting technologies and
  procedures, including an experiment that included remote attendance
  for NomCom eligibility.  This document updates RFC8713 by defining a
  new set of eligibility criteria from first principles, with
  consideration to the increased salience of remote attendance.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-elegy-rfc8989bis/



No IPR declarations have been submitted directly on this I-D.

Shepherd report for draft-ietf-elegy-rfc8989bis

Document History
Does the working group (WG) consensus represent the strong concurrence of a
few individuals, with others being silent, or did it reach broad agreement?
- Broad agreement.

Was there controversy about particular points, or were there decisions where
the consensus was particularly rough?
- No.

Has anyone threatened an appeal or otherwise indicated extreme discontent? If
so, please summarize the areas of conflict in separate email messages to the
responsible Area Director. (It should be in a separate email because this
questionnaire is publicly available.)
- No.

For protocol documents, are there existing implementations of the contents of
the document? Have a significant number of potential implementers indicated
plans to implement? Are any existing implementations reported somewhere,
either in the document itself (as RFC 7942 recommends) or elsewhere
(where)?
- n/a

Additional Reviews
Do the contents of this document closely interact with technologies in other
IETF working groups or external organizations, and would it therefore benefit
from their review? Have those reviews occurred? If yes, describe which
reviews took place.
- No.

Describe how the document meets any required formal expert review criteria,
such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
- n/a

If the document contains a YANG module, has the final version of the module
been checked with any of the recommended validation tools for syntax and
formatting validation? If there are any resulting errors or warnings, what is
the justification for not fixing them at this time? Does the YANG module
comply with the Network Management Datastore Architecture (NMDA) as specified in RFC 8342?
- n/a

Describe reviews and automated checks performed to validate sections of the
final version of the document written in a formal language, such as XML code,
BNF rules, MIB definitions, CBOR's CDDL, etc.
- n/a

Document Shepherd Checks
Based on the shepherd's review of the document, is it their opinion that this
document is needed, clearly written, complete, correctly designed, and ready
to be handed off to the responsible Area Director?
- Yes.

Several IETF Areas have assembled lists of common issues that their
reviewers encounter. For which areas have such issues been identified
and addressed? For which does this still need to happen in subsequent
reviews?
- n/a

What type of RFC publication is being requested on the IETF stream (Best
Current Practice, Proposed Standard, Internet Standard,
Informational, Experimental or Historic)? Why is this the proper type
of RFC? Do all Datatracker state attributes correctly reflect this intent?
- BCP.  This document updates RFC 8713 and will become part of BCP 10.

Have reasonable efforts been made to remind all authors of the intellectual
property rights (IPR) disclosure obligations described in BCP 79? To
the best of your knowledge, have all required disclosures been filed? If
not, explain why. If yes, summarize any relevant discussion, including links
to publicly-available messages when applicable.
- This question is bizarrely worded.  In any case, the document complies with BCP 79 and there is no relation between this document any any known IPR.

Has each author, editor, and contributor shown their willingness to be
listed as such? If the total number of authors and editors on the front page
is greater than five, please provide a justification.
- Yes.

Document any remaining I-D nits in this document. Simply running the idnits
tool is not enough; please review the "Content Guidelines" on
authors.ietf.org. (Also note that the current idnits tool generates
some incorrect warnings; a rewrite is underway.)
- None.

Should any informative references be normative or vice-versa? See the IESG
Statement on Normative and Informative References.
- No.

List any normative references that are not freely available to anyone. Did
the community have sufficient access to review any such normative
references?
- n/a

Are there any normative downward references (see RFC 3967 and BCP
97) that are not already listed in the DOWNREF registry? If so,
list them.
- No.

Are there normative references to documents that are not ready to be
submitted to the IESG for publication or are otherwise in an unclear state?
If so, what is the plan for their completion?
- No.

Will publication of this document change the status of any existing RFCs? If
so, does the Datatracker metadata correctly reflect this and are those RFCs
listed on the title page, in the abstract, and discussed in the
introduction? If not, explain why and point to the part of the document
where the relationship of this document to these other RFCs is discussed.
- Yes.  See above.

Describe the document shepherd's review of the IANA considerations section,
especially with regard to its consistency with the body of the document.
Confirm that all aspects of the document requiring IANA assignments are
associated with the appropriate reservations in IANA registries. Confirm
that any referenced IANA registries have been clearly identified. Confirm
that each newly created IANA registry specifies its initial contents,
allocations procedures, and a reasonable name (see RFC 8126).
- What sort of “description” are you looking for?  There are no IANA actions required, and the document says so.

List any new IANA registries that require Designated Expert Review for
future allocations. Are the instructions to the Designated Expert clear?
Please include suggestions of designated experts, if appropriate.
- n/a

Shepherd report for draft-ietf-elegy-rfc8989bis

Document History
Does the working group (WG) consensus represent the strong concurrence of a
few individuals, with others being silent, or did it reach broad agreement?
- Broad agreement.

Was there controversy about particular points, or were there decisions where
the consensus was particularly rough?
- No.

Has anyone threatened an appeal or otherwise indicated extreme discontent? If
so, please summarize the areas of conflict in separate email messages to the
responsible Area Director. (It should be in a separate email because this
questionnaire is publicly available.)
- No.

For protocol documents, are there existing implementations of the contents of
the document? Have a significant number of potential implementers indicated
plans to implement? Are any existing implementations reported somewhere,
either in the document itself (as RFC 7942 recommends) or elsewhere
(where)?
- n/a

Additional Reviews
Do the contents of this document closely interact with technologies in other
IETF working groups or external organizations, and would it therefore benefit
from their review? Have those reviews occurred? If yes, describe which
reviews took place.
- No.

Describe how the document meets any required formal expert review criteria,
such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.
- n/a

If the document contains a YANG module, has the final version of the module
been checked with any of the recommended validation tools for syntax and
formatting validation? If there are any resulting errors or warnings, what is
the justification for not fixing them at this time? Does the YANG module
comply with the Network Management Datastore Architecture (NMDA) as specified in RFC 8342?
- n/a

Describe reviews and automated checks performed to validate sections of the
final version of the document written in a formal language, such as XML code,
BNF rules, MIB definitions, CBOR's CDDL, etc.
- n/a

Document Shepherd Checks
Based on the shepherd's review of the document, is it their opinion that this
document is needed, clearly written, complete, correctly designed, and ready
to be handed off to the responsible Area Director?
- Yes.

Several IETF Areas have assembled lists of common issues that their
reviewers encounter. For which areas have such issues been identified
and addressed? For which does this still need to happen in subsequent
reviews?
- n/a

What type of RFC publication is being requested on the IETF stream (Best
Current Practice, Proposed Standard, Internet Standard,
Informational, Experimental or Historic)? Why is this the proper type
of RFC? Do all Datatracker state attributes correctly reflect this intent?
- BCP.  This document updates RFC 8713 and will become part of BCP 10.

Have reasonable efforts been made to remind all authors of the intellectual
property rights (IPR) disclosure obligations described in BCP 79? To
the best of your knowledge, have all required disclosures been filed? If
not, explain why. If yes, summarize any relevant discussion, including links
to publicly-available messages when applicable.
- This question is bizarrely worded.  In any case, the document complies with BCP 79 and there is no relation between this document any any known IPR.

Has each author, editor, and contributor shown their willingness to be
listed as such? If the total number of authors and editors on the front page
is greater than five, please provide a justification.
- Yes.

Document any remaining I-D nits in this document. Simply running the idnits
tool is not enough; please review the "Content Guidelines" on
authors.ietf.org. (Also note that the current idnits tool generates
some incorrect warnings; a rewrite is underway.)
- None.

Should any informative references be normative or vice-versa? See the IESG
Statement on Normative and Informative References.
- No.

List any normative references that are not freely available to anyone. Did
the community have sufficient access to review any such normative
references?
- n/a

Are there any normative downward references (see RFC 3967 and BCP
97) that are not already listed in the DOWNREF registry? If so,
list them.
- No.

Are there normative references to documents that are not ready to be
submitted to the IESG for publication or are otherwise in an unclear state?
If so, what is the plan for their completion?
- No.

Will publication of this document change the status of any existing RFCs? If
so, does the Datatracker metadata correctly reflect this and are those RFCs
listed on the title page, in the abstract, and discussed in the
introduction? If not, explain why and point to the part of the document
where the relationship of this document to these other RFCs is discussed.
- Yes.  See above.

Describe the document shepherd's review of the IANA considerations section,
especially with regard to its consistency with the body of the document.
Confirm that all aspects of the document requiring IANA assignments are
associated with the appropriate reservations in IANA registries. Confirm
that any referenced IANA registries have been clearly identified. Confirm
that each newly created IANA registry specifies its initial contents,
allocations procedures, and a reasonable name (see RFC 8126).
- What sort of “description” are you looking for?  There are no IANA actions required, and the document says so.

List any new IANA registries that require Designated Expert Review for
future allocations. Are the instructions to the Designated Expert clear?
Please include suggestions of designated experts, if appropriate.
- n/a