Process Experiment: Document Shepherding Throughout a Document's Lifecycle
draft-leiba-extended-doc-shepherd-03

Summary: Needs a YES.

(Ron Bonica) Discuss

Discuss (2012-11-13 for -01)
Updating my DISCUSS, having realized that this document is an individual submission, not on the independent stream.....

The document  "gives one Area Director's  view of how that extended shepherding function might work, and what  tasks might be involved throughout the document's lifecycle." Insomuch as its scope is limited to "one Area Director's view", its value is somewhat limited.

Also, I wonder what it means to have IETF consensus on "one area director's view". Does it mean that the community agrees with that view? Or that the community believes that the AD actually holds that view?

(Gonzalo Camarillo) Discuss

Discuss (2012-11-15 for -01)
As it has been pointed out during the discussions about this draft, it is unusual and probably problematic to publish the opinion of a single AD about our processes as an Informational RFC. If we recast it as a process experiment, we will need to agree on the type of consensus we will seek to run such an experiment.

(Adrian Farrel) Discuss

Discuss (2012-11-05 for -00)
I am updating my Discuss having debated the draft with the author and some other folk around the IETF.

I do not believe there is value to the IETF in publishing RFCs that document individual people's opinions or thoughts about process. I think this would be the thin edge of a very long wedge.

I understand that publishing documents about the IETF process on the Independent Stream is not a good idea.

I understand that the author is expressing his opinions and so is not interested in IETF consensus (although obviously interested in discussion).

I can see that issuing an IETF last call effectively worked as a "call for comments".

However, I understand that the ideas in this document may well prove to be practical and useful. Therefore, I think that this I-D should be turned into a process experiment per RFC 3933. This would not require a significant change to the document, and would make it possible for ADs/WGs to make use of the ideas in this document. Furthermore, it would give the I-D a hook to hang on, make IETF last call and consensus meaningful, and make publishing an RFC relevant.

(Stephen Farrell) Discuss

Discuss (2012-11-12 for -01)

(1) I don't know where we are on treating this as a process
experiment, but the document doesn't describe any process
experiment and is still written as a set of personal
reflections. I think it really must be revised before being
approved so that it does describe a process experiment.  In
fact, I think the text changes required will be significant
enough that this ought not be on a telechat agenda until
those are done. I agree with Adrian that if presented as
something that is not a process experiment, this would be
equally or more problematic.

(2) abstract: "It seems reasonable and helpful to begin
shepherding when there's a call for adoption as a working
group document..." I disagree. To me that seems like more
process. We have more than enough process already. Adding
more process, as is done here, seems neither reasonable nor
helpful. I would therefore prefer this not be published.  I
accept that the goal is process improvement, but am of the
opinion that the result would be more process stagnation. I
believe my opinion is as grounded in fact as the document
itself. Which leads to the discuss: where are the facts upon
which this (suggested) process addition is based?  If there
are none, why are we adding more process with no evidence to
back up the suggested added process burden?

(3) s3.1, there is no need for chairs to decide who is
responsible for a document unless or until they want to - why
create this new "Responsible Chair" role as a seemingly
IETF-wide consruct when its not? If there is no need then the
rest of the document seems broken as it depends on this role.
I'm also not sure we ought assume chairs divide resposibility
in that way.

(4) Allowing a shepherd to issue and decide a call for
adoption is a mistake IMO and one that would damage our
current processes by confusing participants as to who makes
consensus calls and changing who serves at whose pleasure.
Saying this "needs" to be done is just a bad plan IMO. (I
note that 3.1, step 4 is also recursive as 3.1 is also termed
"call for adoption" so this doesn't even seem sufficiently
worked out.) This continues in later sections, where chair
roles during WG processing are assigned to shepherds. I think
all of that is a bad plan.
Comment (2012-11-12 for -01)
No email
send info

- s1, 2nd para: do we have any evidence that documents
"often" languish unintentionally? 

- s1, 3rd para: our goal is not to move forward in the
process, but to have a process that allows improvement of the
Internet. Equating the former with the latter is an error.

- s1, 5th para: providing summaries can make it more likely
people try to slavishly follow those and impose process where
none is needed, and then repeat that a few times before a
document is done. Is that really a good approach?

- s2, last para: this seems to call for a chair to re-do all
the work a non-chair shepherd has done, i.e. it encourages
more pointless process. 

- s3, codifying 12 steps seems undesirable to me, sometimes a
nice tidy document might do steps 1-5 when along comes an
upstart way better solution. 

- s3.1, the chairs can well forget to record the shepherd's
name and nobody will lose out - in almost all cases the
shepherd is not needed until *much* later. This "should not"
is a good example of the kind of process-creep evident 
throughout and demonstrated via the use of imperatives.

- I stopped with the detailed comments at this point. I think
my detailed comments thereafter are predictable other than
those below. In the main, imperatives need to be avoided.

- 3.6, IETF LC isn't mandatory for all documents is it? I
know we do it almost always nowadays, but I don't think you
can assume that that practice will remain for ever.

(Martin Stiemerling) Discuss

Discuss (2012-11-14 for -01)
I wonder if we need to discuss any process experiment before we even have discussed if such a document is needed at all.  Why should we introduce a new process if we do not see any urgent need for this in the community?

The draft describes a single person's opinion. The abstract and the intro is confusing, as it says it is an AD's view of the process. However, this is rather a personal opinion of an IETF community member, who is also AD. 

I could see this draft turned into a slide set for the EDU teams for Sunday, explaining the document run through the whole process.

Further:
Section 1., paragraph 4:

>    This document suggests specific tasks a Working Group Chair should be
>    doing or delegating in order to maintain forward progress,
>    accountability, and quality control on a working group document.  It
>    adds to what's in RFC 4858, intending to extend it, not replace it.
>    Major extensions involve assigning a Shepherd and defining specific
>    tasks earlier in a document's life, and possibly delegating Document
>    Shepherd tasks to a Shepherd who is neither a Chair nor the Working
>    Group Secretary (consistent with the IESG Statement on Document
>    Shepherds [Stmt]).

  I assume that a WG is badly screwed if we need already a document
  shepherd right in the beginning. We have the WG chairs who are
  responsible for WG as whole and the WG on its own. 

  If the chairs are overloaded, they can ask for a WG secretary
  (Section 6.2 in RFC 2418).

(Russ Housley) Yes

(Benoît Claise) No Objection

Comment (2012-11-15 for -01)
No email
send info
I support Adrian's DISCUSS, and the solution to go for the process experiment per RFC 3933.

I see potentially a couple of advantages to this new process (delegation, potentially involve junior IETF people, potentially faster document deliveries with active management). However, while reading the document, I've been wondering: which problem does the new process try to solve? This should be explained better.

Right now, the draft reads

  This document proposes an extended set of Document Shepherd tasks,
   well beyond those covered in RFC 4858.

So, somehow, this is an update from RFC4848, but actually, not quite, because this is one AD's opinion.
I read RFC4848, and I believe that the current Document Shepherds don't do half of what's in there.
That's maybe part of the problem to be fixed, before asking them to do more.

(Ralph Droms) No Objection

Comment (2012-11-14 for -01)
No email
send info
I would like to see this document go forward
as a process experiment, perhaps preceded by
some review as to the actual demand for the
process improvements.

(Wesley Eddy) No Objection

Comment (2012-11-12 for -01)
No email
send info
After publication, errata may collect and go unprocessed for quite some time (years in many cases) ... should the shepherd be encouraged to help make sure this doesn't happen, somehow?

I tend to agree with Adrian's DISCUSS comments.

(Pete Resnick) No Objection

Comment (2012-11-13 for -01)
No email
send info
I tend to agree with the DISCUSSes; I think this would be better done as a process experiment rather than just being posted as the thoughts of a single AD on IETF process.

(Sean Turner) No Objection

Comment (2012-11-14 for -01)
No email
send info
I very strongly agree with Stephen's and Adrian's discuss positions.

I know these are your thoughts I figured I'd sprinkle some of mine in and some edits:

I have a feeling (with no numbers to back it up) that there is only a small number of drafts for which shepherding right from the beginning would be reasonable.

a/s1: r/Area Director's/person's
s1: r/been the Responsible Chair/shepherded

s1, para 2: From my experience, things languish in WGs most often based on lack of energy either from the author or the WG.  People were gung-ho to adopt it and then they get distracted or loose interest.  I can't recall a time where the WG consciously said let's work on X first and then get on to Y - but maybe that's just me.

s1, para 3: r/We/I

s1, para 3: If drafts are actually stuck then I can see it, but energized authors can get through this, I think, more effectively than a shepherd.  Because of that I don't know that we need to assign somebody.  The authors need to know to throw a flag and say I think we're stuck here - and that's going to the chairs and if they're non-responsive the ADs.  What I often see though is that people say hey my draft is stuck because nobody is commenting on it.  Not sure how to make more people comment on drafts - usually that means nobody is interested.

s1, para 4: If we're doing this as a process experiment this needs to be fixed.

s3: add "0. Submit individual draft"  Few drafts start straight out in the WG and this would be a good place to point out the replaced by information the secretariat and we like to know.

s3.1: Responsible Chair for a draft?  I never did and I just can't see the WG chairs I have actually doing that.  Is there a tool change that allows this that I don't know about? :)

s3.2: It's hard to keep discussion going when there's often little energy or interest to get it done.  Not every WG draft need be published - I think that point needs to be made.  Some judgement needs to injected in to keeping it going or in to letting it die.

s3.2: I think you mean to say when the Shepherd is the chair they issue the WGLC or is this a request for a WGLC?:

  The Shepherd can take
  the responsibility of either asking for a WGLC (when the Shepherd
  isn't the chair) or issuing a WGLC (when the Shepherd is a chair)
  and of analyzing comments
  to determine whether things are ready to go ahead.

s3.3: Are you asking for early reviews of every draft from every directorate? If we're going to run this as a process experiment, I think we ought to let everybody know a little more explicitly - i.e., send 'em a message that says hey we're about to flood you with early reviews.

s3.3: I just can't see the Shepherd starting the write-up until right after the chair decides that they might send it to the AD.

s3.3: r/   1.  Issue an official "Working Group Last Call" message on the list,
       with a reasonable deadline given./
   / 1.  Chair: Issue an official "Working Group Last Call" message on the list,
       with a reasonable deadline given.

s3.4: I disagree that the y/n response provide "responses are of little or no use" - many of the y/n questions related to discuss criteria.  Regardless of how detailed the write-up is, I'm sure the IESG will revisit any point it wants to.

s3.4: r/IESG Secretary/IESG Secretary (iesg-secretary@ietf.org).

s3.4: should you say the document state changes to "Publication Requested"

s3.12: r/We're/You're

s5: There are DoS attacks based on the size of Shepherd write-up ;)

(Stewart Bryant) Abstain

Comment (2012-11-15 for -01)
No email
send info
I am torn between a Discuss and an Abstain. 

Apart from the fundamentals of the approach that Adrian calls up, this could be considered as a de facto written standing order to APPS Area WG Chairs, and I do not think we should be issuing such written instructions other than though the normal consensus process for IETF process change.

(Robert Sparks) Abstain

Comment (2012-11-14 for -01)
No email
send info
Given the quality of the conversations resulting from the drafts initial 
submission, reviews, and last call, I don't see value in publishing the
document in its current form as an RFC.

The discussion has focused mostly on whether and how to capture things _like_ 
this, rather than what's actually being said. That's natural, given how the draft and the last call were formulated. 
That said, the discussion didn'tclearly produce good ways for us to capture an individual's opinion, and there
have been good arguments made that this particular way could be harmful.
The proposals to reformulate this as a process experiment seem to point to a better
path (but I'm still skeptical it is the best path).

I'm glad we tried this approach. I don't think we should try it again. I think
we would be better served as a community to push towards community agreement on
the _substance_ of this draft, and continue to look for ways to make it easy to
make such a large bite of one person's insights accessible for that discussion.

I agree with much of the content of the draft. There are a few places where I 
would have said things differently (particularly to avoid giving the impression
that "we" are encouraging all working groups to do things exactly the same way
rather than doing what's right for the situation the working group is in. 

So, with respect to the effort that's gone into this so far, and hope that we
continue to work to improve the shepherding system, incorporating many of these
ideas. I will abstain.

Barry Leiba Recuse

Ignas Bagdonas No Record

Deborah Brungard No Record

Alissa Cooper No Record

Roman Danyliw No Record

Benjamin Kaduk No Record

Suresh Krishnan No Record

Warren Kumari No Record

Mirja Kühlewind No Record

Alexey Melnikov No Record

Alvaro Retana No Record

Adam Roach No Record

Martin Vigoureux No Record

Éric Vyncke No Record

Magnus Westerlund No Record