Problem Statement E. Davies (ed.)
Internet-Draft Nortel Networks
Expires: August 25, 2003 February 24, 2003
IETF Problem Statement
draft-ietf-problem-issue-statement-00.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 25, 2003.
Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract
This memo summarizes perceived problems in the structure, function
and processes of the Internet Engineering Task Force (IETF). We are
attempting to identify these problems, so that they can be addressed
and corrected by the IETF community.
The problems have been digested and categorized from an extensive
discussion which took place on the 'problem-statement' mailing list
from November 2002 to February 2003. The problem list has been
further analyzed by an editorial team, and is provided as input to
the Problem Statement working group.
Davies (ed.) Expires August 25, 2003 [Page 1]
Internet-Draft IETF Problem Statement February 2003
Table of Contents
1. Introduction: Issues/Problems in the IETF process . . . . . 3
1.1 Consequences of Past Growth . . . . . . . . . . . . . . . . 3
1.2 The Aim is Improvement, not Finger-pointing . . . . . . . . 4
1.3 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 4
2. Root Cause Problems . . . . . . . . . . . . . . . . . . . . 6
2.1 The IETF does not have a common understanding of its
Mission . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 The IETF does not use Effective Engineering Practices . . . 7
2.3 IETF contributors appear to be less engaged than in
earlier days . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Authority and Influence in the IETF are concentrated in
too few hands . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 IETF Decision making processes are flawed . . . . . . . . . 10
2.6 IETF Participants and Leaders are inadequately trained . . . 10
3. Security Considerations . . . . . . . . . . . . . . . . . . 11
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . 12
References . . . . . . . . . . . . . . . . . . . . . . . . . 13
Author's Address . . . . . . . . . . . . . . . . . . . . . . 13
A. Summarization of Problems from Mailing List . . . . . . . . 14
A.1 Problem area/subject matter issues . . . . . . . . . . . . . 14
A.2 Working Group process issues . . . . . . . . . . . . . . . . 14
A.2.1 Who the participants are and represent . . . . . . . . . . . 15
A.2.2 How the consensus process works . . . . . . . . . . . . . . 16
A.2.3 How the interaction works . . . . . . . . . . . . . . . . . 19
A.2.4 Measuring the outcome . . . . . . . . . . . . . . . . . . . 20
A.2.5 Lack of Transparency: Design Teams and Interim meetings . . 21
A.2.6 Problems in the Large - Cross fertilization between
WGs/Areas and System level Thinking . . . . . . . . . . . . 21
A.2.7 Quality control issues - Documents . . . . . . . . . . . . . 22
A.2.8 Quality control issues - Personnel . . . . . . . . . . . . . 24
A.2.9 Timing/Latency . . . . . . . . . . . . . . . . . . . . . . . 25
A.3 IETF organizational and Perception issues . . . . . . . . . 27
A.3.1 The Old Boy network . . . . . . . . . . . . . . . . . . . . 27
A.3.2 External Perception of the IETF . . . . . . . . . . . . . . 28
A.3.3 Loose control of IETF over its own work . . . . . . . . . . 29
A.3.4 Money . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
A.3.5 Relationship to ISOC . . . . . . . . . . . . . . . . . . . . 31
A.4 IETF management issues (non-WG) . . . . . . . . . . . . . . 31
A.5 IESG processing problems . . . . . . . . . . . . . . . . . . 32
A.6 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
A.7 Community review and coordination problems . . . . . . . . . 33
A.8 Reform process issues . . . . . . . . . . . . . . . . . . . 34
Intellectual Property and Copyright Statements . . . . . . . 35
Davies (ed.) Expires August 25, 2003 [Page 2]
Internet-Draft IETF Problem Statement February 2003
1. Introduction: Issues/Problems in the IETF process
Discussions over the last couple of months have revealed a
significant number of perceived problems with the way the Internet
Engineering Taskforce (IETF) operates. In advance of trying to change
the IETF procedures and rules to deal with these problems, the IETF
should have a clear, agreed-upon description of what problems we are
trying to solve.
The Problem Statement working group was charged with producing the
document describing those problems. This document attempts to fulfill
that aim.
The document was derived by summarizing the extensive discussions
which took place initially on the wgchairs mailing list and
subsequently on the 'problem-statement' mailing list from November
2002 through to February 2003, and incorporating additional input
from relevant drafts published during this period (see [1], [2] and
[3]) and the minutes of recent plenary discussions. This produced a,
still extensive, list of perceived problems which were classified
into a number of related groups which is included in this document as
Appendix A. Note that the classification as it stands in this draft
was suggested by the processes which go on in the IETF.
For each group of problems collected in Appendix A, the set of
perceived problems have been further reduced to a paragraph or so of
text which seeks to encapsulate the issues for that group.
The final part of the editorial team work has been to encapsulate
these perceived problems into a small set of root cause issues, and a
short list of subsidiary issues which appear to be the most pressing
items engendered by the root cause. This list is set out in Section
2.
Section 1.1 gives a short explanation of the teams' thinking in
coming to the current view of the root causes.
The team has deliberately not reclassified the perceived problems in
line with our view of the root causes: This is because this a first
cut at deriving the root causes and other people may wish to take a
different view of the base material.
1.1 Consequences of Past Growth
As we examined the long list of problems summarized in Appendix A, it
became clear that the problems of the IETF are neither new nor are
they symptoms of a problem which is novel in the science of
organizations.
Davies (ed.) Expires August 25, 2003 [Page 3]
Internet-Draft IETF Problem Statement February 2003
The IETF started off as a small, focused organization with a clearly
defined mission and participants who had been working in this area
for a significant period of time. Over the period 1989-1999 the IETF
grew by a factor of ten or more in terms of number of participants,
and in terms of work in progress. Many of the problems and symptoms
appear to be fundamentally caused by the organization failing to
adapt its management and processes to its new larger size, and
failing to clearly define its future mission once the initial mission
had been completed or outgrown.
These failures are just those that afflict many small organizations
trying to make the transition from a small organization which can be
run informally and where essentially all participants fully share the
aims, values and motivations of the leadership, to a medium sized
organization where there are too many participants for informal
leadership and later arrivals are not as clear about the ethos of the
organization.
Some IETF participants have been aware of these issues for a long
time. Editorial team members were able to produce evidence dating
back to 1997 and 1992 that drew similar conclusions.
1.2 The Aim is Improvement, not Finger-pointing
We believe and acknowledge that all of the parties in the IETF are
acting in good faith, both in the normal operations which are
critiqued in this memo, and in the review process itself. No blame
for any of the perceived problems should be directed to any
individual, and the sole aim of the review process is to identify how
the IETF can improve itself so that it knows what it is about and
becomes fit for that purpose in the shortest possible timeframe.
1.3 Acknowledgements
Apart from the contributions of all those who provided input on the
problem statement mailing list, the final reduction of the problems
was assisted by the editorial review board consisting of:
Avri Doria <avri@apocalypse.org> (WG co-chair)
Dave Crocker <dcrocker@brandenburg.com>
Jeanette Hoffmann <jeanette@wz-berlin.de>
Margaret Wasserman <mrw@windriver.com>
Melinda Shore <mshore@cisco.com> (WG co-chair)
Rob Austein <sra@hactrn.net>.
Spencer Dawkins <sdawkins@cynetanetworks.com>
Special thanks are due to Margaret Wasserman for extensive reviewing
and contributions to the wording of Section 2.
Davies (ed.) Expires August 25, 2003 [Page 4]
Internet-Draft IETF Problem Statement February 2003
The early part of the reduction of the problem statement mailing list
input was done by Harald Alvestrand and the latter part by Elwyn
Davies and the editorial team. In total there were something like 750
extensive and thoughtful contributions (some making several points).
The thread was started by a call for volunteers in helping draft a
problem statement, but quickly turned into a discussion of what the
problems were.
Davies (ed.) Expires August 25, 2003 [Page 5]
Internet-Draft IETF Problem Statement February 2003
2. Root Cause Problems
This section forms the heart of this analysis, and lists the issues
which we believe lie at the core of the problems. Apart from the
first issue which is fundamental, the problems are not necessarily in
priority order, but they will be seen to be interlinked in various
ways.
2.1 The IETF does not have a common understanding of its Mission
The IETF lacks a clearly defined and commonly understood Mission: As
a result the goals and priorities for the IETF as a whole and any
Working Groups (WGs) that are chartered are also unclear.
The lack of a common mission has many consequences, of which the
principal ones appear to be:
o The IETF is unsure what it is trying to achieve and hence cannot
know what its optimum internal organization should be to achieve
its aims.
o The IETF cannot determine what its 'scope' should be, and hence
cannot decide whether a piece of proposed work is either in-scope
or out-of-scope.
o The IETF is unsure who its customers are, and consequently has
managed to drive certain classes of customer, who would otherwise
provide important input to the process, nore or less completely
out of the process.
o Working Groups can potentially be hijacked by sectional interests
to the detriment of the IETF's reputation.
o The misty vision has restricted the associated architectural view
to an outline top level view. It would be desirable to have
roadmaps and architectural views for portions of work which extend
beyond a single working group: It may also be the case that it is
no longer possible to fit the whole Internet within a single
architecture
o The lack of precision regarding our goals leads to WG charters and
requirements that are poorly thought out and/or not aligned with
the overall architecture.
The IETF has previously stated that it is not and does not wish to
become a Standards Determining Organization (SDO) but it is widely
perceived externally as another SDO and compared unfavorably with
other SDOs. The differentiation and the rationale for it are not
Davies (ed.) Expires August 25, 2003 [Page 6]
Internet-Draft IETF Problem Statement February 2003
generally understood but this does not mean that all the unfavorable
comparisons are invalid, and several of them are addressed in the
next section.
2.2 The IETF does not use Effective Engineering Practices
For an organization with 'engineering' in its title and participants
who are likely to trot out the statement "Trust me, I'm an engineer!"
when confronted with the need to find a solution to a particularly
knotty problem, the IETF has extremely ineffective Engineering
Practices.
The apparent inability of the IETF to deliver specifications within
the timeframe that the markets need and the excessive perfectionism
that is exhibited in some cases could both be improved if appropriate
Engineering Practices were in use.
Engineering requires appropriate trade-offs: Engineering success
needs refinement only to the point of 'fitness for purpose' which
should help to balance the tension between time to market and
perfectionism. The use of appropriate Engineering Practices should,
for example, prevent specifications being recycled in pursuit of
perfection when they are already adequate improvements on the status
quo.
Some of the key areas where the IETF's practices appear to need
tightening up include:
o Lack of explicit quality auditing throughout the standards
development process.
o Lack of written guidelines or templates for the content of
documents (as opposed to the overall layout) and matching lists of
review criteria.
o Poorly defined success criteria for WGs and individual documents.
o Lack of criteria for determining when a piece of work is
overrunning and/or is unlikely to be concluded successfully either
at all or within an acceptable time frame: Lack of process for
either extending the time frame, adjusting the scope or
terminating the work item or the whole Working Group.
o Tools to support the engineering process are minimal.
o The IETF explicitly avoids developing test tools for verifying
that protocols meet its specifications.
Davies (ed.) Expires August 25, 2003 [Page 7]
Internet-Draft IETF Problem Statement February 2003
In addition, IETF processes, and Working Group processes in
particular, suffer because commonly accepted Project Management
techniques are not regularly applied to the progress of work in the
organization.
o Project entry, goal setting and tracking processes are all either
missing or implemented less effectively than the norm for
commercial organizations in related activities.
o Charters regularly fail to set sufficiently granular milestones at
which progress of WGs, individuals and documents can be evaluated.
o Better estimation procedures need to be used to determine what the
expected delivery timescale for Working Group outputs should be.
These estimates must be compared with the acceptable market and
customer time frames for the work to be ready, and the scope of
the work adjusted if timely delivery appears impossible. This
process must be repeated from time to time during the project.
Finally, even where the IETF does have Engineering Practices defined,
there are frequently cases where they are ignored or distorted.
2.3 IETF contributors appear to be less engaged than in earlier days
There are a number of respects in which IETF participants and
contributors appear to have become less fully engaged with the IETF
processes than previously, for example:
o Although there may be large attendances at many WG meetings, in
many cases 5% or less of the participants have read the drafts
which are under discussion or have a bearing on the decisions to
be made.
o Commitments to write, edit or review a document are not carried
out in a timely fashion.
o Little or no response is seen when a request for 'last-call'
review is issued either at WG or IETF level.
Whilst this might be put down to contributors having less time
available in their work schedule during the downturn of the last two
years, this is not the whole story because there were signs of this
malaise back at the height of the boom in 2000.
This problem exacerbates the problems which the IETF has had with
timely delivery and may weaken the authority of IETF specifications
if decisions are seen to be taken by badly informed participants and
without widespread review.
Davies (ed.) Expires August 25, 2003 [Page 8]
Internet-Draft IETF Problem Statement February 2003
2.4 Authority and Influence in the IETF are concentrated in too few
hands
It appears that both authority and influence in the IETF are
concentrated in too few hands, and those with authority (primarily
the Internet Engineering Steering Group (IESG)) are insufficiently
accountable for some of their actions.
The IETF appears to have created a 'ruling class' system which tends
to re-select the same leaders from a limited pool of people who have
proved competent and committed in the past. Members of this 'ruling
class' tend to talk mainly to each other and former members of the
'ruling class'. Newcomers to the organization and others outside the
'ruling class' are reluctant to challenge the apparent authority of
the extended 'ruling class' during debates and consequently influence
remains concentrated in a relatively small group of people. This
reluctance may also be exacerbated if participants come from a
different cultural background than the dominant one.
The management and technical review processes currently in place were
adequate for the older, smaller organization, but are apparently not
scalable to the current size of the organization. The reliance of the
existing process on the small number of people who have authority in
the IETF both slows up the process and limits the number of formally
recognized 'preferred' positions within the IETF, thereby limiting
the (intangible) rewards for participants. This can lead to useful
and effective participants leaving because they cannot obtain any
recognition (the only currency the IETF has to pay participants),
which they use to fuel their own enthusiasm and help justify their
continued attendance at IETF meetings to cost constrained employers.
The current IESG processes allow one (or two) people to block or veto
the work put together and approved by the many in a Working Group,
possibly without good reason being given. This can happen in two
ways:
o IESG members who fail to put work on the IESG agenda. The
'responsible Area Director (AD)' for a WG can hold up progress in
the WG indefinitely, with no way for the WG to bypass the AD short
of using the formal appeal or recall processes, both of which
signal such a desperate breakdown in relations that, in practice,
no WG has ever attempted to use them to overcome this sort of
unreasonable delay.
o Documents can be blocked by one AD tabling a 'DISCUSS' issue
regarding a document. Although a mechanism exists whereby the
whole IESG can override a single AD's DISCUSS, any two ADs can
block a piece of work indefinitely.
Davies (ed.) Expires August 25, 2003 [Page 9]
Internet-Draft IETF Problem Statement February 2003
Apart from the frustration that this can cause inside the
organization, and the perception of disunity from outside the
organization, work which has been properly authorized as being within
scope of the IETF and properly quality checked during development
should almost never come up against such a blockage.
2.5 IETF Decision making processes are flawed
The IETF appears to be poor at making timely and reasonable decisions
that can be guaranteed to be adhered to during the remainder of a
process or until shown to be incorrect.
Participants are frequently allowed to re-open previously closed
issues just to replay parts of the previous discussion without
introducing new material. This may be either because the decision
has not been clearly documented or it may be a maneuver to try to get
a decision changed because the participant did not concur with the
consensus originally. In either case revisiting decisions stops the
process moving forward, and in the worst cases can completely derail
a working group. On the other hand, the decision making process must
allow discussions to be re-opened if significant new information
comes to light or additional experience is gained which appears to
justify alternative conclusions for a closed issue.
2.6 IETF Participants and Leaders are inadequately trained
Participants and leaders at all levels in the IETF need to be taught
the principles of the organization (Mission and Architecture(s)) and
trained in carrying out the processes which they have to use in
developing specifications, etc.
The IETF currently has voluntary and inconsistent processes for
training its participants which may be why significant numbers of
participants seem to fail to conform to the proper principles when
working in the IETF context.
The people in authority have generally been steeped in the principles
of the IETF (as they see them) and first-time non-compliance by newer
participants is sometimes treated as an opportunity for abuse rather
than by recognition of a training failure.
Lack of training compounded with concentration of influence in the
'ruling class' can lead to newcomers being ignored during
discussions, consequently being ineffective either in their own eyes
or their employers and so leaving the IETF.
Davies (ed.) Expires August 25, 2003 [Page 10]
Internet-Draft IETF Problem Statement February 2003
3. Security Considerations
This document does not of itself have security implications, but it
may have identified problems which raise security considerations for
other work. Any such implications should be considered in the
companion document which will be produced setting out how the IETF
should set about solving the problems identified.
Davies (ed.) Expires August 25, 2003 [Page 11]
Internet-Draft IETF Problem Statement February 2003
4. IANA Considerations
There are no IANA considerations defined in this document.
Davies (ed.) Expires August 25, 2003 [Page 12]
Internet-Draft IETF Problem Statement February 2003
References
[1] Huston, G. and M. Rose, "A Proposal to Improve IETF
Productivity", draft-huston-ietf-pact-00 (work in progress),
October 2002.
[2] Blanchet, M., "Suggestions to Streamline the IETF Process",
draft-blanchet-evolutionizeietf-suggestions-00 (work in
progress), November 2002.
[3] Hardie, T., "Working Groups and their Stuckees",
draft-hardie-wg-stuckees-00 (work in progress), February 2003.
Author's Address
Elwyn B. Davies
Nortel Networks
Harlow Laboratories
London Road
Harlow, Essex CM17 9NA
UK
Phone: +44 1279 405 498
EMail: elwynd@nortelnetworks.com
Davies (ed.) Expires August 25, 2003 [Page 13]
Internet-Draft IETF Problem Statement February 2003
Appendix A. Summarization of Problems from Mailing List
A.1 Problem area/subject matter issues
Summary:
These problems are fundamentally caused by the lack of
well-defined mission, and consequent effects, such as lack of
priorities and goals, and undefined boundaries to the scope of
problems which the IETF will tackle.
Issues derived from Mailing list:
o The IETF is not really sure of its mission (Margaret Wasserman)
o At the moment, we collectively accept almost any working group,
and allow them to produce almost any result. (Joel Halpern)
o WG charters and consequential requirements are not tightly defined
at the beginning of the process (Marc Blanchet: P3)
o I-Ds that are of the "wouldn't it be a good idea if this worked"
type confuse the process (Eric Burger)
o Insularity is unavoidable given the large number of working groups
(Melinda Shore)
o There are no roadmaps for areas or other large chunks of the
architecture, just for WG efforts (David Harrington and others)
o Too little architectural guidance in the IETF (Carl-Uno Manros)
o Do we really know who are customers are? May have become
irrelevant to some hardware manufacturers (Dean Willis)
o Do we produce (sufficiently) interesting specifications?
o It is impossible to define a single architecture for the Internet
(Brian Carpenter)
A.2 Working Group process issues
The major process within the IETF is the lifecycle of Working Groups
(WG) and the standards documents which they are chartered to produce.
This section contains problems which appear to relate to various
aspects of the WG process
Davies (ed.) Expires August 25, 2003 [Page 14]
Internet-Draft IETF Problem Statement February 2003
A.2.1 Who the participants are and represent
Summary:
The IETF is often described as a volunteer organization but that
is really a myth, but with honorable exceptions among the
independents and academics. The great majority of participants are
paid by their employers to attend the IETF, and are directly or
indirectly rewarded by those employers for 'success' within the
IETF.
The IETF differs from other SDOs (whether the IETF wishes to class
itself as an SDO or not is irrelevant here - people outside the
IETF treat it as an SDO) in that although an employer may support
a participant, the employer has no say in roles which a
participant may be offered. It is up to the the individual
particpant to carve out a reputation for him/herself by initially
volunteering for tasks in the WGs and later being tagged by an AD
as a WG chair or for assistance in an Area or selected by the
NOMCOM for membership of the IESG or Internet Architecture Board
(IAB). Many participants use significant amounts of their own
time as well as their employer's time in order to achieve
'success' but the average IETFer is probably more of a 'pressed
man' than a volunteer.
Most participants are initially selected to attend the IETF with a
view to the individual helping to ensure that some standard of
value to the employer is created and are supported at the IETF
with this end in view: To that extent, at least, they are beholden
to their employers so that the work that they carry out in the
IETF will generally support their employer's business, even when
the IETF tries to maintain the fiction that all participants
attend as individuals and their company affiliations are left
outside the hall.
This situation has a number of problems for the IETF:
* Achieving true consensus requires that individuals can act
truly independently, ignoring company mandates, which may be
impossible for many more junior participants.
* The IETF clearly does not employ any of the participants and
hence lacks the ultimate sanctions (financial penalty or
employment loss) that an employer can use to control and direct
employees. This means that enforcing deadlines and controlling
behavior is extremely difficult (often referred to as the 'cat
herding problem') and it tends to make managers less inclined
to delegate ('never delegate anything you don't have time to do
yourself' in case the delegated task doesn't get done): In
Davies (ed.) Expires August 25, 2003 [Page 15]
Internet-Draft IETF Problem Statement February 2003
this respect the IETF does resemble a volunteer organisationm.
* The IETF is normally unwilling (unable?) to challenge an
individual who is misusing the freedom of the IETF to push a
company mandated position against the better interests of the
community, even if instances of misuse can be identified.
* The very diverse backgrounds of those now attending the IETF
may make it much more difficult to arrive at consensus.
Even where employers remain reasonably altruistic in their support
of the IETF, recent market downturns have reduced the amount of
time that some employees are allowed to spend on IETF activities,
and the number of employees allowed to attend meetings has been
shrinking.
On the other hand, there are a limited number of people who have
by prolonged, effective service reached position of ongoing
influence and personal authority in the IETF world. Because the
IETF is an elite organization, many of these people do not have
other fora to which they could 'graduate', and hence remain active
in the IETF. Many less experienced IETFers appear to be in awe of
these veterans, and this can lead to undue deference and
unwarranted acceptance of their positions. Some employers seem
willing to offer such people posts which allow them to continue to
devote significant amounts of effort to IETF activities: It is
clear that they can have an exceptional influence on IETF work,
and there could be an avenue for companies to exploit this
inappropriately.
Finally, some potentially important participants (e.g. staff from
network operators) also appear to being driven out by the IETF's
apparent disregard for their interests. This problem is related to
the failure of the IETF to acknowledge who its customers are, as
noted in Appendix A.1.
Issues derived from Mailing list:
o Individuals' behavior has changed (less commitment)? (Aaron Falk)
o Less consensus on what parts of the IETF's work is "core" and
"critical" (Aaron Falk)
A.2.2 How the consensus process works
Davies (ed.) Expires August 25, 2003 [Page 16]
Internet-Draft IETF Problem Statement February 2003
Summary:
The rough consensus mechanism that is used to settle the output of
the WGs is the key distinguishing factor of the IETF. The use of
rough consensus together with resistance to producing standards
with 'profiles' or multitudinous options results in IETF
specifications which generate multiple independent implementations
characterized by maximal commonality of features.
The IETF has grown up from a relatively small organization in
which essentially all participants shared a common purpose and
ethos, understanding the principles and the underlying
architecture which underpins the whole enterprise.
Over the last few years, the IETF has grown in various ways:
* The IETF is producing standards for more than a single
constellation of activities. Diverse architectures may be
needed and diverse opinions are bound to arise, and people
operating in one constellation may not understand the
motivations and requirements of others.
* The IETF is consequently larger with many more participants and
more WGs in progress at any one time.
* The IETF is no longer a totally English language or
overwhelmingly North American culture based entity.
Participants with alternative mother tongues and home cultures
make up a significant part of the participants.
* The influence of the IETF has grown as the Internet and private
networks using the same protocols have become all pervasive.
The IETF is now consulted by and involved in liaisons with many
other SDOs and related organizations.
This growth has resulted in the IETF demonstrating many of the
same strains that conventional commercial organizations undergo
when they grow from the 'small' category into the 'medium'
category.
The consensus process and all that underlies it were initially
part of the shared ethos of the early cohorts of IETF
participants. The advent of large numbers of newer participants,
who have never really been taught the full principles of the
organization, probably detracts from the ability to achieve a true
'rough consensus' which is the 'best' engineering solution given
the technical and political constraints. The limited
'Introduction to the IETF' session given at each of the meetings
is voluntary and is probably not sufficient to drive an ongoing
Davies (ed.) Expires August 25, 2003 [Page 17]
Internet-Draft IETF Problem Statement February 2003
mindset, particularly if attendees have been used to some other
ethos.
Notwithstanding the primacy of discussion and consensus on the
mailing list, it is clear that some decisions are made in less
than ideal circumstances at meetings where observers participate
in hums or votes where it is not necessarily clear that they
fully understand the issue. Similarly, some participants with
different cultural backgrounds may be effectively disenfranchised
by unclear and totally verbal processes at such meetings.
Achieving consensus has tended to become a tail end operation
rather than a continuing process, particularly where there are
competing (semi-proprietary) alternatives. This leads to multiple
options or development strands being pursued for too long in the
WG (contrary to the basic tenets of the IETF) and the whole
process is slowed down by dilution of effort and ongoing dispute.
Issues derived from Mailing list:
o Consensus process is not working, or is too slow, due to lack of
interest in the common good (Melinda Shore)
o What consensus means has been lost, or the spirit of consensus is
missing (Edward Lewis)
o Discussions in WGs turn into battles along "party lines" between
companies, not an effort towards interoperability (Edward Lewis)
o Heavy investment in alternatives pre-standardization leads to
vested interest in outcomes, preventing consensus when there is no
clear technical lead
o Rat holing over fundamental disagreements on the goals of the
process leads to bog-down; requirements documents can sometimes
help avoid this (Tony Hain)
o WG process is too slow, because of feeping creaturism, deadlocked
conflicts or lack of worker bandwidth (Jari Arkko)
o Waiting for WG meetings at IETF meetings to identify/reach
consensus sometimes delays the advancement of the WG - this
happens even though the mailing list should be the prime
discussion forum. Judging consensus based on the mailing list
comments sometimes obscures the silent majority opinion (Marc
Blanchet: P4) and sometimes reflects exhaustion rather than true
consensus (Elwyn Davies).
Davies (ed.) Expires August 25, 2003 [Page 18]
Internet-Draft IETF Problem Statement February 2003
o It's not possible to identify the time when major changes to an
I-D set are unlikely, so that it's reasonably safe to start
designing implementations (James Luciani)
o Cultural issues can lead to certain constituencies being sidelined
(Marc Blanchet: P10/Elwyn Davies)
o The IETF consensus process appears not to be good at selecting
between two or more reasonably well-qualified alternatives.
Standardizing several options tends to reduce market acceptance
(MPLS CR-LDP/RSVP-TE) or cause long delays (SNMPv2). IETF doesn't
do compromises with all the options (as do some SDOs) normally
(and it shouldn't probably). (Margaret Wasserman)
A.2.3 How the interaction works
Summary:
Mailing list discussions are not necessarily the 'sine qua non'
for achieving IETF style rough consensus. Active involvement of
the WG chair(s) can help to keep discussions on track and polite,
but an inflamed discussion in a large WG causes mail overload for
most participants.
Breaking the deadlock between 'religiously' held positions is
extremely difficult and is a major cause of the failure to achieve
consensus.
Issues derived from Mailing list:
o Mailing lists are not (always) an effective tool (Aaron Falk)
o Large groups do not interact well enough to form consensus (James
Kempf)
o Large WGs means that we get more groupings of intransigent people,
leading to difficulty with consensus (Donald Eastlake)
o Mailing lists with large numbers of postings (500 in a week, for
instance) are hard for people to follow (Thomas Narten)
o People sticking to fixed positions on mailing lists leads to lots
of mail but no consensus (James Kempf)
o Maintaining decorum and forward progress on a mailing list is not
easy (many)
o Consensus tests at WG meetings by means of 'hum' responses to
Davies (ed.) Expires August 25, 2003 [Page 19]
Internet-Draft IETF Problem Statement February 2003
verbal questions are often unclear and lead to bad choices (Marc
Blanchet: P9)
o Non-English speakers find it easier to read questions rather than
be sure they understand a spoken question (Marc Blanchet: P10)
o Large teleconferences have not been tried. (Jari Arkko)
A.2.4 Measuring the outcome
Summary:
The IETF needs a strategy to go with its mission (when it is fully
spelled out, see Appendix A.1). At present the IETF does not
include success criteria for standards which it designs beyond the
purely technical. This may be a problem because the customers
criteria are neither perfection nor purely technical effectiveness
- business utility or fitness for purpose plays a part as well.
The alienation of certain types of commercial organization (e.g.
operators) from the IETF means that they do not assist in
determining the success factors at the time the standards are
being developed.
Issues derived from Mailing list:
o [How much do we lose from the 'travel filter' and the silent
majority [Editor] ] If a person/company does not have money to
travel to a meeting, is it likely that the company will have the
money to implement the protocol or make it a commercial success?
What might need to be changed to better achieve RFC2418 existing
advice? (Margaret Wasserman/Dave Crocker)
o IETF traditionally not accepting model of vendors collaborating to
get commercial fix up, rather open collaboration to do the 'best
thing' (Eric Rescoria)
o What is success? Lots of people talking relevance...maybe just the
same as 'commercial success' but may also involve elements of
quality and timeliness. Is it better that we use the official
model of 'open, fair, technically sound and useful'. (Margaret
Wasserman)
o Large vendors may not have the most appropriate experience
(Margaret Wasserman)
o Difference between making incremental upgrades to existing
protocols etc where incumbents have experience/influence vs. new
Davies (ed.) Expires August 25, 2003 [Page 20]
Internet-Draft IETF Problem Statement February 2003
services/protocols where startups have at least as much experience
and may be who is going to execute (Henning Schulzrinne)
o High quality has not necessarily been the answer, more 'just
enough quality delivered in a timely fashion' (Marshall Rose)
o Must also be extensible (Scott w Brim)
A.2.5 Lack of Transparency: Design Teams and Interim meetings
Summary:
The outputs from design teams and interim meetings appear to have
a disproportionate chance of becoming the final output of a WG.
It appears to be difficult to make really significant changes to a
document which has emerged from either of these processes or, in
the worst case, to have the output of a design team discarded and
totally reworked. We know that in principle the documents have no
greater validity than any other but the combination of the
apparent waste of resource and the necessity of making progress
tends to militate against them being subject to major savaging.
Issues derived from Mailing list:
o Design Teams introduce lack of transparency. They are perceived as
a way to bypass the normal working of the WG, to push forward the
opinions of a (self-)selected subgroup and as closed fiefdoms
which are not selected in an open fashion (Marc Blanchet: P6)
o Design team work is rarely challenged or subjected to external
quality control by the rest of the community in the same way that
more publicly constructed documents are tested (Elwyn Davies)
o Interim meetings are not as transparent as normal WG operations:
Scheduling problems and travel constraints limit the attendees and
reduce the input spectrum (Marc Blanchet; P7)
A.2.6 Problems in the Large - Cross fertilization between WGs/Areas and
System level Thinking
Summary:
The IETF lacks a formal structure for handling complex problems
that might require firstly to be analyzed into simpler problems
and then to create an architecture, within the overall IETF
architecture, which provides context for the solutions of the
smaller problems, and will superintend the operation of one or
more WGs set up to solve the individual smaller problems.
Davies (ed.) Expires August 25, 2003 [Page 21]
Internet-Draft IETF Problem Statement February 2003
WGs appear to be poor at doing real blank sheet design.
WGs should be ready to bring in Subject Matter Experts from
outside the central domain of a WG problem as soon as it becomes
clear that there will be interactions with other areas, rather
than at a late stage and only when beaten up by the ADs.
Issues derived from Mailing list:
o IETF is not good at tackling large 'system level' problems where
they need to be broken down into a number of smaller questions and
the smaller questions given coordinated answers. This may need an
extra type of organizational structure in the IETF to deal with
complexity. (Elwyn Davies/Margaret Wasserman)
o WGs are rarely a good way to get from 'interesting buzzword' to
'serious set of requirements and definitions' (John C Klensin)
o WGs are rarely good at doing real blank sheet design (either a
good design shows up from outside the WG and is endorsed, maybe
with fine tuning, or the WG cycles and produces garbage, which is
approved because of exhaustion) (John C Klensin)
o Experts from areas outside the central domain of a WG problem are
rarely brought in until late in the WG process which can lead to
disconnect, reinvention of wheels, incompatibilities and missing
pieces (Marc Blanchet: P5)
A.2.7 Quality control issues - Documents
Summary:
The IETF does not have a well-defined and well-documented quality
control process during the normal operation of creating a document
in a WG.
Quality Control on WG process and documents is all head and tail,
and no middle. Nobody associated with the creation of a document
is explicitly responsible for its quality.
The need for three levels of standard maturity needs to be
revisited, yet again, particularly with respect to the enforcement
of existence of interoperable implementations at Proposed Standard
level.
Issues derived from Mailing list:
Davies (ed.) Expires August 25, 2003 [Page 22]
Internet-Draft IETF Problem Statement February 2003
o Some (valid) comments get lost in the rough and tumble of a
mailing list. Without a formal problem tracking system (which is
used for some documents) it is difficult to ensure that a
particular version of a document has addressed all the problems
raised (Marc Blanchet: P8)
o WGs are not diligent enough in ensuring quality control (Randy
Bush)
o Problems (for instance security) get discovered at IESG time, not
earlier; Quality control on documents is mostly back-end loaded
(to WG last call and IESG review) rather than being done at every
stage (e.g. as on admission as a WG item) - this is known to
create much more rework than checking at earlier stages. (Randy
Bush/Bernard Aboba)
o Not enough guru bandwidth for appropriate security review (Derek
Atkins, Steve Bellovin)
o Too little clarity and brevity in WG output (Aaron Falk)
o Lack of running code means that unimplementable specifications get
on track (Kurt Zeilenga)
o There are a number of issues surrounding MIB modules (Bert Wijnen)
o 'Damn the quality control torpedoes, full speed ahead for the
standard' (open mike sentiment)
o We think there are cases where things go to Proposed Standard
specifying things that can't work. One way to avoid that is to
insist that someone try it before submission. (Harald Alvestrand,
summarizing)
o Normative references to documents at earlier stages of
standardization not allowed causes blockages in document
publishing (John C Klensin)
o Should the IESG really need to review all informational RFCs
(especially those not associated with WGs)? (Jari Arkko)
o The review checklists for RFCs (and implicitly I-Ds) are not very
detailed: From my experience, inexperienced reviewer/auditors
could use a much more detailed set of guidelines of things to
check for (Elwyn Davies) It would be nice to know just what rules
are applied for each maturity level (Resnick/Alvestrand)
o Are different standards of review possible at different stages of
Davies (ed.) Expires August 25, 2003 [Page 23]
Internet-Draft IETF Problem Statement February 2003
the standards process? (Margaret Wasserman) IN theory there is but
it is not clear that this is in fact applied (i.e. PS reviewing
may be more strict than might have been envisaged?) (Scott
Bradner)
o If most of the Internet runs on protocols which have only reached
Proposed Standard (PS) and the return on investment of taking a
protocols to Draft or Full standard is seen as small, then why
persist with a three level standards process? (Margaret Wasserman)
o Glorification of I-Ds and PS documents - often leading to lack of
formal interoperability testing (James Kempf)
o Apparent presumption that almost any I-D that gets onto Standards
Track in a WG will inevitably become the standard without
substantial modification in any form of its basic design or
contents (James Kempf) --- If this were really true IESG would
have been defanged. (Brian Carpenter) -- possibly seeing results
of design teams, AD associated influence and backdoor review. - Is
IESG effect detrimental to WG effort (i.e. does the Old Boy
network effectively denigrate work of WGs)
o [Solution note: Need to be able to document current utility of a
protocol. features which are being used interoperably and feature
not being used or being used non-interoperably - this might give a
vehicle for the 'good enough' status. (Fred Baker)]
o ADs and IESG (apparently) do not provide significant architectural
input between the BOF stage and the IESG review stage of a WG/
document. There is no provision for a formal architectural review
to be requested by either party, although presumably informal
review could be done if requested. (Pete Resnick et al)
A.2.8 Quality control issues - Personnel
Summary:
Documents do not currently have a designated quality auditor to
assist the editors and authors. Quality auditors would be
expected to be aware of the whole context in which the document
will sit so that they can ensure cross-document consistency, and
compliance with the relevant architecture which are often missing
at present.
Editors and authors do not always seem to be aware of what makes a
quality (protocol standards) document. If they were it would be
likely to speed up the document production process by minimizing
rework at the outset.
Davies (ed.) Expires August 25, 2003 [Page 24]
Internet-Draft IETF Problem Statement February 2003
Any quality reviewing process for use during document creation
that might be put in place needs to be lightweight (not hours of
committee work) but effective. Authors need to be prepared to
accept reasonable correction - 'ego free document creation'!
The aim of the WG reviewing process should be to make the IESG
review a 'rubber stamp' formality, rather than the current 'battle
of wills'.
Issues derived from Mailing list:
o Authors/editors do not understand what makes a document Good; they
should be trained (Avri Doria)
o WGs do not explicitly appoint quality reviewers for documents.
Cross document reviewing is not much in evidence leading to
inconsistencies (esp. terminology). (Elwyn Davies)
o WG chairs not adequately trained (need to understand process and
tools available to them, e.g. milestone maintenance) (Marc
Blanchet: P1)
o WG chairs should be more aware of who their peers are and interact
more with them (Spencer Dawkins)
o WG chairs do not always have adequate time for task: All WGs need
two chairs minimum (Marc Blanchet: P2)
o A single AD should not be able to effectively block a WG once it
is started (Dave Crocker)
o Ensure all ADs attend all relevant interim meetings (John C
Klensin)
o Ego clash between WG members and IESG: Provoked particularly by
tail-end pronouncements from the IESG on-high requiring extensive
modification of exquisitely crafted WG documents (Pete Resnick,
Randy Bush, Melinda Shore)
A.2.9 Timing/Latency
Summary:
WG charters tend to contain milestones which are too far in the
future to allow effective monitoring of progress against
milestones, or milestones which prove to be too short when the
actual work to be done has been measured.
Davies (ed.) Expires August 25, 2003 [Page 25]
Internet-Draft IETF Problem Statement February 2003
WG control by the ADs and the IESG does not tend to use modern
management tools (such as management by exception) resulting in
excessive load on the ADs and poor performance of the WGs against
plans.
Deadlines which are missed are currently either ignored or in the
worst cases used as excuses to shut a WG down. Any missed
deadline should be discussed between the WG chair(s) and/or
document team and the AD(s) to decide if the deadline can be
extended without compromising the usefulness of the result,
whether the scope of the work should be scaled back to bring
delivery closer or as a last resort the work should be scrapped>
The WG should also be consulted.
Issues derived from Mailing list:
o Documents evolve very slowly if they are only revised once per
IETF meeting cycle: This is a matter of document editors/reviewers
not necessarily having enough time as well as WG chairs hot
forcing the issue (Aaron Falk) [But .. see the problem with
volunteers [Editor]]
o WG Plans are too long/Milestones too far in the future: Checkpoint
milestones (rather than completion milestones are rarely used)
o Milestones are set unrealistically without really understanding
the work to be done. ADs encourage WG chairs to set short
deadlines to get things out and then when they are not met they
fall into disrepute (management truism) (Scott W Brim)
o WG plans/milestones ignored once set and not enforced either by WG
chairs or ADs (Fred Baker)
o No firm time limits are set on WGs when they are created (e.g.
produce something in 18 months or be deleted)
o Are WG meetings too short (many WGs that do good work find they
need one or more interims)? How to get balance between good
face-to-face time and mailing list time? (John C Klensin/Randy
Bush)
o Longer meetings may push attendees into becoming 'standards
drones' (John C Klensin)
o More expeditious closing down of WGs that aren't meeting deadlines
(or going in circles) (many)
o Too many documents in each WG (but master documents in other
Davies (ed.) Expires August 25, 2003 [Page 26]
Internet-Draft IETF Problem Statement February 2003
groups are actually just combinations of all of ours)? (John C
Klensin)
o Timeliness/competence curve changes shape after a couple of years
- overall competence starts to drop off after that time (Marshall
Rose)
o Improve feedback process to ensure that WG chairs will know there
are consequences of not meeting deadlines (Marc Blanchet)
o Documents seem to be taking much longer to reach RFC status than
they used to (roughly 3 times worse than when records were first
kept). Associated with there being roughly three times as much
work going on in IETF as then, this is probably an unsupportable
burden.
o Milestones are unrealistic to the extent that they currently do
not include any time for IESG review. However, although this could
be considered to be a fixed quantity, it is relatively
unpredictable because it would be dependent on the load at the
time it was submitted for review and it would depend on the size
and quality of the document submitted. However , if there are any
dependencies in documents, clearly one must make some allowance
for IESG review in order to get a realistic view of when a
document or group of documents will be finished. It might also
help to determine if the IESG is keeping up, and whether the
workload being chartered was excessive..(Elwyn Davies)
o Measuring time in working groups is misleading - PACT focuses on
what we know how to measure instead of what we care about
(success/efficiency/quality?) (Joel M Halpern)
A.3 IETF organizational and Perception issues
A.3.1 The Old Boy network
Summary:
The IETF is a self-perpetuating oligarchy, containing a small
population of 'big cheeses' who have a disproportionate influence
on the outcomes of the IETF, and who are so deified that newer
participants are overawed by them and consequently accept their
words without demur. WG chairs and ADs are sometimes perceived to
fall into this category.
Issues derived from Mailing list:
Davies (ed.) Expires August 25, 2003 [Page 27]
Internet-Draft IETF Problem Statement February 2003
o The sociological structure of the IETF, despite the good offices
of NOMCOM, is, IMHO, a self-perpetuating oligarchy (Margaret
Wasserman, Elwyn Davies).
o There are a number of strong personalities who have come to have a
possibly disproportionate influence. Some/Most of these are or
have been Area Directors, but their influence tends to be
pervasive and their views are rarely challenged and even more
rarely overruled. The influence persists even when they are no
longer in office. (Elwyn Davies).
o It often appears that many newer participants are overawed by the
'big cheeses' and I have observed consensus overturned by their
intervention. Sometimes this is justified by Wisdom of the
Ancients, but on other occasions less so. (Elwyn Davies).
o The WG chairs and ADs are sometimes seen (or at least perceived)
to use their status to push their own documents or proposals (Marc
Blanchet: P11)
A.3.2 External Perception of the IETF
Summary:
The IETF is mostly seen by outsiders as a kind of Standards
Determining Organization (SDO) even though the IETF tries not to
be or act like an SDO. If it wishes to be seen as something else,
then the IETF has to clearly set out what this is and how it will
act differently from a conventional SDO to distinguish itself.
Having decided on this, a publicity campaign will be needed to get
the message across to supporters whilst putting across that it
fulfills a useful role in its 'new' guise.
For the time being, the IETF is compared with other SDOs and is
not seen as the equal of other SDOs, especially by participants
and leaders of these other SDOs. Apart from the desire to be seen
as something other than an SDO, this may well be because of the
different processes which the IETF uses. Consequently, the IETF
needs to work to improve the quality of the work that comes out,
avoid misuse of its processes and ensure that its work is
delivered in a timely fashion, in order to maintain its influence
on the Internet.
Issues derived from Mailing list:
o The IETF is the standards body of last resort: If you can't get
your standard accepted by some 'more reputable' body, then there
is 'always the IETF'. (Nortel staff member via Elwyn Davies)
Davies (ed.) Expires August 25, 2003 [Page 28]
Internet-Draft IETF Problem Statement February 2003
o The IETF is perceived as too slow: See latency discussion (Brian
Carpenter: Issue 1)
o The IETF is perceived as unfocussed (Brian Carpenter: Issue 2).
Related to IETF inability to deal with large scale architectural/
system level problems (see above).
o IESG is too picky (won't charter my WG, won't approve my RFC, etc)
(Brian Carpenter: Issue 3) [This is exact inverse of internal
perception - see Joel Halpern's comment above]. The presence of
both views may mean that the IESG is pretty much getting it right
within its current remit.
o Operators perceive that the IETF neither understands nor
particularly cares about their concerns. In general this means we
don't hear enough from them (Brian Carpenter: issue 9)
o The IETF is perceived as doing near-term work very well but
long-term work very badly (PACT draft, etc)
A.3.3 Loose control of IETF over its own work
Summary:
The IETF does not use adquate project management mechanisms to
control the projects that are going on.
The IETF does not have effective Engineering Practices.
The fact that participants who are performing all sorts of tasks
for the IETF are not employed by the IETF limits the control that
the IETF has over the quality and timeliness of their work, since
the IETF is not in a position to apply significant penalties for
non-performance beyond 'loss of reputation', and this is not
always reflected back into the participant's home organization.
The IETF allows some of its processes (esp. publication of
informational RFCs) to be used by people who are not fully
integrated into the IETF structure: This can result in excessive
work loads from non-core activities on an already over- stretched
management, and a risk of the system being abused by unscrupulous
vendor mwrketeers.
Issues derived from Mailing list:
o The 'openness' of the IETF can be abused by 'marketroids':
Informational RFCs used as 'standards' by unscrupulous vendor
marketeers, who feed proprietary proposals into Informational RFCs
Davies (ed.) Expires August 25, 2003 [Page 29]
Internet-Draft IETF Problem Statement February 2003
unless carefully monitored (Brian Carpenter: issue 6)
o The voluntary nature of IETF organization makes it difficult to
ensure that the owner of any given task will actually have time
for it (Brian Carpenter: Issue 7(i))
o The volume of work being undertaken by the IETF may be getting to
the point where it is a crippling overload for a volunteer
organization (Brian Carpenter: issue 7(ii))
o The voluntary nature of the IETF tends to discourage delegation in
all those with any responsibility: "Never delegate anything which
you haven't got time to do yourself" is not a phrase I've heard
used around IETF but it is very easy to fall into this mode in an
'all/mostly voluntary' organization if you have a job with a
deadline and you don't really trust others to stick to the
deadline. This tends to reinforce the sort of oligarchy mentioned
above, where those who have been found to be efficient take a
disproportionate amount of the work irrespective of their true
competence. (Elwyn Davies)
o Termination reviews on expired-in-grade RFCs not being done on old
PSs and DSs (Kurt D Zeilenga)
o The IETF needs more technical leadership, without moving to a
dictatorial culture as in...
* IAB doesn't do enough proactive architecting (Rob Natale)
* IESG does not do enough steering especially in terms of
proactive longer-term guidance of Area Direction and cross-Area
interactions (Rob Natale)
* WG leadership is, on the whole, far more inclined to
over-compensate on the side of democracy than on the side of
sticking to the mission (Rob Natale)
A.3.4 Money
Summary:
To a very large extent, the IETF has endeavored to ignore the
effects money on the network and its own operations. The IETF has
tended to assume that the right people for the job will continue
to be available to them to do the job without having to fund any
people and without becoming beholden to the companies that fund
its participants.
Davies (ed.) Expires August 25, 2003 [Page 30]
Internet-Draft IETF Problem Statement February 2003
The IETF may be limiting itself by being unable to deal with many
of the inter-domain and service-related issues that are relevant
to today's network because of the IETF's status viz-a-viz the (US)
anti-trust laws.
Issues derived from Mailing list:
o The Internet Society has recently (2000) created 'Platinum'
membership to extract funds from companies to support standards
work in IETF. The question is what 'quid pro quo' any such
companies might wish to extract as output in return for their
contribution. The IETF has so far managed with little or no
financial constraint on its operations - this may be about to
change. (Brian Carpenter: issue 10)
o The IETF is ham-strung by the anti-trust cruft that stops it
addressing the inter-domain issues that are really at the heart of
today's federated internet. (Elwyn Davies)
o Subject matter experts cannot afford to attend interim meetings:
Should IETF be funding their expenses? (Margaret Wasserman)
A.3.5 Relationship to ISOC
Summary:
TBD
Issues derived from Mailing list:
o Appointment of NOMCOM chair is a 'hidden process' (Brian
Carpenter)
o Appointment of NOMCOM chair plus some money is limit of (not very
visible) ISOC interactions (Brian Carpenter, Elwyn Davies)
A.4 IETF management issues (non-WG)
Summary:
TBD
Issues derived from Mailing list:
o Predictability, Accountability, Competency and Timeliness are
important to the IETF, and we don't have enough of them (draft by
Marshall Rose, Geoff Huston)
Davies (ed.) Expires August 25, 2003 [Page 31]
Internet-Draft IETF Problem Statement February 2003
o IESG and IAB spend precious time on political sludge. This cannot
be avoided but it reduces IESG and IAB bandwidth: Choices made in
the IETF change available (political) policy options and bad
policy decisions outside the IETF can damage the technology, so we
must be engaged (Brian Carpenter: issue 11)
o There is a risk of micro-management at all stages in the
IAB->IESG->AD->WG chair->Editor/Author chain. This needs to be
avoided whenever possible by suitable training (Brian Carpenter:
issue 12)
o Need to introduce management by exception (Elwyn Davies)
o Need for general management training for IESG members (Margaret
Wasserman)
o IESG is overloaded - need to find means of delegation and workload
reduction (Margaret Wasserman)
o Possibly as an attempt to control marketroid Informational RFCs,
the IESG is doing reviews on them for which they appear to have no
mandate. Whilst this may be helping credibility, it is an extra
load on the IESG, and appears to be holding up legitimate
Informational RFCs as well as 'end-runs' (John C Klensin)
o Lack of Transparency of Management Functions
o Lack of transparency makes it hard to diagnose what the
organization's problems are (Hilarie Ormond)
o IETF process is opaque, and therefore hard to coordinate with
(Dean Willis)
o IESG is seen as opaque or arbitrary, at least from outside the Old
Boy network. Apparent emulation of a black hole (in regard to
document review particularly) in some cases can be frustrating
(Brian Carpenter: issue 4(i))
o IAB is seen as somewhat opaque from outside the Old Boy network,
although it is not a hurdle for document approvals (Brian
Carpenter: Issue 4(ii))
A.5 IESG processing problems
Summary:
TBD
Davies (ed.) Expires August 25, 2003 [Page 32]
Internet-Draft IETF Problem Statement February 2003
Issues derived from Mailing list:
o Time from WG finished to IESG approval is too long
o Queuing delays inside the IETF process are far too long (Don
Eastlake) (suggests that 1-2 months is most often a
non-problematic delay, but that 1 year is a disaster)
o Poor quality documents tend to act as DoS attacks... sucking away
process/review bandwidth from quality documents. (Kurt Zeilenga)
o For any X in the IETF process (WG, IESG, RFC Editor), it takes too
long (David Meyer)
o The process' slowness means that I-Ds take on the force of a
standard (Eric Burger)
o Too many steps in the overall IETF process: No real urgency to
progress Proposed Standards to Draft/Full Standard with the result
that internet mainly runs on Proposed Standards (where it is
running on I-Ds or Informational RFCs) (Brian Carpenter: Issue 5)
o Not enough information for the WG chairs or authors to identify
the IESG concern and effect the resolution in a timely manner
(Dean Willis)
o Too many WGs in progress for IESG to handle reviewing in a timely
fashion (Dave Crocker/PACT)
A.6 Tools
Summary:
The IETF is missing several useful tools that could assist with
the management of WG processes. A standard set of tools would
facilitate progress and minimise set and learning time for each
group and well-organised documentation for the existing tools and
any new ones is essential.
Issues derived from Mailing list:
o Need better map of tools available to WG chairs and document
authors (Edward Lewis)
A.7 Community review and coordination problems
TBD
Davies (ed.) Expires August 25, 2003 [Page 33]
Internet-Draft IETF Problem Statement February 2003
Issues derived from Mailing list:
o Architectural decisions with IETF-wide impact doesn't get adequate
community review (Margaret Wasserman)
o Few people review drafts in IETF Last Call (Avri Doria)
o We're not using the three-stage standards process as designed
(Spencer Dawkins)
o We as a community are not following the rules we set for ourselves
(James Luciani)
o It's hard to get a discussion going when the IAB works on
architectural considerations (Rohan Mahy)
o Documents that bypass the WG process sometimes have huge impact on
the work we have to do (Glenn Parsons)
o Some consider IAB architectural guidelines/question to be
"meddling" by "outsiders" (Melinda Shore)
o Meetings are too big, with too many newcomers and tourists
diluting the technical discussion, because of sociological
effects. However the open-access nature of the IETF must be
maintained at all costs (Brian Carpenter: issue 8)
A.8 Reform process issues
Summary:
TBD
Issues derived from Mailing list:
o We don't have well-defined success criteria, and don't measure the
ones we could have (Bernard Aboba)
o We should act like the engineers we claim to be and measure things
to determine if there are problems (Frank Kastenholz)
o Trying to measure the progress of the IETF doesn't gather enough
interest to be worth it (Marshall Rose)
Davies (ed.) Expires August 25, 2003 [Page 34]
Internet-Draft IETF Problem Statement February 2003
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP-11. Copies of
claims of rights made available for publication and any assurances of
licenses to be made available, or the result of an attempt made to
obtain a general license or permission for the use of such
proprietary rights by implementors or users of this specification can
be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assignees.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
Davies (ed.) Expires August 25, 2003 [Page 35]
Internet-Draft IETF Problem Statement February 2003
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
Davies (ed.) Expires August 25, 2003 [Page 36]
