Problem Statement                                        E. Davies (ed.)
Internet-Draft                                           Nortel Networks
Expires: August 25, 2003                               February 24, 2003


                         IETF Problem Statement
               draft-ietf-problem-issue-statement-00.txt

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at http://
   www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on August 25, 2003.

Copyright Notice

   Copyright (C) The Internet Society (2003). All Rights Reserved.

Abstract

   This memo summarizes perceived problems in the structure, function
   and processes of the Internet Engineering Task Force (IETF).  We are
   attempting to identify these problems, so that they can be addressed
   and corrected by the IETF community.

   The problems have been digested and categorized from an extensive
   discussion which took place on the 'problem-statement' mailing list
   from November 2002 to February 2003.  The problem list has been
   further analyzed by an editorial team, and is provided as input to
   the Problem Statement working group.






Davies (ed.)            Expires August 25, 2003                 [Page 1]


Internet-Draft           IETF Problem Statement            February 2003


Table of Contents

   1.    Introduction: Issues/Problems in the IETF process  . . . . .  3
   1.1   Consequences of Past Growth  . . . . . . . . . . . . . . . .  3
   1.2   The Aim is Improvement, not Finger-pointing  . . . . . . . .  4
   1.3   Acknowledgements . . . . . . . . . . . . . . . . . . . . . .  4
   2.    Root Cause Problems  . . . . . . . . . . . . . . . . . . . .  6
   2.1   The IETF does not have a common understanding of its
         Mission  . . . . . . . . . . . . . . . . . . . . . . . . . .  6
   2.2   The IETF does not use Effective Engineering Practices  . . .  7
   2.3   IETF contributors appear to be less engaged than in
         earlier days . . . . . . . . . . . . . . . . . . . . . . . .  8
   2.4   Authority and Influence in the IETF are concentrated in
         too few hands  . . . . . . . . . . . . . . . . . . . . . . .  9
   2.5   IETF Decision making processes are flawed  . . . . . . . . . 10
   2.6   IETF Participants and Leaders are inadequately trained . . . 10
   3.    Security Considerations  . . . . . . . . . . . . . . . . . . 11
   4.    IANA Considerations  . . . . . . . . . . . . . . . . . . . . 12
         References . . . . . . . . . . . . . . . . . . . . . . . . . 13
         Author's Address . . . . . . . . . . . . . . . . . . . . . . 13
   A.    Summarization of Problems from Mailing List  . . . . . . . . 14
   A.1   Problem area/subject matter issues . . . . . . . . . . . . . 14
   A.2   Working Group process issues . . . . . . . . . . . . . . . . 14
   A.2.1 Who the participants are and represent . . . . . . . . . . . 15
   A.2.2 How the consensus process works  . . . . . . . . . . . . . . 16
   A.2.3 How the interaction works  . . . . . . . . . . . . . . . . . 19
   A.2.4 Measuring the outcome  . . . . . . . . . . . . . . . . . . . 20
   A.2.5 Lack of Transparency: Design Teams and Interim meetings  . . 21
   A.2.6 Problems in the Large - Cross fertilization between
         WGs/Areas and System level Thinking  . . . . . . . . . . . . 21
   A.2.7 Quality control issues - Documents . . . . . . . . . . . . . 22
   A.2.8 Quality control issues - Personnel . . . . . . . . . . . . . 24
   A.2.9 Timing/Latency . . . . . . . . . . . . . . . . . . . . . . . 25
   A.3   IETF organizational and Perception issues  . . . . . . . . . 27
   A.3.1 The Old Boy network  . . . . . . . . . . . . . . . . . . . . 27
   A.3.2 External Perception of the IETF  . . . . . . . . . . . . . . 28
   A.3.3 Loose control of IETF over its own work  . . . . . . . . . . 29
   A.3.4 Money  . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
   A.3.5 Relationship to ISOC . . . . . . . . . . . . . . . . . . . . 31
   A.4   IETF management issues (non-WG)  . . . . . . . . . . . . . . 31
   A.5   IESG processing problems . . . . . . . . . . . . . . . . . . 32
   A.6   Tools  . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
   A.7   Community review and coordination problems . . . . . . . . . 33
   A.8   Reform process issues  . . . . . . . . . . . . . . . . . . . 34
         Intellectual Property and Copyright Statements . . . . . . . 35






Davies (ed.)            Expires August 25, 2003                 [Page 2]


Internet-Draft           IETF Problem Statement            February 2003


1. Introduction: Issues/Problems in the IETF process

   Discussions over the last couple of months have revealed a
   significant number of perceived problems with the way the Internet
   Engineering Taskforce (IETF) operates. In advance of trying to change
   the IETF procedures and rules to deal with these problems, the IETF
   should have a clear, agreed-upon description of what problems we are
   trying to solve.

   The Problem Statement working group was charged with producing the
   document describing those problems. This document attempts to fulfill
   that aim.

   The document was derived by summarizing the extensive discussions
   which took place initially on the wgchairs mailing list and
   subsequently on the 'problem-statement' mailing list from November
   2002 through to February 2003, and incorporating additional input
   from relevant drafts published during this period (see [1], [2] and
   [3]) and the minutes of recent plenary discussions. This produced a,
   still extensive, list of perceived problems which were classified
   into a number of related groups which is included in this document as
   Appendix A. Note that the classification as it stands in this draft
   was suggested by the processes which go on in the IETF.

   For each group of problems collected in Appendix A, the set of
   perceived problems have been further reduced to a paragraph or so of
   text which seeks to encapsulate the issues for that group.

   The final part of the editorial team work has been to encapsulate
   these perceived problems into a small set of root cause issues, and a
   short list of subsidiary issues which appear to be the most pressing
   items engendered by the root cause.  This list is set out in Section
   2.

   Section 1.1 gives a short explanation of the teams' thinking in
   coming to the current view of the root causes.

   The team has deliberately not reclassified the perceived problems in
   line with our view of the root causes: This is because this a first
   cut at deriving the root causes and other people may wish to take a
   different view of the base material.

1.1 Consequences of Past Growth

   As we examined the long list of problems summarized in Appendix A, it
   became clear that the problems of the IETF are neither new nor are
   they symptoms of a problem which is novel in the science of
   organizations.



Davies (ed.)            Expires August 25, 2003                 [Page 3]


Internet-Draft           IETF Problem Statement            February 2003


   The IETF started off as a small, focused organization with a clearly
   defined mission and participants who had been working in this area
   for a significant period of time. Over the period 1989-1999 the IETF
   grew by a factor of ten or more in terms of number of participants,
   and in terms of work in progress.  Many of the problems and symptoms
   appear to be fundamentally caused by the organization failing to
   adapt its management and processes to its new larger size, and
   failing to clearly define its future mission once the initial mission
   had been completed or outgrown.

   These failures are just those that afflict many small organizations
   trying to make the transition from a small organization which can be
   run informally and where essentially all participants fully share the
   aims, values and motivations of the leadership, to a medium sized
   organization where there are too many participants for informal
   leadership and later arrivals are not as clear about the ethos of the
   organization.

   Some IETF participants have been aware of these issues for a long
   time. Editorial team members were able to produce evidence dating
   back to 1997 and 1992 that drew similar conclusions.

1.2 The Aim is Improvement, not Finger-pointing

   We believe and acknowledge that all of the parties in the IETF are
   acting in good faith, both in the normal operations which are
   critiqued in this memo, and in the review process itself. No blame
   for any of the perceived problems should be directed to any
   individual, and the sole aim of the review process is to identify how
   the IETF can improve itself so that it knows what it is about and
   becomes fit for that purpose in the shortest possible timeframe.

1.3 Acknowledgements

   Apart from the contributions of all those who provided input on the
   problem statement mailing list, the final reduction of the problems
   was assisted by the editorial review board consisting of:

      Avri Doria <avri@apocalypse.org> (WG co-chair)
      Dave Crocker <dcrocker@brandenburg.com>
      Jeanette Hoffmann <jeanette@wz-berlin.de>
      Margaret Wasserman <mrw@windriver.com>
      Melinda Shore <mshore@cisco.com> (WG co-chair)
      Rob Austein <sra@hactrn.net>.
      Spencer Dawkins <sdawkins@cynetanetworks.com>

   Special thanks are due to Margaret Wasserman for extensive reviewing
   and contributions to the wording of Section 2.



Davies (ed.)            Expires August 25, 2003                 [Page 4]


Internet-Draft           IETF Problem Statement            February 2003


   The early part of the reduction of the problem statement mailing list
   input was done by Harald Alvestrand and the latter part by Elwyn
   Davies and the editorial team. In total there were something like 750
   extensive and thoughtful contributions (some making several points).
   The thread was started by a call for volunteers in helping draft a
   problem statement, but quickly turned into a discussion of what the
   problems were.












































Davies (ed.)            Expires August 25, 2003                 [Page 5]


Internet-Draft           IETF Problem Statement            February 2003


2. Root Cause Problems

   This section forms the heart of this analysis, and lists the issues
   which we believe lie at the core of the problems. Apart from the
   first issue which is fundamental, the problems are not necessarily in
   priority order, but they will be seen to be interlinked in various
   ways.

2.1 The IETF does not have a common understanding of its Mission

   The IETF lacks a clearly defined and commonly understood Mission: As
   a result the goals and priorities for the IETF as a whole and any
   Working Groups (WGs) that are chartered are also unclear.

   The lack of a common mission has many consequences, of which the
   principal ones appear to be:

   o  The IETF is unsure what it is trying to achieve and hence cannot
      know what its optimum internal organization should be to achieve
      its aims.

   o  The IETF cannot determine what its 'scope' should be, and hence
      cannot decide whether a piece of proposed work is either in-scope
      or out-of-scope.

   o  The IETF is unsure who its customers are, and consequently has
      managed to drive certain classes of customer, who would otherwise
      provide important input to the process, nore or less completely
      out of the process.

   o  Working Groups can potentially be hijacked by sectional interests
      to the detriment of the IETF's reputation.

   o  The misty vision has restricted the associated architectural view
      to an outline top level view.  It would be desirable to have
      roadmaps and architectural views for portions of work which extend
      beyond a single working group:  It may also be the case that it is
      no longer possible to fit the whole Internet within a single
      architecture

   o  The lack of precision regarding our goals leads to WG charters and
      requirements that are poorly thought out and/or not aligned with
      the overall architecture.

   The IETF has previously stated that it is not and does not wish to
   become a Standards Determining Organization (SDO) but it is widely
   perceived externally as another SDO and compared unfavorably with
   other SDOs.  The differentiation and the rationale for it are not



Davies (ed.)            Expires August 25, 2003                 [Page 6]


Internet-Draft           IETF Problem Statement            February 2003


   generally understood but this does not mean that all the unfavorable
   comparisons are invalid, and several of them are addressed in the
   next section.

2.2 The IETF does not use Effective Engineering Practices

   For an organization with 'engineering' in its title and participants
   who are likely to trot out the statement "Trust me, I'm an engineer!"
   when confronted with the need to find a solution to a particularly
   knotty problem, the IETF has extremely ineffective Engineering
   Practices.

   The apparent inability of the IETF to deliver specifications within
   the timeframe that the markets need and the excessive perfectionism
   that is exhibited in some cases could both be improved if appropriate
   Engineering Practices were in use.

   Engineering requires appropriate trade-offs: Engineering success
   needs refinement only to the point of 'fitness for purpose' which
   should help to balance the tension between time to market and
   perfectionism. The use of appropriate Engineering Practices should,
   for example, prevent specifications being recycled in pursuit of
   perfection when they are already adequate improvements on the status
   quo.

   Some of the key areas where the IETF's practices appear to need
   tightening up include:

   o  Lack of explicit quality auditing throughout the standards
      development process.

   o  Lack of written guidelines or templates for the content of
      documents (as opposed to the overall layout) and matching lists of
      review criteria.

   o  Poorly defined success criteria for WGs and individual documents.

   o  Lack of criteria for determining when a piece of work is
      overrunning and/or is unlikely to be concluded successfully either
      at all or within an acceptable time frame: Lack of process for
      either extending the time frame, adjusting the scope or
      terminating the work item or the whole Working Group.

   o  Tools to support the engineering process are minimal.

   o  The IETF explicitly avoids developing test tools for verifying
      that protocols meet its specifications.




Davies (ed.)            Expires August 25, 2003                 [Page 7]


Internet-Draft           IETF Problem Statement            February 2003


   In addition, IETF processes, and Working Group processes in
   particular, suffer because commonly accepted Project Management
   techniques are not regularly applied to the progress of work in the
   organization.

   o  Project entry, goal setting and tracking processes are all either
      missing or implemented less effectively than the norm for
      commercial organizations in related activities.

   o  Charters regularly fail to set sufficiently granular milestones at
      which progress of WGs, individuals and documents can be evaluated.

   o  Better estimation procedures need to be used to determine what the
      expected delivery timescale for Working Group outputs should be.
      These estimates must be compared with the acceptable market and
      customer time frames for the work to be ready, and the scope of
      the work adjusted if timely delivery appears impossible.  This
      process must be repeated from time to time during the project.

   Finally, even where the IETF does have Engineering Practices defined,
   there are frequently cases where they are ignored or distorted.

2.3 IETF contributors appear to be less engaged than in earlier days

   There are a number of respects in which IETF participants and
   contributors appear to have become less fully engaged with the IETF
   processes than previously, for example:

   o  Although there may be large attendances at many WG meetings, in
      many cases 5% or less of the participants have read the drafts
      which are under discussion or have a bearing on the decisions to
      be made.

   o  Commitments to write, edit or review a document are not carried
      out in a timely fashion.

   o  Little or no response is seen when a request for 'last-call'
      review is issued either at WG or IETF level.

   Whilst this might be put down to contributors having less time
   available in their work schedule during the downturn of the last two
   years, this is not the whole story because there were signs of this
   malaise back at the height of the boom in 2000.

   This problem exacerbates the problems which the IETF has had with
   timely delivery and may weaken the authority of IETF specifications
   if decisions are seen to be taken by badly informed participants and
   without widespread review.



Davies (ed.)            Expires August 25, 2003                 [Page 8]


Internet-Draft           IETF Problem Statement            February 2003


2.4 Authority and Influence in the IETF are concentrated in too few
    hands

   It appears that both authority and influence in the IETF are
   concentrated in too few hands, and those with authority (primarily
   the Internet Engineering Steering Group (IESG)) are insufficiently
   accountable for some of their actions.

   The IETF appears to have created a 'ruling class' system which tends
   to re-select the same leaders from a limited pool of people who have
   proved competent and committed in the past.  Members of this 'ruling
   class' tend to talk mainly to each other and former members of the
   'ruling class'.  Newcomers to the organization and others outside the
   'ruling class' are reluctant to challenge the apparent authority of
   the extended 'ruling class' during debates and consequently influence
   remains concentrated in a relatively small group of people.  This
   reluctance may also be exacerbated if participants come from a
   different cultural background than the dominant one.

   The management and technical review processes currently in place were
   adequate for the older, smaller organization, but are apparently not
   scalable to the current size of the organization. The reliance of the
   existing process on the small number of people who have authority in
   the IETF both slows up the process and limits the number of formally
   recognized 'preferred' positions within the IETF, thereby limiting
   the (intangible) rewards for participants.  This can lead to useful
   and effective participants leaving because they cannot obtain any
   recognition (the only currency the IETF has to pay participants),
   which they use to fuel their own enthusiasm and help justify their
   continued attendance at IETF meetings to cost constrained employers.

   The current IESG processes allow one (or two) people to block or veto
   the work put together and approved by the many in a Working Group,
   possibly without good reason being given.  This can happen in two
   ways:

   o  IESG members who fail to put work on the IESG agenda. The
      'responsible Area Director (AD)' for a WG can hold up progress in
      the WG indefinitely, with no way for the WG to bypass the AD short
      of using the formal appeal or recall processes, both of which
      signal such a desperate breakdown in relations that, in practice,
      no WG has ever attempted to use them to overcome this sort of
      unreasonable delay.

   o  Documents can be blocked by one AD tabling a 'DISCUSS' issue
      regarding a document.  Although a mechanism exists whereby the
      whole IESG can override a single AD's DISCUSS, any two ADs can
      block a piece of work indefinitely.



Davies (ed.)            Expires August 25, 2003                 [Page 9]


Internet-Draft           IETF Problem Statement            February 2003


   Apart from the frustration that this can cause inside the
   organization, and the perception of disunity from outside the
   organization, work which has been properly authorized as being within
   scope of the IETF and properly quality checked during development
   should almost never come up against such a blockage.

2.5 IETF Decision making processes are flawed

   The IETF appears to be poor at making timely and reasonable decisions
   that can be guaranteed to be adhered to during the remainder of a
   process or until shown to be incorrect.

   Participants are frequently allowed to re-open previously closed
   issues just to replay parts of the previous discussion without
   introducing new material.  This may be either because the decision
   has not been clearly documented or it may be a maneuver to try to get
   a decision changed because the participant did not concur with the
   consensus originally.  In either case revisiting decisions stops the
   process moving forward, and in the worst cases can completely derail
   a working group.  On the other hand, the decision making process must
   allow discussions to be re-opened if significant new information
   comes to light or additional experience is gained which appears to
   justify alternative conclusions for a closed issue.

2.6 IETF Participants and Leaders are inadequately trained

   Participants and leaders at all levels in the IETF need to be taught
   the principles of the organization (Mission and Architecture(s)) and
   trained in carrying out the processes which they have to use in
   developing specifications, etc.

   The IETF currently has voluntary and inconsistent processes for
   training its participants which may be why significant numbers of
   participants seem to fail to conform to the proper principles when
   working in the IETF context.

   The people in authority have generally been steeped in the principles
   of the IETF (as they see them) and first-time non-compliance by newer
   participants is sometimes treated as an opportunity for abuse rather
   than by recognition of a training failure.

   Lack of training compounded with concentration of influence in the
   'ruling class' can lead to newcomers being ignored during
   discussions, consequently being ineffective either in their own eyes
   or their employers and so leaving the IETF.






Davies (ed.)            Expires August 25, 2003                [Page 10]


Internet-Draft           IETF Problem Statement            February 2003


3. Security Considerations

   This document does not of itself have security implications, but it
   may have identified problems which raise security considerations for
   other work.  Any such implications should be considered in the
   companion document which will be produced setting out how the IETF
   should set about solving the problems identified.












































Davies (ed.)            Expires August 25, 2003                [Page 11]


Internet-Draft           IETF Problem Statement            February 2003


4. IANA Considerations

   There are no IANA considerations defined in this document.
















































Davies (ed.)            Expires August 25, 2003                [Page 12]


Internet-Draft           IETF Problem Statement            February 2003


References

   [1]  Huston, G. and M. Rose, "A Proposal to Improve IETF
        Productivity", draft-huston-ietf-pact-00 (work in progress),
        October 2002.

   [2]  Blanchet, M., "Suggestions to Streamline the IETF Process",
        draft-blanchet-evolutionizeietf-suggestions-00 (work in
        progress), November 2002.

   [3]  Hardie, T., "Working Groups and their Stuckees",
        draft-hardie-wg-stuckees-00 (work in progress), February 2003.


Author's Address

   Elwyn B. Davies
   Nortel Networks
   Harlow Laboratories
   London Road
   Harlow, Essex  CM17 9NA
   UK

   Phone: +44 1279 405 498
   EMail: elwynd@nortelnetworks.com


























Davies (ed.)            Expires August 25, 2003                [Page 13]


Internet-Draft           IETF Problem Statement            February 2003


Appendix A. Summarization of Problems from Mailing List

A.1 Problem area/subject matter issues

   Summary:
      These problems are fundamentally caused by the lack of
      well-defined mission, and consequent effects, such as lack of
      priorities and goals, and undefined boundaries to the scope of
      problems which the IETF will tackle.

   Issues derived from Mailing list:

   o  The IETF is not really sure of its mission (Margaret Wasserman)

   o  At the moment, we collectively accept almost any working group,
      and allow them to produce almost any result. (Joel Halpern)

   o  WG charters and consequential requirements are not tightly defined
      at the beginning of the process (Marc Blanchet: P3)

   o  I-Ds that are of the "wouldn't it be a good idea if this worked"
      type confuse the process (Eric Burger)

   o  Insularity is unavoidable given the large number of working groups
      (Melinda Shore)

   o  There are no roadmaps for areas or other large chunks of the
      architecture, just for WG efforts (David Harrington and others)

   o  Too little architectural guidance in the IETF (Carl-Uno Manros)

   o  Do we really know who are customers are? May have become
      irrelevant to some hardware manufacturers (Dean Willis)

   o  Do we produce (sufficiently) interesting specifications?

   o  It is impossible to define a single architecture for the Internet
      (Brian Carpenter)


A.2 Working Group process issues

   The major process within the IETF is the lifecycle of Working Groups
   (WG) and the standards documents which they are chartered to produce.
   This section contains problems which appear to relate to various
   aspects of the WG process





Davies (ed.)            Expires August 25, 2003                [Page 14]


Internet-Draft           IETF Problem Statement            February 2003


A.2.1 Who the participants are and represent

   Summary:
      The IETF is often described as a volunteer organization but that
      is really a myth, but with honorable exceptions among the
      independents and academics. The great majority of participants are
      paid by their employers to attend the IETF, and are directly or
      indirectly rewarded by those employers for 'success' within the
      IETF.

      The IETF differs from other SDOs (whether the IETF wishes to class
      itself as an SDO or not is irrelevant here - people outside the
      IETF treat it as an SDO) in that although an employer may support
      a participant, the employer has no say in roles which a
      participant may be offered.  It is up to the the individual
      particpant to carve out a reputation for him/herself by initially
      volunteering for tasks in the WGs and later being tagged by an AD
      as a WG chair or for assistance in an Area or selected by the
      NOMCOM for membership of the IESG or Internet Architecture Board
      (IAB).  Many participants use significant amounts of their own
      time as well as their employer's time in order to achieve
      'success' but the average IETFer is probably more of a 'pressed
      man' than a volunteer.

      Most participants are initially selected to attend the IETF with a
      view to the individual helping to ensure that some standard of
      value to the employer is created and are supported at the IETF
      with this end in view: To that extent, at least, they are beholden
      to their employers so that the work that they carry out in the
      IETF will generally support their employer's business, even when
      the IETF tries to maintain the fiction that all participants
      attend as individuals and their company affiliations are left
      outside the hall.

      This situation has a number of problems for the IETF:

      *  Achieving true consensus requires that individuals can act
         truly independently, ignoring company mandates, which may be
         impossible for many more junior participants.

      *  The IETF clearly does not employ any of the participants and
         hence lacks the ultimate sanctions (financial penalty or
         employment loss) that an employer can use to control and direct
         employees.  This means that enforcing deadlines and controlling
         behavior is extremely difficult (often referred to as the 'cat
         herding problem') and it tends to make managers less inclined
         to delegate ('never delegate anything you don't have time to do
         yourself' in case the delegated task doesn't get done):  In



Davies (ed.)            Expires August 25, 2003                [Page 15]


Internet-Draft           IETF Problem Statement            February 2003


         this respect the IETF does resemble a volunteer organisationm.

      *  The IETF is normally unwilling (unable?) to challenge an
         individual who is misusing the freedom of the IETF to push a
         company mandated position against the better interests of the
         community, even if instances of misuse can be identified.

      *  The very diverse backgrounds of those now attending the IETF
         may make it much more difficult to arrive at consensus.

      Even where employers remain reasonably altruistic in their support
      of the IETF, recent market downturns have reduced the amount of
      time that some employees are allowed to spend on IETF activities,
      and the number of employees allowed to attend meetings has been
      shrinking.

      On the other hand, there are a limited number of people who have
      by prolonged, effective service reached position of ongoing
      influence and personal authority in the IETF world.  Because the
      IETF is an elite organization, many of these people do not have
      other fora to which they could 'graduate', and hence remain active
      in the IETF. Many less experienced IETFers appear to be in awe of
      these veterans, and this can lead to undue deference and
      unwarranted acceptance of their positions. Some employers seem
      willing to offer such people posts which allow them to continue to
      devote significant amounts of effort to IETF activities: It is
      clear that they can have an exceptional influence on IETF work,
      and there could be an avenue for companies to exploit this
      inappropriately.

      Finally, some potentially important participants (e.g. staff from
      network operators) also appear to being driven out by the IETF's
      apparent disregard for their interests. This problem is related to
      the failure of the IETF to acknowledge who its customers are, as
      noted in Appendix A.1.

   Issues derived from Mailing list:

   o  Individuals' behavior has changed (less commitment)? (Aaron Falk)

   o  Less consensus on what parts of the IETF's work is "core" and
      "critical" (Aaron Falk)


A.2.2 How the consensus process works






Davies (ed.)            Expires August 25, 2003                [Page 16]


Internet-Draft           IETF Problem Statement            February 2003


   Summary:
      The rough consensus mechanism that is used to settle the output of
      the WGs is the key distinguishing factor of the IETF. The use of
      rough consensus together with resistance to producing standards
      with 'profiles' or multitudinous options results in IETF
      specifications which generate multiple independent implementations
      characterized by maximal commonality of features.

      The IETF has grown up from a relatively small organization in
      which essentially all participants shared a common purpose and
      ethos, understanding the principles and the underlying
      architecture which underpins the whole enterprise.

      Over the last few years, the IETF has grown in various ways:

      *  The IETF is producing standards for more than a single
         constellation of activities.  Diverse architectures may be
         needed and diverse opinions are bound to arise, and people
         operating in one constellation may not understand the
         motivations and requirements of others.

      *  The IETF is consequently larger with many more participants and
         more WGs in progress at any one time.

      *  The IETF is no longer a totally English language or
         overwhelmingly North American culture based entity.
         Participants with alternative mother tongues and home cultures
         make up a significant part of the participants.

      *  The influence of the IETF has grown as the Internet and private
         networks using the same protocols have become all pervasive.
         The IETF is now consulted by and involved in liaisons with many
         other SDOs and related organizations.

      This growth has resulted in the IETF demonstrating many of the
      same strains that conventional commercial organizations undergo
      when they grow from the 'small' category into the 'medium'
      category.

      The consensus process and all that underlies it were initially
      part of the shared ethos of the early cohorts of IETF
      participants.  The advent of large numbers of newer participants,
      who have never really been taught the full principles of the
      organization, probably detracts from the ability to achieve a true
      'rough consensus' which is the 'best' engineering solution given
      the technical and political constraints.  The limited
      'Introduction to the IETF' session given at each of the meetings
      is voluntary and is probably not sufficient to drive an ongoing



Davies (ed.)            Expires August 25, 2003                [Page 17]


Internet-Draft           IETF Problem Statement            February 2003


      mindset, particularly if attendees have been used to some other
      ethos.

      Notwithstanding the primacy of discussion and consensus on the
      mailing list, it is clear that some decisions are made in less
      than ideal circumstances at meetings where observers participate
      in hums or votes where it is not  necessarily clear that they
      fully understand the issue. Similarly, some participants with
      different cultural backgrounds may be effectively disenfranchised
      by unclear and totally verbal processes at such meetings.

      Achieving consensus has tended to become a tail end operation
      rather than a continuing process, particularly where there are
      competing (semi-proprietary) alternatives.  This leads to multiple
      options or development strands being pursued for too long in the
      WG (contrary to the basic tenets of the IETF) and the whole
      process is slowed down by dilution of effort and ongoing dispute.

   Issues derived from Mailing list:

   o  Consensus process is not working, or is too slow, due to lack of
      interest in the common good (Melinda Shore)

   o  What consensus means has been lost, or the spirit of consensus is
      missing (Edward Lewis)

   o  Discussions in WGs turn into battles along "party lines" between
      companies, not an effort towards interoperability (Edward Lewis)

   o  Heavy investment in alternatives pre-standardization leads to
      vested interest in outcomes, preventing consensus when there is no
      clear technical lead

   o  Rat holing over fundamental disagreements on the goals of the
      process leads to bog-down; requirements documents can sometimes
      help avoid this (Tony Hain)

   o  WG process is too slow, because of feeping creaturism, deadlocked
      conflicts or lack of worker bandwidth (Jari Arkko)

   o  Waiting for WG meetings at IETF meetings to identify/reach
      consensus sometimes delays the advancement of the WG - this
      happens even though the mailing list should be the prime
      discussion forum. Judging consensus based on the mailing list
      comments sometimes obscures the silent majority opinion (Marc
      Blanchet: P4) and sometimes reflects exhaustion rather than true
      consensus (Elwyn Davies).




Davies (ed.)            Expires August 25, 2003                [Page 18]


Internet-Draft           IETF Problem Statement            February 2003


   o  It's not possible to identify the time when major changes to an
      I-D set are unlikely, so that it's reasonably safe to start
      designing implementations (James Luciani)

   o  Cultural issues can lead to certain constituencies being sidelined
      (Marc Blanchet: P10/Elwyn Davies)

   o  The IETF consensus process appears not to be good at selecting
      between two or more reasonably well-qualified alternatives.
      Standardizing several options tends to reduce market acceptance
      (MPLS CR-LDP/RSVP-TE) or cause long delays (SNMPv2). IETF doesn't
      do compromises with all the options (as do some SDOs) normally
      (and it shouldn't probably). (Margaret Wasserman)


A.2.3 How the interaction works

   Summary:
      Mailing list discussions are not necessarily the 'sine qua non'
      for achieving IETF style rough consensus.  Active involvement of
      the WG chair(s) can help to keep discussions on track and polite,
      but an inflamed discussion in a large WG causes mail overload for
      most participants.

      Breaking the deadlock between 'religiously' held positions is
      extremely difficult and is a major cause of the failure to achieve
      consensus.

   Issues derived from Mailing list:

   o  Mailing lists are not (always) an effective tool (Aaron Falk)

   o  Large groups do not interact well enough to form consensus (James
      Kempf)

   o  Large WGs means that we get more groupings of intransigent people,
      leading to difficulty with consensus (Donald Eastlake)

   o  Mailing lists with large numbers of postings (500 in a week, for
      instance) are hard for people to follow (Thomas Narten)

   o  People sticking to fixed positions on mailing lists leads to lots
      of mail but no consensus (James Kempf)

   o  Maintaining decorum and forward progress on a mailing list is not
      easy (many)

   o  Consensus tests at WG meetings by means of 'hum' responses to



Davies (ed.)            Expires August 25, 2003                [Page 19]


Internet-Draft           IETF Problem Statement            February 2003


      verbal questions are often unclear and lead to bad choices (Marc
      Blanchet: P9)

   o  Non-English speakers find it easier to read questions rather than
      be sure they understand a spoken question (Marc Blanchet: P10)

   o  Large teleconferences have not been tried. (Jari Arkko)


A.2.4 Measuring the outcome

   Summary:
      The IETF needs a strategy to go with its mission (when it is fully
      spelled out, see Appendix A.1). At present the IETF does not
      include success criteria for standards which it designs beyond the
      purely technical. This may be a problem because the customers
      criteria are neither perfection nor purely technical effectiveness
      - business utility or fitness for purpose plays a part as well.

      The alienation of certain types of commercial organization (e.g.
      operators) from the IETF means that they do not assist in
      determining the success factors at the time the standards are
      being developed.

   Issues derived from Mailing list:

   o  [How much do we lose from the 'travel filter' and the silent
      majority [Editor] ] If a person/company does not have money to
      travel to a meeting, is it likely that the company will have the
      money to implement the protocol or make it a commercial success?
      What might need to be changed to better achieve RFC2418 existing
      advice? (Margaret Wasserman/Dave Crocker)

   o  IETF traditionally not accepting model of vendors collaborating to
      get commercial fix up, rather open collaboration to do the 'best
      thing' (Eric Rescoria)

   o  What is success? Lots of people talking relevance...maybe just the
      same as 'commercial success' but may also involve elements of
      quality and timeliness. Is it better that we use the official
      model of 'open, fair, technically sound and useful'. (Margaret
      Wasserman)

   o  Large vendors may not have the most appropriate experience
      (Margaret Wasserman)

   o  Difference between making incremental upgrades to existing
      protocols etc where incumbents have experience/influence vs. new



Davies (ed.)            Expires August 25, 2003                [Page 20]


Internet-Draft           IETF Problem Statement            February 2003


      services/protocols where startups have at least as much experience
      and may be who is going to execute (Henning Schulzrinne)

   o  High quality has not necessarily been the answer, more 'just
      enough quality delivered in a timely fashion' (Marshall Rose)

   o  Must also be extensible (Scott w Brim)


A.2.5 Lack of Transparency: Design Teams and Interim meetings

   Summary:
      The outputs from design teams and interim meetings appear to have
      a disproportionate chance of becoming the final output of a WG.
      It appears to be difficult to make really significant changes to a
      document which has emerged from either of these processes or, in
      the worst case, to have the output of a design team discarded and
      totally reworked.  We know that in principle the documents have no
      greater validity than any other but the combination of the
      apparent waste of resource and the necessity of making progress
      tends to militate against them being subject to major savaging.

   Issues derived from Mailing list:

   o  Design Teams introduce lack of transparency. They are perceived as
      a way to bypass the normal working of the WG, to push forward the
      opinions of a (self-)selected subgroup and as closed fiefdoms
      which are not selected in an open fashion (Marc Blanchet: P6)

   o  Design team work is rarely challenged or subjected to external
      quality control by the rest of the community in the same way that
      more publicly constructed documents are tested (Elwyn Davies)

   o  Interim meetings are not as transparent as normal WG operations:
      Scheduling problems and travel constraints limit the attendees and
      reduce the input spectrum (Marc Blanchet; P7)


A.2.6 Problems in the Large - Cross fertilization between WGs/Areas and
      System level Thinking

   Summary:
      The IETF lacks a formal structure for handling complex problems
      that might require firstly to be analyzed into simpler problems
      and then to create an architecture, within the overall IETF
      architecture, which provides context for the solutions of the
      smaller problems, and will superintend the operation of one or
      more WGs set up to solve the individual smaller problems.



Davies (ed.)            Expires August 25, 2003                [Page 21]


Internet-Draft           IETF Problem Statement            February 2003


      WGs appear to be poor at doing real blank sheet design.

      WGs should be ready to bring in Subject Matter Experts from
      outside the central domain of a WG problem as soon as it becomes
      clear that there will be interactions with other areas, rather
      than at a late stage and only when beaten up by the ADs.

   Issues derived from Mailing list:

   o  IETF is not good at tackling large 'system level' problems where
      they need to be broken down into a number of smaller questions and
      the smaller questions given coordinated answers. This may need an
      extra type of organizational structure in the IETF to deal with
      complexity. (Elwyn Davies/Margaret Wasserman)

   o  WGs are rarely a good way to get from 'interesting buzzword' to
      'serious set of requirements and definitions' (John C Klensin)

   o  WGs are rarely good at doing real blank sheet design (either a
      good design shows up from outside the WG and is endorsed, maybe
      with fine tuning, or the WG cycles and produces garbage, which is
      approved because of exhaustion) (John C Klensin)

   o  Experts from areas outside the central domain of a WG problem are
      rarely brought in until late in the WG process which can lead to
      disconnect, reinvention of wheels, incompatibilities and missing
      pieces (Marc Blanchet: P5)


A.2.7 Quality control issues - Documents

   Summary:
      The IETF does not have a well-defined and well-documented quality
      control process during the normal operation of creating a document
      in a WG.

      Quality Control on WG process and documents is all head and tail,
      and no middle.  Nobody associated with the creation of a document
      is explicitly responsible for its quality.

      The need for three levels of standard maturity needs to be
      revisited, yet again, particularly with respect to the enforcement
      of existence of interoperable implementations at Proposed Standard
      level.

   Issues derived from Mailing list:





Davies (ed.)            Expires August 25, 2003                [Page 22]


Internet-Draft           IETF Problem Statement            February 2003


   o  Some (valid) comments get lost in the rough and tumble of a
      mailing list. Without a formal problem tracking system (which is
      used for some documents) it is difficult to ensure that a
      particular version of a document has addressed all the problems
      raised (Marc Blanchet: P8)

   o  WGs are not diligent enough in ensuring quality control (Randy
      Bush)

   o  Problems (for instance security) get discovered at IESG time, not
      earlier; Quality control on documents is mostly back-end loaded
      (to WG last call and IESG review) rather than being done at every
      stage (e.g. as on admission as a WG item) - this is known to
      create much more rework than checking at earlier stages. (Randy
      Bush/Bernard Aboba)

   o  Not enough guru bandwidth for appropriate security review (Derek
      Atkins, Steve Bellovin)

   o  Too little clarity and brevity in WG output (Aaron Falk)

   o  Lack of running code means that unimplementable specifications get
      on track (Kurt Zeilenga)

   o  There are a number of issues surrounding MIB modules (Bert Wijnen)

   o  'Damn the quality control torpedoes, full speed ahead for the
      standard' (open mike sentiment)

   o  We think there are cases where things go to Proposed Standard
      specifying things that can't work. One way to avoid that is to
      insist that someone try it before submission. (Harald Alvestrand,
      summarizing)

   o  Normative references to documents at earlier stages of
      standardization not allowed causes blockages in document
      publishing (John C Klensin)

   o  Should the IESG really need to review all informational RFCs
      (especially those not associated with WGs)? (Jari Arkko)

   o  The review checklists for RFCs (and implicitly I-Ds) are not very
      detailed: From my experience, inexperienced reviewer/auditors
      could use a much more detailed set of guidelines of things to
      check for (Elwyn Davies) It would be nice to know just what rules
      are applied for each maturity level (Resnick/Alvestrand)

   o  Are different standards of review possible at different stages of



Davies (ed.)            Expires August 25, 2003                [Page 23]


Internet-Draft           IETF Problem Statement            February 2003


      the standards process? (Margaret Wasserman) IN theory there is but
      it is not clear that this is in fact applied (i.e. PS reviewing
      may be more strict than might have been envisaged?) (Scott
      Bradner)

   o  If most of the Internet runs on protocols which have only reached
      Proposed Standard (PS) and the return on investment of taking a
      protocols to Draft or Full standard is seen as small, then why
      persist with a three level standards process? (Margaret Wasserman)

   o  Glorification of I-Ds and PS documents - often leading to lack of
      formal interoperability testing (James Kempf)

   o  Apparent presumption that almost any I-D that gets onto Standards
      Track in a WG will inevitably become the standard without
      substantial modification in any form of its basic design or
      contents (James Kempf) --- If this were really true IESG would
      have been defanged. (Brian Carpenter) -- possibly seeing results
      of design teams, AD associated influence and backdoor review. - Is
      IESG effect detrimental to WG effort (i.e. does the Old Boy
      network effectively denigrate work of WGs)

   o  [Solution note: Need to be able to document current utility of a
      protocol. features which are being used interoperably and feature
      not being used or being used non-interoperably - this might give a
      vehicle for the 'good enough' status. (Fred Baker)]

   o  ADs and IESG (apparently) do not provide significant architectural
      input between the BOF stage and the IESG review stage of a WG/
      document. There is no provision for a formal architectural review
      to be requested by either party, although presumably informal
      review could be done if requested. (Pete Resnick et al)


A.2.8 Quality control issues - Personnel

   Summary:
      Documents do not currently have a designated quality auditor to
      assist the  editors and authors. Quality auditors would be
      expected to be aware of the whole context in which the document
      will sit so that they can ensure cross-document consistency, and
      compliance with the relevant architecture which are often missing
      at present.

      Editors and authors do not always seem to be aware of what makes a
      quality (protocol standards) document. If they were it would be
      likely to speed up the document production process by minimizing
      rework at the outset.



Davies (ed.)            Expires August 25, 2003                [Page 24]


Internet-Draft           IETF Problem Statement            February 2003


      Any quality reviewing process for use during document creation
      that might be put in place needs to be lightweight (not hours of
      committee work) but effective. Authors need to be prepared to
      accept reasonable correction - 'ego  free document creation'!

      The aim of the WG reviewing process should be to make the IESG
      review a 'rubber stamp' formality, rather than the current 'battle
      of wills'.

   Issues derived from Mailing list:

   o  Authors/editors do not understand what makes a document Good; they
      should be trained (Avri Doria)

   o  WGs do not explicitly appoint quality reviewers for documents.
      Cross document reviewing is not much in evidence leading to
      inconsistencies (esp. terminology). (Elwyn Davies)

   o  WG chairs not adequately trained (need to understand process and
      tools available to them, e.g. milestone maintenance) (Marc
      Blanchet: P1)

   o  WG chairs should be more aware of who their peers are and interact
      more with them (Spencer Dawkins)

   o  WG chairs do not always have adequate time for task: All WGs need
      two chairs minimum (Marc Blanchet: P2)

   o  A single AD should not be able to effectively block a WG once it
      is started (Dave Crocker)

   o  Ensure all ADs attend all relevant interim meetings (John C
      Klensin)

   o  Ego clash between WG members and IESG: Provoked particularly by
      tail-end pronouncements from the IESG on-high requiring extensive
      modification of exquisitely crafted WG documents (Pete Resnick,
      Randy Bush, Melinda Shore)


A.2.9 Timing/Latency

   Summary:
      WG charters tend to contain milestones which are too far in the
      future to allow effective monitoring of progress against
      milestones, or milestones which prove to be too short when the
      actual work to be done has been measured.




Davies (ed.)            Expires August 25, 2003                [Page 25]


Internet-Draft           IETF Problem Statement            February 2003


      WG control by the ADs and the IESG does not tend to use modern
      management tools (such as management by exception) resulting in
      excessive load on the ADs and poor performance of the WGs against
      plans.

      Deadlines which are missed are currently either ignored or in the
      worst cases used as excuses to  shut a WG down.  Any missed
      deadline should be discussed between the WG chair(s) and/or
      document team and the AD(s) to decide if the deadline can be
      extended without compromising the usefulness of the result,
      whether the scope of the work should be scaled back to bring
      delivery closer or as a last resort the work should be scrapped>
      The WG should also be consulted.

   Issues derived from Mailing list:

   o  Documents evolve very slowly if they are only revised once per
      IETF meeting cycle: This is a matter of document editors/reviewers
      not necessarily having enough time as well as WG chairs hot
      forcing the issue (Aaron Falk) [But .. see the problem with
      volunteers [Editor]]

   o  WG Plans are too long/Milestones too far in the future: Checkpoint
      milestones (rather than completion milestones are rarely used)

   o  Milestones are set unrealistically without really understanding
      the work to be done. ADs encourage WG chairs to set short
      deadlines to get things out and then when they are not met they
      fall into disrepute (management truism) (Scott W Brim)

   o  WG plans/milestones ignored once set and not enforced either by WG
      chairs or ADs (Fred Baker)

   o  No firm time limits are set on WGs when they are created (e.g.
      produce something in 18 months or be deleted)

   o  Are WG meetings too short (many WGs that do good work find they
      need one or more interims)? How to get balance between good
      face-to-face time and mailing list time? (John C Klensin/Randy
      Bush)

   o  Longer meetings may push attendees into becoming 'standards
      drones' (John C Klensin)

   o  More expeditious closing down of WGs that aren't meeting deadlines
      (or going in circles) (many)

   o  Too many documents in each WG (but master documents in other



Davies (ed.)            Expires August 25, 2003                [Page 26]


Internet-Draft           IETF Problem Statement            February 2003


      groups are actually just combinations of all of ours)? (John C
      Klensin)

   o  Timeliness/competence curve changes shape after a couple of years
      - overall competence starts to drop off after that time (Marshall
      Rose)

   o  Improve feedback process to ensure that WG chairs will know there
      are consequences of not meeting deadlines (Marc Blanchet)

   o  Documents seem to be taking much longer to reach RFC status than
      they used to (roughly 3 times worse than when records were first
      kept). Associated with there being roughly three times as much
      work going on in IETF as then, this is probably an unsupportable
      burden.

   o  Milestones are unrealistic to the extent that they currently do
      not include any time for IESG review. However, although this could
      be considered to be a fixed quantity, it is relatively
      unpredictable because it would be dependent on the load at the
      time it was submitted for review and it would depend on the size
      and quality of the document submitted. However , if there are any
      dependencies in documents, clearly one must make some allowance
      for IESG review in order to get a realistic view of when a
      document or group of documents will be finished. It might also
      help to determine if the IESG is keeping up, and whether the
      workload being chartered was excessive..(Elwyn Davies)

   o  Measuring time in working groups is misleading - PACT focuses on
      what we know how to measure instead of what we care about
      (success/efficiency/quality?) (Joel M Halpern)


A.3 IETF organizational and Perception issues

A.3.1 The Old Boy network

   Summary:
      The IETF is a self-perpetuating oligarchy, containing a small
      population of 'big cheeses' who have a disproportionate influence
      on the outcomes of the IETF, and who are so deified that newer
      participants are overawed by them and consequently accept their
      words without demur.  WG chairs and ADs are sometimes perceived to
      fall into this category.

   Issues derived from Mailing list:





Davies (ed.)            Expires August 25, 2003                [Page 27]


Internet-Draft           IETF Problem Statement            February 2003


   o  The sociological structure of the IETF, despite the good offices
      of NOMCOM, is, IMHO, a self-perpetuating oligarchy (Margaret
      Wasserman, Elwyn Davies).

   o  There are a number of strong personalities who have come to have a
      possibly disproportionate influence. Some/Most of these are or
      have been Area Directors, but their influence tends to be
      pervasive and their views are rarely challenged and even more
      rarely overruled. The influence persists even when they are no
      longer in office. (Elwyn Davies).

   o  It often appears that many newer participants are overawed by the
      'big cheeses' and I have observed consensus overturned by their
      intervention. Sometimes this is justified by Wisdom of the
      Ancients, but on other occasions less so. (Elwyn Davies).

   o  The WG chairs and ADs are sometimes seen (or at least perceived)
      to use their status to push their own documents or proposals (Marc
      Blanchet: P11)


A.3.2 External Perception of the IETF

   Summary:
      The IETF is mostly seen by outsiders as a kind of Standards
      Determining Organization (SDO) even though the IETF tries not to
      be or act like an SDO. If it wishes to be seen as something else,
      then the IETF has to clearly set out what this is and how it will
      act differently from a conventional SDO to distinguish itself.
      Having decided on this, a publicity campaign will be needed to get
      the message across to supporters whilst putting across that it
      fulfills a useful role in its 'new' guise.

      For the time being, the IETF is compared with other SDOs and is
      not seen as the equal of other SDOs, especially by participants
      and leaders of these other SDOs. Apart from the desire to be seen
      as something other than an SDO, this may well be because of the
      different processes which the IETF uses. Consequently, the IETF
      needs to work to improve the quality of the work that comes out,
      avoid misuse of its processes and ensure that its work is
      delivered in a timely fashion, in order to maintain its influence
      on the Internet.

   Issues derived from Mailing list:

   o  The IETF is the standards body of last resort: If you can't get
      your standard accepted by some 'more reputable' body, then there
      is 'always the IETF'. (Nortel staff member via Elwyn Davies)



Davies (ed.)            Expires August 25, 2003                [Page 28]


Internet-Draft           IETF Problem Statement            February 2003


   o  The IETF is perceived as too slow: See latency discussion (Brian
      Carpenter: Issue 1)

   o  The IETF is perceived as unfocussed (Brian Carpenter: Issue 2).
      Related to IETF inability to deal with large scale architectural/
      system level problems (see above).

   o  IESG is too picky (won't charter my WG, won't approve my RFC, etc)
      (Brian Carpenter: Issue 3) [This is exact inverse of internal
      perception - see Joel Halpern's comment above]. The presence of
      both views may mean that the IESG is pretty much getting it right
      within its current remit.

   o  Operators perceive that the IETF neither understands nor
      particularly cares about their concerns. In general this means we
      don't hear enough from them (Brian Carpenter: issue 9)

   o  The IETF is perceived as doing near-term work very well but
      long-term work very badly (PACT draft, etc)


A.3.3 Loose control of IETF over its own work

   Summary:
      The IETF does not use adquate project management mechanisms to
      control the projects that are going on.

      The IETF does not have effective Engineering Practices.

      The fact that participants who are performing all sorts of tasks
      for the IETF are not employed by the IETF limits the control that
      the IETF has over the quality and timeliness of their work, since
      the IETF is not in a position to apply significant penalties for
      non-performance beyond 'loss of reputation', and this is not
      always reflected back into the participant's home organization.

      The IETF allows some of its processes (esp. publication of
      informational RFCs) to be used by people who are not fully
      integrated into the IETF structure:  This can result in excessive
      work loads from non-core activities on an already over- stretched
      management, and a risk of the system being abused by unscrupulous
      vendor mwrketeers.

   Issues derived from Mailing list:

   o  The 'openness' of the IETF can be abused by 'marketroids':
      Informational RFCs used as 'standards' by unscrupulous vendor
      marketeers, who feed proprietary proposals into Informational RFCs



Davies (ed.)            Expires August 25, 2003                [Page 29]


Internet-Draft           IETF Problem Statement            February 2003


      unless carefully monitored (Brian Carpenter: issue 6)

   o  The voluntary nature of IETF organization makes it difficult to
      ensure that the owner of any given task will actually have time
      for it (Brian Carpenter: Issue 7(i))

   o  The volume of work being undertaken by the IETF may be getting to
      the point where it is a crippling overload for a volunteer
      organization (Brian Carpenter: issue 7(ii))

   o  The voluntary nature of the IETF tends to discourage delegation in
      all those with any responsibility: "Never delegate anything which
      you haven't got time to do yourself" is not a phrase I've heard
      used around IETF but it is very easy to fall into this mode in an
      'all/mostly voluntary' organization if you have a job with a
      deadline and you don't really trust others to stick to the
      deadline. This tends to reinforce the sort of oligarchy mentioned
      above, where those who have been found to be efficient take a
      disproportionate amount of the work irrespective of their true
      competence. (Elwyn Davies)

   o  Termination reviews on expired-in-grade RFCs not being done on old
      PSs and DSs (Kurt D Zeilenga)

   o  The IETF needs more technical leadership, without moving to a
      dictatorial culture as in...

      *  IAB doesn't do enough proactive architecting (Rob Natale)

      *  IESG does not do enough steering especially in terms of
         proactive longer-term guidance of Area Direction and cross-Area
         interactions (Rob Natale)

      *  WG leadership is, on the whole, far more inclined to
         over-compensate on the side of democracy than on the side of
         sticking to the mission (Rob Natale)


A.3.4 Money

   Summary:
      To a very large extent, the IETF has endeavored to ignore the
      effects money on the network and its own operations. The IETF has
      tended to assume that the right people for the job will continue
      to be available to them to do the job without having to fund any
      people and without becoming beholden to the companies that fund
      its participants.




Davies (ed.)            Expires August 25, 2003                [Page 30]


Internet-Draft           IETF Problem Statement            February 2003


      The IETF may be limiting itself by being unable to deal with many
      of the inter-domain and service-related issues that are relevant
      to today's network because of the IETF's status viz-a-viz the (US)
      anti-trust laws.

   Issues derived from Mailing list:

   o  The Internet Society has recently (2000) created 'Platinum'
      membership to extract funds from companies to support standards
      work in IETF. The question is what 'quid pro quo' any such
      companies might wish to extract as output in return for their
      contribution. The IETF has so far managed with little or no
      financial constraint on its operations - this may be about to
      change. (Brian Carpenter: issue 10)

   o  The IETF is ham-strung by the anti-trust cruft that stops it
      addressing the inter-domain issues that are really at the heart of
      today's federated internet. (Elwyn Davies)

   o  Subject matter experts cannot afford to attend interim meetings:
      Should IETF be funding their expenses? (Margaret Wasserman)


A.3.5 Relationship to ISOC

   Summary:
      TBD

   Issues derived from Mailing list:

   o  Appointment of NOMCOM chair is a 'hidden process' (Brian
      Carpenter)

   o  Appointment of NOMCOM chair plus some money is limit of (not very
      visible) ISOC interactions (Brian Carpenter, Elwyn Davies)


A.4 IETF management issues (non-WG)

   Summary:
      TBD

   Issues derived from Mailing list:

   o  Predictability, Accountability, Competency and Timeliness are
      important to the IETF, and we don't have enough of them (draft by
      Marshall Rose, Geoff Huston)




Davies (ed.)            Expires August 25, 2003                [Page 31]


Internet-Draft           IETF Problem Statement            February 2003


   o  IESG and IAB spend precious time on political sludge. This cannot
      be avoided but it reduces IESG and IAB bandwidth: Choices made in
      the IETF change available (political) policy options and bad
      policy decisions outside the IETF can damage the technology, so we
      must be engaged (Brian Carpenter: issue 11)

   o  There is a risk of micro-management at all stages in the
      IAB->IESG->AD->WG chair->Editor/Author chain. This needs to be
      avoided whenever possible by suitable training (Brian Carpenter:
      issue 12)

   o  Need to introduce management by exception (Elwyn Davies)

   o  Need for general management training for IESG members (Margaret
      Wasserman)

   o  IESG is overloaded - need to find means of delegation and workload
      reduction (Margaret Wasserman)

   o  Possibly as an attempt to control marketroid Informational RFCs,
      the IESG is doing reviews on them for which they appear to have no
      mandate. Whilst this may be helping credibility, it is an extra
      load on the IESG, and appears to be holding up legitimate
      Informational RFCs as well as 'end-runs' (John C Klensin)

   o  Lack of Transparency of Management Functions

   o  Lack of transparency makes it hard to diagnose what the
      organization's problems are (Hilarie Ormond)

   o  IETF process is opaque, and therefore hard to coordinate with
      (Dean Willis)

   o  IESG is seen as opaque or arbitrary, at least from outside the Old
      Boy network. Apparent emulation of a black hole (in regard to
      document review particularly) in some cases can be frustrating
      (Brian Carpenter: issue 4(i))

   o  IAB is seen as somewhat opaque from outside the Old Boy network,
      although it is not a hurdle for document approvals (Brian
      Carpenter: Issue 4(ii))


A.5 IESG processing problems

   Summary:
      TBD




Davies (ed.)            Expires August 25, 2003                [Page 32]


Internet-Draft           IETF Problem Statement            February 2003


   Issues derived from Mailing list:

   o  Time from WG finished to IESG approval is too long

   o  Queuing delays inside the IETF process are far too long (Don
      Eastlake) (suggests that 1-2 months is most often a
      non-problematic delay, but that 1 year is a disaster)

   o  Poor quality documents tend to act as DoS attacks... sucking away
      process/review bandwidth from quality documents. (Kurt Zeilenga)

   o  For any X in the IETF process (WG, IESG, RFC Editor), it takes too
      long (David Meyer)

   o  The process' slowness means that I-Ds take on the force of a
      standard (Eric Burger)

   o  Too many steps in the overall IETF process: No real urgency to
      progress Proposed Standards to Draft/Full Standard with the result
      that internet mainly runs on Proposed Standards (where it is
      running on I-Ds or Informational RFCs) (Brian Carpenter: Issue 5)

   o  Not enough information for the WG chairs or authors to identify
      the IESG concern and effect the resolution in a timely manner
      (Dean Willis)

   o  Too many WGs in progress for IESG to handle reviewing in a timely
      fashion (Dave Crocker/PACT)


A.6 Tools

   Summary:
      The IETF is missing several useful tools that could assist with
      the management of WG processes.  A standard set of tools would
      facilitate progress and minimise set and learning time for each
      group and well-organised documentation for the existing tools and
      any new ones is essential.

   Issues derived from Mailing list:

   o  Need better map of tools available to WG chairs and document
      authors (Edward Lewis)


A.7 Community review and coordination problems

   TBD



Davies (ed.)            Expires August 25, 2003                [Page 33]


Internet-Draft           IETF Problem Statement            February 2003


   Issues derived from Mailing list:

   o  Architectural decisions with IETF-wide impact doesn't get adequate
      community review (Margaret Wasserman)

   o  Few people review drafts in IETF Last Call (Avri Doria)

   o  We're not using the three-stage standards process as designed
      (Spencer Dawkins)

   o  We as a community are not following the rules we set for ourselves
      (James Luciani)

   o  It's hard to get a discussion going when the IAB works on
      architectural considerations (Rohan Mahy)

   o  Documents that bypass the WG process sometimes have huge impact on
      the work we have to do (Glenn Parsons)

   o  Some consider IAB architectural guidelines/question to be
      "meddling" by "outsiders" (Melinda Shore)

   o  Meetings are too big, with too many newcomers and tourists
      diluting the technical discussion, because of sociological
      effects. However the open-access nature of the IETF must be
      maintained at all costs (Brian Carpenter: issue 8)


A.8 Reform process issues

   Summary:
      TBD

   Issues derived from Mailing list:

   o  We don't have well-defined success criteria, and don't measure the
      ones we could have (Bernard Aboba)

   o  We should act like the engineers we claim to be and measure things
      to determine if there are problems (Frank Kastenholz)

   o  Trying to measure the progress of the IETF doesn't gather enough
      interest to be worth it (Marshall Rose)








Davies (ed.)            Expires August 25, 2003                [Page 34]


Internet-Draft           IETF Problem Statement            February 2003


Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   intellectual property or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; neither does it represent that it
   has made any effort to identify any such rights. Information on the
   IETF's procedures with respect to rights in standards-track and
   standards-related documentation can be found in BCP-11. Copies of
   claims of rights made available for publication and any assurances of
   licenses to be made available, or the result of an attempt made to
   obtain a general license or permission for the use of such
   proprietary rights by implementors or users of this specification can
   be obtained from the IETF Secretariat.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights which may cover technology that may be required to practice
   this standard. Please address the information to the IETF Executive
   Director.


Full Copyright Statement

   Copyright (C) The Internet Society (2003). All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assignees.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION



Davies (ed.)            Expires August 25, 2003                [Page 35]


Internet-Draft           IETF Problem Statement            February 2003


   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.











































Davies (ed.)            Expires August 25, 2003                [Page 36]