Network Working Group                                      C. Grundemann
Internet-Draft                                                   J. Zorz
Intended status: Informational                          Internet Society
Expires: May 1, 2015                                    October 28, 2014


                         Operators and the IETF
                     draft-opsawg-operators-ietf-00

Abstract

   Internet Society has launched a new project to address the perceived
   gap between Operators and the IETF.  The objective of this project is
   ultimately to facilitate communications between the operator
   community and the IETF to help ensure that operational realities
   inform the development of key standards.  The first phase of this
   project was a survey of the operator community that was conducted
   over the first half of 2014.  This I-D aims to synthesize the initial
   survey results, along with information we collected directly from
   operators during the survey window.  The primary purpose of doing
   this is to start a conversation which we hope will lead to increases
   in the level of operational input and feedback to the IETF standards
   making process.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 1, 2015.

Copyright Notice



Grundemann & Zorz          Expires May 1, 2015                  [Page 1]


Internet-Draft           Operators and the IETF             October 2014


   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Survey respondents . . . . . . . . . . . . . . . . . . . . . .  4
     2.1.  Job type . . . . . . . . . . . . . . . . . . . . . . . . .  4
     2.2.  IETF Involvement . . . . . . . . . . . . . . . . . . . . .  5
       2.2.1.  Do not currently participate in the IETF . . . . . . .  5
       2.2.2.  Do not currently participate on IETF mailing lists . .  6
       2.2.3.  Do not currently participate in IETF meetings  . . . .  8
   3.  Potential Challenges . . . . . . . . . . . . . . . . . . . . .  9
     3.1.  Time . . . . . . . . . . . . . . . . . . . . . . . . . . .  9
     3.2.  Culture  . . . . . . . . . . . . . . . . . . . . . . . . . 10
     3.3.  Money  . . . . . . . . . . . . . . . . . . . . . . . . . . 12
     3.4.  Awareness  . . . . . . . . . . . . . . . . . . . . . . . . 12
     3.5.  Other  . . . . . . . . . . . . . . . . . . . . . . . . . . 13
   4.  Possible solutions . . . . . . . . . . . . . . . . . . . . . . 13
     4.1.  Communication  . . . . . . . . . . . . . . . . . . . . . . 14
       4.1.1.  Mailing List Digests . . . . . . . . . . . . . . . . . 14
       4.1.2.  Alternative Communication Mediums  . . . . . . . . . . 15
     4.2.  Outreach . . . . . . . . . . . . . . . . . . . . . . . . . 16
     4.3.  Inclusion  . . . . . . . . . . . . . . . . . . . . . . . . 19
   5.  Conclusions  . . . . . . . . . . . . . . . . . . . . . . . . . 22
   6.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 22
   7.  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 23
   8.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 23
     8.1.  Normative References . . . . . . . . . . . . . . . . . . . 23
     8.2.  Informative References . . . . . . . . . . . . . . . . . . 23
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23









Grundemann & Zorz          Expires May 1, 2015                  [Page 2]


Internet-Draft           Operators and the IETF             October 2014


1.  Introduction

   The Internet Engineering Task Force (IETF) is an open standards
   organization concerned primarily with developing and promoting open
   Internet standards, with the goal of making the Internet better.
   Their goal, mission statement, and cardinal principles are documented
   in [RFC3935], section 1.

   In a perfect world, the IETF would create standards for protocols
   that meet all relevant network operator requirements, sync with
   operational realities, and are relatively easy to deploy and
   troubleshoot.  In this perfect world, deployment and
   operationalization concerns are consistently addressed and the level
   of operator engagement always makes sense.  Operators always know
   when their input is needed and always provide that needed input.  In
   reality, though, many operators are not engaged enough in the IETF to
   create that perfect world.  It's widely understood that a significant
   portion of operators (particularly small- to mid-size) don't join
   IETF mailing lists or attend IETF meetings, effectively writing
   themselves out of the process.  This leaves vendors, academics, and
   some large organizations to rule many decision-making processes
   within the IETF.  The operators expected to deploy these technologies
   often don't even know that the standards are being developed.  In
   other words, critical new technologies are currently being developed
   without sufficient direct operator input.

   This disparity is not new, nor has it gone unnoticed.  Nine years ago
   this month an essay by Randy Bush was published in ACM SIGCOMM
   Computer Communication Review.  It's titled "Into the Future with the
   Internet Vendor Task Force: A Very Curmudgeonly View - or - Testing
   Spaghetti - A Wall's Point of View" and it calls out this very issue
   of "the devaluation of operational realities."

   However, with the advent of OpenFlow, Software Defined Networks
   (SDN), Network Function Virtualization (NFV), and an overall trend
   toward open source, particularly open application programming
   interfaces (APIs), in the network space, this disparity may now be
   seen as related to a larger and growing problem.  If open APIs become
   the de-facto definition of interoperability requirements, the role of
   the standardization bodies, and the opportunity for operators to
   influence specifications, diminishes.  As a result the functional
   interoperability (and interchangeability) of vendors and devices will
   decrease, potentially leading to a more proprietary and less open and
   global nature of the Internet.

   In response, the Internet Society has launched a new project to
   address the perceived gap between Operators and the IETF.  The
   objective of this project is ultimately to facilitate communications



Grundemann & Zorz          Expires May 1, 2015                  [Page 3]


Internet-Draft           Operators and the IETF             October 2014


   between the operator community and the IETF to help ensure that
   operational realities inform the development of key standards.  We
   want to foster a larger and more engaged operator community around
   the IETF and protocol development work.  To ensure that we take the
   most effective action, we focused initially on talking to operators
   around the world, gathering information and defining the problem
   statement(s).  The first phase of this project was a survey of the
   operator community that was conducted over the first half of 2014.
   The survey closed on 1 July with over 350 responses.

   This I-D begins the next phase, which we hope will lead to productive
   improvements to the level of operational input into the IETF
   processes.  The primary purpose of this paper is to synthesize the
   survey results, along with information we collected directly from
   operators during the survey window.

   In the next section we take a quick look at the survey respondents,
   including their roles and their relationships to the IETF.  Then,
   under the following section, Potential Challenges, we delve into the
   reasons respondents gave for not participating in the IETF.  What are
   their personal challenges to injecting more operational reality?  We
   then move on to discussing ideas for overcoming these obstacles in
   the next section, Possible Solutions.  In all cases, we are primarily
   reporting on the results of the Operators and the IETF survey, open
   from January through June 2014 and the text may not necessarily
   reflect the views of the Internet Society.


2.  Survey respondents

   Before diving into what the survey respondents said, and what that
   might mean, let's first take a look at who these respondents were.

2.1.  Job type

   The first set of questions dealt with the respondents' role.  We
   wanted to ensure that we reached our target audience - technical
   network operators.  A note on our use of the term "operator:" In the
   survey, in this paper, and throughout this Operators and the IETF
   project, we use the term operator to mean anyone who operates an IP
   network.  This includes Internet service providers (ISPs) but also
   content providers, data centers, enterprises, wireless networks,
   campus networks, and everything in between.

   o  I'm an Operator - 44% strongly agree and 34% agree, 12% disagree
      and 10% strongly disagree.





Grundemann & Zorz          Expires May 1, 2015                  [Page 4]


Internet-Draft           Operators and the IETF             October 2014


   o  My role is primarily technical - 63% strongly agree and 29% agree,
      9% disagree and 2% strongly disagree.

   The respondents were overwhelmingly technical and the vast majority
   were operators.  We also drilled into specific roles of respondents:

   o  I'm an Engineer - 60% strongly agree and 32% agree, 5% disagree
      and 3% strongly disagree.

   o  I'm an Architect - 42% strongly agree and 38% agree, 14% disagree
      and 6% strongly disagree.

   o  I'm an Developer - 9% strongly agree and 29% agree, 39% disagree
      and 23% strongly disagree.

   o  I'm an Manager - 24% strongly agree and 26% agree, 27% disagree
      and 23% strongly disagree.

   92% of respondents self-identified as engineers and 80% as
   architects, while only 38% said they were developers and about half
   said they were managers.

2.2.  IETF Involvement

   The next set of questions dove into the respondents' current level of
   participation in, and awareness of, the IETF.

   Exactly half of the respondents reported that they did NOT currently
   participate in the IETF at all.  Almost one third said they
   participate in the IETF via mailing lists only.  Only about 18%
   reported that they were active in the IETF via both mailing lists and
   in-person meetings.  This is about the mix we were expecting.

   Since we are looking primarily for challenges and obstacles to
   participation, we wanted to hear from the people who do not
   participate.  Mixing in some people with more experience on how the
   IETF works today helps provide useful context, though.  We then asked
   each group of respondents more detailed questions about their level
   of awareness, based on their participation level in the IETF.

2.2.1.  Do not currently participate in the IETF

   Among those who do not currently participate in the IETF at all, a
   wide majority know that the IETF exists, know what it does, find IETF
   documents relevant to their jobs, and feel that they need to
   participate.

   So why aren't they showing up?  Other responses within this set of



Grundemann & Zorz          Expires May 1, 2015                  [Page 5]


Internet-Draft           Operators and the IETF             October 2014


   questions provide clues: Over half of this group reported that they
   do not know how to participate, and almost half said that they do not
   feel welcome.

   Detailed results:

   o  I never heard of IETF: 82% strongly disagree, 14% disagree, 1%
      agree, 3% strongly agree In short - 4% report not having heard of
      the IETF before the survey

   o  I don't know what IETF does: 57% strongly disagree, 35% disagree,
      8% agree, 0% strongly agree In short - 8% don't participate
      because they don't know what the IETF does

   o  I don't know how to participate: 21% strongly disagree, 21%
      disagree, 44% agree, 14% strongly agree In short - 58% of those
      who do not participate in the IETF do not know how

   o  I don't believe IETF documents are relevant to my job: 53%
      strongly disagree, 33% disagree, 11% agree, 3% strongly agree In
      short - 86% believe that IETF documents are relevant to their work

   o  I don't feel my operator input is welcomed: 14% strongly disagree,
      42% disagree, 33% agree, 11% strongly agree In short - 44% do not
      participate because they feel unwelcome

   o  I rely on my vendors to represent me: 27% strongly disagree, 37%
      disagree, 30% agree, 6% strongly agree In short - 36% rely on
      their vendors to represent them at the IETF

   o  I don't need to participate, I just need the output: 24% strongly
      disagree, 49% disagree, 23% agree, 4% strongly agree In short -
      27% choose not to participate because they are only concerned with
      the output of the IETF (RFCs)

2.2.2.  Do not currently participate on IETF mailing lists

   Among respondents who do not currently participate in IETF mailing
   lists, over two thirds believe that it does, in fact, fall within
   their job to do so.  While most respondents had at least heard of
   IETF mailing lists, about half reported not knowing what happens on
   those lists and 40% said they don't know how to join.  Also, about a
   third of respondents do not participate on IETF mailing lists because
   "list noise" (off-topic and unhelpful content) is too high.

   Almost three quarters report staying off the lists because they don't
   think they have enough time.  Almost half of those who are not on an
   IETF mailing list thought they had to show up to meetings to



Grundemann & Zorz          Expires May 1, 2015                  [Page 6]


Internet-Draft           Operators and the IETF             October 2014


   participate and were unaware that most IETF work actually takes place
   on the mailing lists.

   Detailed results:

   o  I've never heard of IETF mailing lists: 44% strongly disagree, 25%
      disagree, 25% agree, 6% strongly In short - 31% had never heard of
      IETF mailing lists before this survey

   o  I don't know what happens on IETF mailing lists: 23% strongly
      disagree, 23% disagree, 44% agree, 10% strongly agree, In short -
      54% don't know what happens on an IETF mailing list

   o  I don't know how to join an IETF mailing list: 35% strongly
      disagree, 25% disagree, 34% agree, 6% strongly agree In short -
      40% aren't on an IETF mailing list because they don't know how to
      join

   o  I'm not interested: 29% strongly disagree, 55% disagree, 14%
      agree, 2% strongly agree In short - 16% of respondents don't
      participate due to lack of interest

   o  I find the content too technical or abstract: 27% strongly
      disagree, 47% disagree, 25% agree, 1% strongly agree In short -
      26% find IETF mailing list content too technical or too abstract

   o  I don't have enough time: 8% strongly disagree, 20% disagree, 41%
      agree, 31% strongly agree In short - 72% say they don't
      participate because they don't have time

   o  I don't find the content relevant: 25% strongly disagree, 58%
      disagree, 16% agree, 1% strongly agree In short - 83% find IETF
      mailing list content relevant

   o  It's not my job: 22% strongly disagree, 48% disagree, 25% agree,
      5% strongly agree In short - 70% think that following IETF mailing
      lists falls within their job duties, even though they don't
      currently do it

   o  There's too much noise on the lists (off-topic discussions,
      etc...): 15% strongly disagree, 51% disagree, 27% agree, 7%
      strongly agree In short - 34% replied that "list noise" is an
      issue for them

   Before taking this survey:






Grundemann & Zorz          Expires May 1, 2015                  [Page 7]


Internet-Draft           Operators and the IETF             October 2014


   o  54% I was aware that most of the work in the IETF happens on
      mailing lists between meetings

   o  46% I thought I had to show up at IETF meetings to participate

2.2.3.  Do not currently participate in IETF meetings

   While most respondents had at least heard of IETF meetings, nearly
   half of those who do not currently attend IETF meetings reported not
   knowing what happens at a meeting and slightly more said they don't
   know how to participate.  Two thirds of respondents told us they
   don't attend IETF meetings because they don't have the time and over
   three quarters because they don't have the travel budget to attend a
   meeting.  Just less than one third feel it's not their job to
   participate at IETF meetings.  Almost exactly half of the respondents
   who do not attend IETF meetings were unaware that remote
   participation is available.

   Detailed results:

   o  I've never heard of IETF meetings: 60% strongly disagree, 25%
      disagree, 10% agree, 5% strongly agree In short - 15% don't come
      to IETF meetings because they hadn't heard of them before

   o  I don't know what happens at IETF meetings: 31% strongly disagree,
      24% disagree, 35% agree, 10% strongly agree In short - 45% don't
      show up because they don't understand what goes on at an IETF
      meeting

   o  I don't know how to participate in an IETF meeting: 30% strongly
      disagree, 21% disagree, 35% agree, 14% strongly agree In short -
      49% don't participate in meetings because they don't know how to

   o  I'm not interested: 39% strongly disagree, 48% disagree, 10%
      agree, 3% strongly agree In short - 13% avoid IETF meetings due to
      a lack of interest

   o  I find the content too technical or abstract: 33% strongly
      disagree, 48% disagree, 17% agree, 2% strongly agree In short -
      19% don't participate in IETF meetings because the content is too
      technical or too abstract

   o  I don't have enough time: 15% strongly disagree, 21% disagree, 36%
      agree, 28% strongly agree In short - 64% don't come because they
      don't have enough time to participate

   o  I don't have the travel budget: 7% strongly disagree, 11%
      disagree, 37% agree, 45% strongly agree In short - 82% don't



Grundemann & Zorz          Expires May 1, 2015                  [Page 8]


Internet-Draft           Operators and the IETF             October 2014


      attend IETF meetings because they lack the travel budget

   o  I don't find the content relevant: 30% strongly disagree, 56%
      disagree, 10% agree, 4% strongly agree In short - 14% don't come
      to meetings because they content is not relevant to them

   o  It's not my job: 26% strongly disagree, 44% disagree, 23% agree,
      7% strongly agree In short - 30% don't attend IETF meetings
      because it doesn't fit their job duties

   Before taking this survey:

   o  50% I was aware that most of the IETF meeting sessions are
      available to remote participants

   o  50% I thought I had to show up at IETF meetings to participate


3.  Potential Challenges

   In addition to the multiple choice options on the Operators and the
   IETF survey, we included several opportunities for respondents to
   enter freeform text.  We asked questions such as "Tell us about why
   you do not currently participate in the IETF in your own words" and
   "Why do you think other operators do not participate in the IETF?"
   When these responses are evaluated in combination with the numbers
   above, some themes emerge.

   There appear to be four major perceived obstacles to IETF
   participation:

   o  Time

   o  Culture

   o  Money

   o  Awareness

   These one-word captions are, of course, simplifications.  The
   following sections dig into each of these themes to explore the
   potential and perceived challenges a bit more.

3.1.  Time

   One of the top complaints from both survey respondents and operators
   we have spoken with in person is how much time is required to
   participate meaningfully in the IETF.  As Mr. Bush pointed out in his



Grundemann & Zorz          Expires May 1, 2015                  [Page 9]


Internet-Draft           Operators and the IETF             October 2014


   2005 paper:

   "The IETF has grown so large and so enamored of complexity and
   featuritis that it is a full-time job to participate."

   We can see this perception confirmed in the numbers above:

   o  72% of respondents who do not participate in IETF mailing lists
      say they don't participate because they don't have enough time

   o  64% of respondents who don't attend IETF meetings report that they
      don't come because they don't have enough time to participate

   Additionally, the most common response to both "Tell us about why you
   do not currently participate in the IETF in your own words" and "Tell
   us about why you are not currently a member of any IETF mailing lists
   in your own words" were some variation of "not enough time."  This
   was the second most common response to the "Tell us about why you do
   not currently attend IETF meetings in your own words" question as
   well.  While many of those responses were very terse, some took the
   time to explain their challenges a bit more:

   "Time restriction is an issue.  Keeping up with my "day job"
   responsibilities is challenging.  There's difficulty in sorting out
   where the different BoFs and working groups are in the process - very
   hard to step into the middle of an ongoing conversation, translate it
   to my world, and engage in the discussion.  Makes it hard to do more
   than lurk."

   When we dig into the details of some of these comments, it appears
   that time constraints may be tied in some ways to what we're calling
   cultural challenges here.

   "I don't have the time to sift through the entrenched autistic and
   esoteric arguments.  There are very obviously people who are paid to
   participate in the IETF by vendors (and other orgs) for whom it's
   their full time job, or one of the primary purposes of their job, and
   they don't have other significant responsibilities.  It therefore
   makes debating with these people very difficult if your involvement
   in IETF is a secondary (or tertiary) function of your role."

3.2.  Culture

   Another common challenge we heard about while conducting this survey
   is one of culture.  While the IETF is open to participation by
   anyone, some feel that they are not welcome or that they will be
   ignored or treated badly by other participants.




Grundemann & Zorz          Expires May 1, 2015                 [Page 10]


Internet-Draft           Operators and the IETF             October 2014


   We saw above that almost half (44%) of the respondents who do not
   currently participate in the IETF at all avoid it because they don't
   feel their operator input is welcomed.

   This is echoed in some detail in many of the comments left by
   respondents:

   o  IETF are just concern about the old members and don't think of how
      to get new members onboard

   o  "I do not feel that the IETF is responsive to the needs and
      requirements of those delivering services.  The responses to the
      IPv6 DHCP enterprise requirements are an example of the
      disconnection in the IETF.  Many times I have read or participated
      in discussions on different mailing lists about many of the topics
      and the final item pushed out by people in the IETF has been
      "you're stupid and an idiot and we're going to do it my way".  I
      can get that at home with my teenager."

   o  "Conversations are heavily dominated by academics with little or
      no practical experience (but deep theoretical knowledge and
      skills), and vendor professionals who are so senior and
      experienced.  Both folks cast long shadows that are intimidating
      to others who can't devote the time to keeping up with what are
      often detailed and nuanced discussions."

   o  "Despite wanting claims that operators were welcome, as I switched
      from protocol engineer to operator, I saw growing irrelevance.
      The IETF is maintaining an "ivory tower" vision of how the
      Internet is used, whereas the world economy has a different view
      of how to make use of the medium.  The IETF hasn't been welcome to
      new sets of requirements.  In DNS, the IETF has been effectively
      replaced by DNS-OARC."

   o  "[I don't participate in the IETF] Because its become a political
      fight between vendors.  Vendors push their individual agendas
      without caring about user opinions.  A contentious issue will
      bring out half the opposition companies employees to bash and kill
      it regardless of whether there is a true customer that may benefit
      from it."

   o  "I perceive it to be full of pompous, self-serving, out-of-touch
      with reality, technology actors."

   o  "The IETF is not really focused towards operations and,
      historically, operator input has not been well received."

   Another aspect of culture is being more open to participation from



Grundemann & Zorz          Expires May 1, 2015                 [Page 11]


Internet-Draft           Operators and the IETF             October 2014


   around the world:

   "Most studies have been conducted in English, which makes it
   difficult for those who have not mastered the language."

3.3.  Money

   The most common free-form response to "Tell us about why you do not
   currently attend IETF meetings in your own words" as well as the
   fourth most common response to "Tell us about why you do not
   currently participate in the IETF in your own words" was money, and
   general support from the respondents' organization.  Some of the
   nuances here are pointed out in the responses:

   o  "It is too expensive to attend regularly.  It is not my primary
      job to attend IETF meetings, so is secondary to other things."

   o  "I don't have enough budget to attend the conference.  Based up in
      India, my travel budget + accommodation + Food + Visa will come
      around 2000 USD (for Conferences in US) at the minimum, this is my
      2 months salary."

   o  "I'm a self-employed contractor.  I can't afford to pay for it
      myself, and my clients wouldn't pay to send me there because it's
      not what gets their business needs met.  And every hour I spend at
      conferences and the like is an hour I don't get paid."

   Additionally, a full 82% of respondents who don't attend IETF
   meetings reported that it was because they lack the needed travel
   budget.

3.4.  Awareness

   Throughout the survey, respondents told us that their ignorance of
   the IETF, what it does, how it operates, and how individuals can get
   involved, is a barrier to their participation.  Overall, 58% of those
   who do not participate in the IETF at all reported that they do not
   know how to.

   Among respondents who don't follow any IETF mailing lists, almost a
   third had never heard of the mailing lists, over half said they don't
   know what happens on IETF mailing lists, and 40% reported not knowing
   how to join a list.

   Likewise, among those who do not attend IETF meetings, 45% told us
   they don't show up because they don't understand what goes on at an
   IETF meeting and nearly half (49%) said they don't participate in
   meetings because they don't know how to.



Grundemann & Zorz          Expires May 1, 2015                 [Page 12]


Internet-Draft           Operators and the IETF             October 2014


   As with the other themes, this was echoed in the comments:

   o  "No awareness of how I can help, what I can do, and where my goals
      would align with the IETF."

   o  "I do not know how can I participate in IETF.  I would love to
      know how can I participate. not just by subscribing to mailing
      list but by doing some work in my part time."

   o  "I have no idea how to even begin participating"

3.5.  Other

   There are also other reasons that came up in the freeform answers
   that we feel are very important but that don't fit into one of the
   themes above.  They include:

   o  Too technical: "Too broad", "Discussions are on an abstract,
      technical level above my expertise"

   o  Not operational enough: "Not enough operational lists",
      "Operations guys stay home doing operations"

   o  Ignorance: "I never felt the need, it never occurred to me to try"

   o  Management support: "Lack of Management Support"

   o  Out of scope: "Out of scope of my job", "I really don't care",
      "Not relevant to my job"

   o  Format: "The meetings are all about status of a particular draft
      which really isn't that interesting."

   o  Reliance on vendors: "Operators mainly rely on system integrators
      and rarely believe that their voices are heard"

   o  Not relevant to operators: "Not relevant to the short/mid-term
      needs of most operators", "The lead time from idea to RFC to
      vendor support to using in a design is years for me.  I will tend
      to use a technology once it reaches maturity."


4.  Possible solutions

   The ultimate purpose of collecting this information and identifying
   the perceived challenges to operator participation in the IETF is, of
   course, to help us find ways to bring more operational input into the
   IETF.  In other words, we want to determine how we may be able to



Grundemann & Zorz          Expires May 1, 2015                 [Page 13]


Internet-Draft           Operators and the IETF             October 2014


   solve these potential challenges.

   The possible solutions listed here are based on the survey results
   and on numerous conversations with operators around the world.  This
   is not expected to be an exhaustive list but rather a starting point
   for further exploration.  Note that this paper is simply documenting
   potential solutions and is not, at this time, making any
   recommendations.  The next step of this process is to widen the
   discussion and explore all of our options for making the IETF even
   better than it is today.

   The remainder of this section is dedicated to individual ideas; some
   are suggestions for change at the IETF, others point to activities
   that operators or operator groups could do to help, and some are
   clues to what the Internet Society may be able to do.  These ideas
   for possible solutions are grouped into three larger buckets or
   themes.  The themes that the solutions fall into are:

   o  Communication

   o  Outreach

   o  Inclusion

   At this time, this is a very raw collection of direct quotes from the
   Operators and the IETF survey, and others we spoke with anonymously.
   In later versions of this Internet-Draft we intend to update this
   section based on conversations both inside and outside of the IETF.

4.1.  Communication

   The first theme that we saw emerge among possible solutions is one of
   communication.  Most of these ideas focus on alternative methods of
   communication between the IETF and its constituents and specifically
   on new ways to provide condensed information to IETF participants and
   potential participants.

4.1.1.  Mailing List Digests

   o  "Quarterly summaries for those that are not able to attend."

   o  "Provide a curation service that takes key developments in a
      working group and shares them from time to time - save operators
      from having to make sense out of nuanced arguments so that they
      can jump into conversations with reasonable confidence they know
      what's happened so far and therefore won't embarrass themselves."





Grundemann & Zorz          Expires May 1, 2015                 [Page 14]


Internet-Draft           Operators and the IETF             October 2014


   o  "There's probably no silver bullet, but one thing that I would
      find most useful would be a single daily/weekly/monthly digest
      mailing list.  Just headlines and updates from each of the working
      groups.  (Along with links to where to find more information for
      each.)"

   o  "Make it dead simple for folks to see the specific topics being
      discussed and worked on.  If I had some idea what the topics were,
      I would be more likely to participate if there was a topic that I
      had some expertise in and more importantly an opinion about how to
      address the issue."

   o  "a blog with some updates of things would be nice. reading a huge
      amount of emails is not that fast and with a blog post you can
      join the discussion when there is something important ongoing"

   o  "At least provide weekly summaries what's currently in discussion
      and which new drafts or RFCs were published."

   o  "Invest in reducing perceived entropy and lower the time
      commitment to do so - both requires energy inputs.  Action:
      Introduce and invest support staff that write accessible summaries
      (like the former Cisco IPJ) - licensed under CC so that they can
      be freely translated to other languages without breaking the bank:
      Note - I'm associated with IPJ and am biased."

   o  "Highlight specifically which groups' efforts are looking for
      operator input.  Or color-code agendas by "how close" different
      efforts are to needing operator input.  Have those folks write an
      operator's abstract.  Package the background homework to make it
      easy for us to catch up and easy to see if the effort is relevant
      to us.  Give ways for us to input to the process that is separated
      from the "players" usual modes (eg mailing list)."

4.1.2.  Alternative Communication Mediums

   o  "Offer communications options other than e-mail."

   o  "Surveys like this are a good start.  Ask about the vendors we
      have relationships with, what technologies we currently use, what
      we're deploying now, and what we'd like to deploy in future."

   o  "Audio-only podcasts are a really great medium for busy people,
      IMO.  They convey the personality of the people who are presenting
      them whilst still allowing us to do things like drive to work,
      cook, vacuum, or jog."





Grundemann & Zorz          Expires May 1, 2015                 [Page 15]


Internet-Draft           Operators and the IETF             October 2014


   o  "RSS feeds that help busy people keep track of the really
      important happenings would be good (maybe they exist already)."

   o  "Could RFCs be made more friendly in the following fronts: *
      Readability: have an "human" English version and/or summary. *
      Viewability: it seems the format followed is from the 80s.  Make
      changes to the font, format, etc. to make reading them more
      appealing."

   o  "Make it easier and less time consuming, like having a simple
      system for feedback on drafts and decisions."

   o  "Determine the questions to ask of Operators, and then start
      distributing those questions/forms via social media and reddit."

   o  "Transition from ASCII Text RFCs to documents that support
      figures.  There was a time when text-only made sense, but that was
      long ago.  Figures made out of ASCII characters are difficult to
      read and are not able to convey as much information."

   o  "Create polls for various important matters where operators vote.
      Give the chance to operators to propose their ideas in simple
      texts and then have an IETF expert help them translate this simple
      text into an IETF doc.  Ask the operators for various issues
      regarding the current RFCs and get their proposals about them."

   o  "We need tools which makes IETF-related work more effective.  For
      example, my main problem is I can not see any way of easily track/
      find all discussions related to a particular drafts.  Let's say I
      see that a very interesting draft has been published.  Most likely
      there will be a lot of different email threads going on so it is
      really hard to track all discussions/comments."

4.2.  Outreach

   The next theme we heard is one of outreach to the operators,
   primarily through existing network operator groups and forums but
   also through more traditional marketing and publicity techniques:

   o  "More liaisons between the IETF and Operator forums"

   o  "Possibly smaller events like ARIN road show events for the
      general IT community"

   o  "Participation in I.E.T.F needs to be demystified.  Internet
      Society needs to reach out to the operators and the local
      technical community in every country to create awareness that
      I.E.T.F is open for participation, it does have a membership



Grundemann & Zorz          Expires May 1, 2015                 [Page 16]


Internet-Draft           Operators and the IETF             October 2014


      system, and that anyone who participates can equally contribute to
      discussions on the same level as more qualified or frequent
      participants.  And that funding opportunities are open.  And that
      it is important for operators to take part."

   o  "I'd recommend signing up to popular mailing lists like NANOG or
      CISCO and JUNIPER NSP and promote the ongoing IETF topics/agenda,
      current meetings to encourage operators to take part in what they
      find interesting"

   o  "The IETF could do a better job of making themselves known to
      newbie networkers.  Ask anyone who's been in the field for less
      than two years and you're unlikely to get anything more detailed
      than "Oh, uh, I think they write RFCs.""

   o  "Give more information about IETF no local engineers in local
      languages with simple example of advantages of participation."

   o  "Post in relevant worldwide networking mailing lists when you have
      information that wouldn't be spam like.  For example, when you
      post meetings, are also at an event related to the mailing list,
      etc etc."

   o  "co-located sessions in Network Operators meetings"

   o  "Road shows at other well attended operator events (ie: NANOG,
      MAAWG, etc)."

   o  "Promote IETF events and involvement through regional tech events
      & meetups (in the Seattle area: NorthWest Tech Alliance /
      SeattleTechForum) and industry related trade-shows and events
      (NANOG / ARIN / PTC / SuperComputing / etc.).  As well, providing
      a brief summary of how to support / become involved with the IETF
      which is promoted within the IETF pages and/or search results."

   o  "Try outreach programs to EDU operators.  EDUs are the testbeds of
      the past, present, and future.  Engage us!"

   o  "1.  IETF members presence on different conferences. 2.  IETF must
      publish examples of stories or good practices in e-zines or
      participate at national events. 3.  IETF must invest in promotion
      and presence in various technical media, university curriculums,
      national events."

   o  "Collocating (interim) IETF Meetings with RIPE/NANOG/etc meeting -
      I attended an interim IETF meeting after a RIPE meeting - at least
      for the ops groups."




Grundemann & Zorz          Expires May 1, 2015                 [Page 17]


Internet-Draft           Operators and the IETF             October 2014


   o  "Promote the IETF impact and role of standards to large operators
      (education)."

   o  "Also setting up a IETF operators conference that is design to
      bring the long term to the short term relevance and present it
      ways that is significant to operators and not just RFCs that
      vendors and operators have to figure how to make into products.
      This should include the vendors that are creating products based
      on new standards to present what they are doing with it.  Would
      provide real world ties to the work IETF is doing rather than RFCs
      that get lost in code and are generally ignored by future
      companies.  You would be surprised how many new product vendors do
      not have a clue about some of the older RFCs that are still HUGELY
      significant when developing a device.  Things like new vendor mini
      conference that goes over operationally significant RFCs that they
      should be building into their devices."

   o  "As for the Internet Society, probably modifying the mentorship
      programs to be announced more on local mailing lists and also
      require winners to spend some time on mailing lists before
      attending events."

   o  "We are here in ISOC India Chennai Setting up a Remote Engagement
      Hub (India) we will be requesting all Stake holders which includes
      ISP/Telcos."  (Also see LAC-IETF)

   o  "initially be possible to create regional forums that are
      preparatory style operators to approach, understand the benefits
      and challenges for both regional and continental as
      implementations of the ietf can create a kind of good practice for
      operators as Lationamerica less scale and the Caribbean."

   o  "Provide briefings to operator community on technologies that may
      be relevant, and how to get up-to-speed on them.  Actively seek
      operator feedback on those technologies (this has been tried in
      the past, but on a very one-off ad-hoc basis)."

   o  "Increase interaction and outreach between IETF and operator
      forums, probably by identifying a subset of IETF drafts and areas
      that could most benefit from additional operator input such that
      we can focus the help that we're asking for - simply trying to
      convince people to participate generically isn't likely to be
      successful, while asking for specific feedback on specific items
      will be seen as a better use of time.  It may even be useful to
      try to coordinate one meeting per year or every two years with an
      operator forum to encourage cross-pollination."





Grundemann & Zorz          Expires May 1, 2015                 [Page 18]


Internet-Draft           Operators and the IETF             October 2014


   o  "It's a tough question...  You need a "hot RFC" and turn it into a
      media-backed frenzy.  Something focus interest of a large number
      of technical folks.  You may also want to just elevate a few
      select 'products'.  Keep a few key items "up front".  For
      instance, take a look at Mozilla and Mozilla Labs.  Even Google
      and (now defunct) Google Labs.  Push a few key "products" (of the
      IETF's 7136 RFC's) and put them everywhere, showcase a few more.
      Focus and push the technologies forward and you may get more
      participation."

   o  "ensure that meaningful RFC's and other publications get more
      press than TCP over Avian Carrier."

   o  "Strategic Plan of publicity about IETF and its main activities.
      This strategic plan should be in several languages to reach
      everyone."

   o  "I would love to see a list of reasons why operator participation
      is needed and what the pay-off is for the operator, as well as the
      community as a whole."

4.3.  Inclusion

   The final theme we found amongst the offered possible solutions is
   one of inclusion.  How can the IETF be more welcoming, in order to
   bring participants in and keep them engaged?  Survey respondents had
   several ideas:

   o  "Welcoming our participation would be helpful."

   o  "make the operators feel more welcome"

   o  "Education."

   o  "Travel subsidy?"

   o  "Make remote participation easier."

   o  "Asking vendors to bring operators to the meetings.  Planning a
      few IETF meetings right before or after a big operator meeting
      (e.g.  RIPE, NANOG)"

   o  "Encourage non-technical participation, cultural/educational
      program, beginner training."

   o  "Introduce works in multi language."





Grundemann & Zorz          Expires May 1, 2015                 [Page 19]


Internet-Draft           Operators and the IETF             October 2014


   o  "Well you see... [respondent wrote in Chinese characters here,
      which we can not include in Internet-Draft formatting]"

   o  "Probably topical meetings (only for some area of IETF work) might
      lessen the burden on the main meetings."

   o  "Provide some training opportunity during meetings to help justify
      participation."

   o  "Provide scholarship and/or sponsorship to students, small
      operators."

   o  "Provide more sponsorships"

   o  "Facilitate joining committees."

   o  "better discovery of projects. this may seem strange but perhaps a
      match making site to better express interests and goals if
      individuals and projects." (constituencies)

   o  "Smaller focused sessions"

   o  "It's hard for people in an operations-led post to make a
      justification for a portion of their time and budget to be
      invested in participating in the IETF.  There's also the challenge
      that as an operations person does manage to spend some more time
      involved in the IETF and the standards process they end up doing
      less operations and more standardisation work, and therefore their
      input becomes less operationally relevant.  There needs to be a
      greater acceptance among the IETF attendees of the fact that
      operationally focused people will dip in and out of the IETF as
      their work requires."

   o  "Add more operational content."

   o  "Require standards to get the buy-in of a variety of operators."

   o  "if operator input was required in a "peer-review" fashion -- and
      operators were represented from very large to very small"

   o  "Create a Service Provider input channel that is taken seriously.
      The SPAC (Service Provider Action Committee) is an example that
      exists within the Broadband Forum."

   o  "Having dedicated listeners might help.  As in: someone who
      actually listens to what operators say, tries to understand
      whether "reality" looks different than the perfect theoretical
      network design, and ensures that this is not drowned in other



Grundemann & Zorz          Expires May 1, 2015                 [Page 20]


Internet-Draft           Operators and the IETF             October 2014


      discussions."

   o  "New ways of gathering people reducing the cost (remote
      participations from multiple locations?)."

   o  "48 hour days :-) Back-to-back WG meetings with other relevant
      conferences (ICANN or RIPE for DNS related topics would be the
      obvious choices) to reduce travel time/budget."

   o  "Publish agendas early, lower the price of meetings and hotel
      accommodations, have more operator relevant side meetings, gather
      relevant working group meetings on the same day.  Some interesting
      side meetings would be, Internet Exchange meetings like Euro-IX,
      joint NOG meetings, Peering forums, capacity update meetings,
      etc."

   o  "Perhaps consider consolidating time by interest areas so that the
      travel time demands were not as great."

   o  "I guess, having a BCOP (best current operational practices; like
      http://bcop.nanog.org/ and http://www.ipbcop.org/) would attract
      more operators."  "Acknowledge operators.  Provide them ways to
      make a difference.  For example, define a class of documents that
      require the participation of at least two operators.  That can be
      against the usual convention of "individual" participation, but
      anyway we know that it no longer holds..."

   o  "Suggest that the IETF working groups, especially the ones dealing
      with topics that operators would have a significant interest in,
      start to accept that operator requests may be valid even if they
      are not in agreement with existing opinions."

   o  "I would restructure IETF to be more agile and remove the
      political hierarchy of IETF that prevents wider participation.
      The use of operators as working group Operator Councils rather
      than just having Co-Chairs to determine what topics are good and
      not good for that working group. ...  The #1 thing is to remove
      the politics of IETF so people with ideas and good suggestions are
      not stifled by the IETF machine and the fulltime IETF
      politicians."

   o  "While I know it can't be (and shouldn't be done), reduce the
      voice of the people who seem to have nothing else to do than reply
      to emails whole day.  IETF should be about different views not
      destroying everyone who don't agree with you."

   o  "Do some IETF meetings in our region, especially where there is
      dense penetration of internet users as well as a strong technical



Grundemann & Zorz          Expires May 1, 2015                 [Page 21]


Internet-Draft           Operators and the IETF             October 2014


      community (such as Southern and Eastern Africa)."

   o  "Try and group more operationally-relevant sessions together so
      that it doesn't require a full week to participate."

   o  "It needs more open leadership.  The top of the IETF is like merry
      go round.  The same folks make sure their colleagues all get jobs,
      same names, same people, no change"

   o  "Operators don't care how the things are fixed, we do care if the
      requirements are addressed.  Create a WG for operators to
      establish business needs, and customer needs - let them create
      "requirement's documents" in the form of conceptual abstraction
      meta models that can be put out in the body.  Treat this as an
      Open Framework a bit more, and lower the tribal culture a bit
      less."

   o  "One day ticket is good idea.  And place is important for us."

   o  "Better stewardship/shepherding of drafts and stopping the brain
      damaged drafts from wasting WG time.  Not everything requires IETF
      work, nor needs to be written in a standard."

   o  "Internet society should do more educations in operators
      management level.  I notice that my company seems fonder on ATIS
      and Cablelabs.  I guess because the white papers published that
      usually not require most technical people to understand.  Thus,
      management level can know what actually go on in industrial.  On
      the other hand, RFC is very technical.  Management doesn't know
      how that applies to them.  So, some user-friendly introduction of
      WG and how they can help operators may be good to present to
      managements."


5.  Conclusions

   This section will be completed after WG discussion.


6.  IANA Considerations

   This document makes no request of IANA.

   Note to RFC Editor: this section may be removed on publication as an
   RFC.






Grundemann & Zorz          Expires May 1, 2015                 [Page 22]


Internet-Draft           Operators and the IETF             October 2014


7.  Acknowledgments

   Because the survey and many of our conversations were conducted
   anonymously, we have left this section blank at this time.  We have
   however received lots of help from many members of the IETF and
   operator communities.  If you feel that you made a significant
   contribution and would like your name to appear here, please let us
   know and send us cookies :)


8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

8.2.  Informative References

   [RFC3935]  Alvestrand, H., "A Mission Statement for the IETF",
              BCP 95, RFC 3935, October 2004.


Authors' Addresses

   Chris Grundemann
   Internet Society
   150 West 9th Ave.
   Denver, Colorado  80204
   US

   Phone: +1 303 351 1539
   Email: grundemann@isoc.org


   Jan Zorz
   Internet Society
   Frankovo naselje 165
   Skofja Loka  4220
   Slovenia

   Phone: +38659042000
   Email: zorz@isoc.org








Grundemann & Zorz          Expires May 1, 2015                 [Page 23]