NFVRG                                                        C. Meirosu
Internet Draft                                                 Ericsson
Intended status:  Informational                            A. Manzalini
Expires: August 2015                                     Telecom Italia
                                                                 J. Kim
                                                       Deutsche Telekom
                                                            R. Steinert
                                                                   SICS
                                                              S. Sharma
                                                                 iMinds
                                                           G. Marchetto
                                                  Politecnico di Torino
                                                             I. Papafili
                                Hellenic Telecommunications Organization

                                                      February 27, 2015




            DevOps for Software-Defined Telecom Infrastructures
                      draft-unify-nfvrg-devops-01.txt


Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on August 27, 2015.





Meirosu, et al.        Expires August 27, 2015                 [Page 1]


Internet-Draft            DevOps Challenges               February 2015


Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Abstract

   The introduction of virtualization technologies, starting from the
   physical layer and going all the way up to the application plane, is
   transforming the telecom network infrastructure onto an agile, model-
   driven production environment for communication services. Carrier-
   grade network management was optimized for environments built with
   monolithic physical nodes and involves significant deployment,
   integration and maintenance efforts from network service providers.
   The DevOps movement in the data center is a source of inspiration
   regarding how to simplify and automate management processes for
   software-defined infrastructure. This first version of this draft
   identifies three areas that we consider key to applying DevOps
   principles in a telecom service provider environment, namely for
   monitoring, verification and troubleshooting processes. Finally, we
   introduce challenges associated with operationalizing DevOps
   principles at scale in software-defined telecom networks.

Table of Contents


   1. Introduction...................................................3
   2. Conventions used in this document..............................4
   3. DevOps Principles for Software-Defined Telecom Infrastructure..4
   4. Stability Challenges...........................................6
   5. Consistency, Availability and Partitioning Challenges..........8
   6. Observability Challenges.......................................9
   7. Verification Challenges........................................9
   8. Troubleshooting Challenges....................................11
   9. DevOps Performance Metrics....................................12
   10. Security Considerations......................................13
   11. IANA Considerations..........................................13


Meirosu, et al.        Expires August 27, 2015                 [Page 2]


Internet-Draft            DevOps Challenges               February 2015


   12. References...................................................13
      12.1. Normative References....................................13
      12.2. Informative References..................................13
   13. Acknowledgments..............................................14

1. Introduction

   Carrier-grade network management was developed as an incremental
   solution once a particular network technology matured and came to be
   deployed in parallel with legacy technologies. This approach requires
   significant integration efforts when new network services are
   launched. Both centralized and distributed algorithms have been
   developed in order to solve very specific problems related to
   configuration, performance or fault management. However, such
   algorithms consider a network that is by and large functionally
   static. Thus, management processes related to introducing new or
   maintaining functionality are complex, and costly due to significant
   efforts required for verification and integration.

   Network virtualization, by means of Software-Defined Networking (SDN)
   and Network Function Virtualization (NFV), is creating an environment
   where network functions are no longer static and embedded into
   physical boxes deployed at fixed points. The virtualized network is
   dynamic and open to fast-paced innovation enabling efficient network
   management and reduction of operating cost for network operators. A
   significant part of network capabilities are expected to become
   available through interfaces that resemble the APIs widespread within
   datacenters instead of the traditional telecom means of management
   such as the Simple Network Management Protocol, Command Line
   Interfaces or CORBA. Such an API-based approach, combined with the
   programmability offered by SDN interfaces [I-D. draft-irtf-sdnrg-
   layer-terminology-04], open opportunities for handling
   infrastructure, resources, and Virtual Network Functions (VNFs) as
   code, employing techniques from software engineering.

   The efficiency and integration of existing management techniques in
   virtualized and dynamic network environments are limited, however.
   Monitoring tools, e.g. based on simple counters, physical network
   taps and active probing, scale poorly and provide only a small part
   of the observability features required in such a dynamic environment.
   Huge amounts of monitoring data can be collected from the nodes, but
   the typical granularity is coarse-grained. Although debugging and
   troubleshooting techniques developed for software-defined
   environments are a research topic that has gathered interest in the
   research community in the last years, it is yet to be explored how to
   integrate them into an operational network management system.
   Moreover, tools that have been developed in academia are limited to


Meirosu, et al.        Expires August 27, 2015                 [Page 3]


Internet-Draft            DevOps Challenges               February 2015


   solving very particular, well-defined problems, while they were not
   built for automation and integration into network operations
   workflows.

   We acknowledge that several standardization organizations have a
   stake in this area. IETF working groups have activities in the area
   of OAM [I-D.draft-aldrin-sfc-oam-framework] and Verification
   [I-D.draft-lee-sfc-verification-00] for Service Function Chaining. At
   IRTF, the authors of [RFC7149] ask a set of relevant questions
   regarding operations of SDNs. The ETSI NFV ISG defines the MANO
   interfaces [NFVMANO], and TMForum investigates gaps between these
   interfaces and existing specifications in [TR228]. The need for
   programmatic APIs in the orchestration of compute, network and
   storage resources is discussed in
   [I-D.draft-unify-nfvrg-challenges-00].

   From a research perspective, problems related to operations of
   software-defined networks are in part outlined in [SDNsurvey] and
   research referring to both cloud and software-defined networks are
   outlined by the EU FP7 UNIFY project in [D4.1].

   The purpose of this first version of this document is to act as a
   discussion opener in NFVRG by describing a set of principles that are
   relevant for applying DevOps ideas to managing software-defined
   telecom network infrastructures. We identify challenges related to
   developing tools, interfaces and protocols that would support these
   principles and leverage standard APIs for simplifying management
   tasks.



2. Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 [RFC2119].

   In this document, these words will appear with that interpretation
   only when in ALL CAPS. Lower case uses of these words are not to be
   interpreted as carrying RFC-2119 significance.



3. DevOps Principles for Software-Defined Telecom Infrastructure

   In an Internet company, an agile developer is focused on releasing
   small iterations of their code with high velocity and high quality


Meirosu, et al.        Expires August 27, 2015                 [Page 4]


Internet-Draft            DevOps Challenges               February 2015


   into a production environment. The code needs to undergo a
   significant amount of automated testing and verification with pre-
   defined templates in a realistic setting. From the point of view of
   infrastructure management, the verification of the network
   configuration as result of network policy decomposition and
   refinement, as well as the configuration of virtual functions, is one
   of the most sensitive operations. When troubleshooting the cause of
   unexpected behavior, high-granular visibility onto all resources
   supporting the virtual functions (either compute, or network-related)
   is paramount to facilitating fast resolution times. While compute
   resources are typically very well covered by debugging and profiling
   toolsets based on many years of advances in software engineering,
   programmable network resources are a still a novelty and tools
   exploiting their potential are scarce.

   We identify two dimensions of the "developer" role in software-
   defined infrastructure. One dimension refers to the person that
   determines which high-level functions should be part of a particular
   service, decides what logical interconnections are needed between
   these blocks and defines a set of high-level constraints or goals
   related to parameters that define the a Service Function Chain. This
   person might be the product owner for a particular family of services
   offered by a telecom provider. They might be a key account
   representative that adapts an existing service template to the
   requirements of a particular customer by adding or removing a small
   number of functional entities. We refer to this person as the Service
   Developer and for simplicity (access control, training on technical
   background, etc.) we consider the role to be internal to the telecom
   provider. The other dimension of the "developer" role is a person
   that writes the software code for a new virtual network function.
   Depending on the actual virtual network function being developed,
   this person might be internal or external to the telecom provider. We
   refer to them as VNF Developers.

   The role of an Operator in software-defined infrastructure is to
   ensure that the deployment processes were successful and a set of
   performance indicators associated to a service are met while the
   service is supported on virtual infrastructure within the domain of a
   telecom provider.

   In line with the generic DevOps concept outlined in [DevOpsP], we
   consider that the following four principles as important for adapting
   DevOps ideas to software-defined infrastructure:

   * Deploy with repeatable, reliable processes: Service and VNF
   Developers should be supported by automated build, orchestrate and
   deploy processes that are identical in the development, test and


Meirosu, et al.        Expires August 27, 2015                 [Page 5]


Internet-Draft            DevOps Challenges               February 2015


   production environments. Such processes need to be made reliable and
   trusted in the sense that they should reduce the chance of human
   error and provide visibility at each stage of the process, as well as
   have the possibility to enable manual interactions in certain key
   stages.

   * Develop and test against production-like systems: both Service
   Developers and VNF Developers need to have the opportunity to verify
   and debug their respective code in systems that have characteristics
   which are very close to the production environment where the code is
   expected to be ultimately deployed. Customizations of Service
   Function Chains or VNFs could thus be released frequently to a
   production environment in compliance with policies set by the
   Operators. Adequate isolation and protection of the services active
   in the infrastructure from services being tested or debugged should
   be provided by the production environment.

   * Monitor and validate operational quality: Service Developers, VNF
   Developers and Operators must be equipped with tools, automated as
   much as possible, that enable to continuously monitor the operational
   quality of the services deployed on software-defined infrastructure,
   as well as the infrastructure itself. Monitoring tools should be
   complemented by tools that allow verifying and validating the
   operational quality of the service in line with established
   procedures which might be standardized (for example, Y.1564 Ethernet
   Activation [Y1564]) or defined through best practices specific to a
   particular telecom operator.

   * Amplify feedback loops: An integral part of the DevOps ethos is
   building a cross-cultural environment that bridges the cultural gap
   between the desire for continuous change by the Developers and the
   wish by the Operators for stability and reliability of the
   infrastructure, and feedback from customers is collected and
   transmitted throughout the organization. From a technical
   perspective, such cultural aspects could be addressed through common
   sets of tools and APIs that are aimed at providing a vocabulary
   common to Developers and Operators, as well as simplifying the
   reproduction of problematic situations in the development, test and
   operations environments.



4. Stability Challenges

   The dimensions, dynamicity and heterogeneity of networks are growing
   continuously. Monitoring and managing the network behavior in order
   to meet technical and business objectives is becoming more and more


Meirosu, et al.        Expires August 27, 2015                 [Page 6]


Internet-Draft            DevOps Challenges               February 2015


   complicated and challenging, even more when considering the need of
   predicting and taming potential instabilities.

   In general, instability in networks may have primary effects both
   jeopardizing the performance and compromising an optimized use of
   resources, even across multiple layers: in fact, instability of end-
   to-end communication paths may be dependent both on the underlying
   transport network, as well as the higher level components specific to
   flow control and dynamic routing. For example, arguments for
   introducing advanced flow admission control are essentially derived
   from the observation that the network otherwise behaves in an
   inefficient and potentially unstable manner. Even with resources over
   provisioning, a network without an efficient flow admission control
   has instability regions that can even lead to congestion collapse in
   certain configurations. Another example is the instability which is
   characteristic of any dynamically adaptive routing system. Routing
   instability, which can be (informally) defined as the quick change of
   network reachability and topology information, has a number of
   possible origins, including problems with connections, router
   failures, high levels of congestion, software configuration errors,
   transient physical and data link problems, and software bugs.

   As a matter of fact, the states monitored and used to implement the
   different control and management functions in network nodes are
   governed by several low-level configuration commands (today still
   done mostly hand-made); there are several dependencies among these
   states and the logic updating the states (most of which are not kept
   aligned automatically). Normally, high-level network goals (e.g.,
   connectivity matrix, load-balancing, traffic engineering goals,
   survivability requirements, etc) are translated into low-level
   configuration commands (mostly hand-written) individually executed on
   the network elements (e.g., forwarding table, packet filters, link-
   scheduling weights, and queue-management parameters, as well as
   tunnels and NAT mappings). Network instabilities due to configuration
   errors can spread from node to node and propagate throughout the
   network.

   DevOps in the data center is a source of inspiration regarding how to
   simplify and automate management processes for software-defined
   infrastructure.

   As a specific example, automated configuration functions are expected
   to take the form of a "control loop" that monitors (i.e., measures)
   current states of the network, performs a computation, and then
   reconfigures the network. These types of functions must work
   correctly even in the presence of failures, variable delays in
   communicating with a distributed set of devices, and frequent changes


Meirosu, et al.        Expires August 27, 2015                 [Page 7]


Internet-Draft            DevOps Challenges               February 2015


   in network conditions. Nevertheless cascading and nesting of
   automated configuration processes can lead to the emergence of non-
   linear network behaviors, and as such sudden instabilities (i.e.
   identical local dynamic can give rise to widely different global
   dynamics).

5. Consistency, Availability and Partitioning Challenges

   The CAP theorem [CAP] states that any networked shared-data system
   can have at most two of following three properties: 1) Consistency
   (C) equivalent to having a single up-to-date copy of the data; 2)
   high Availability (A) of that data (for updates); and 3) tolerance to
   network Partitions (P). Looking at a telecom software-defined
   infrastructure as a distributed computational system
   (routing/forwarding packets can be seen as a computational problem),
   just two of the three CAP properties will be possible at the same
   time. The general idea is that 2 of the 3 have to be chosen. CP favor
   consistency, AP favor availability, CA there are no partition. This
   has profound implications for technologies that need to be developed
   in line with the "deploy with repeatable, reliable processes"
   principle for configuring the states of the software-defined
   infrastructure. Latency or delay and partitioning properties are
   deeply related, and such relation becomes more important in the case
   of telecom service providers where Devs and Ops interact with widely
   distributed infrastructure. Limitations of interactions between
   centralized management and distributed control need to be carefully
   examined in such environments. Traditionally connectivity was the
   main concern: C and A was about delivering packets to destination.
   The features and capabilities of  SDN and NFV are changing the
   concerns: for example in SDN, control plane Partitions no longer
   imply data plane Partitions, so A does not imply C. In practice, CAP
   reflects the need for a balance between local/distributed operations
   and a remote/centralized operations.

   Furthermore to CAP aspects related to individual protocols,
   interdependencies between CAP choices for both resources and VNFs
   that are interconnected in a forwarding graph need to be considered.
   This is particularly relevant for the  "Monitor and Validate
   Operational Quality" principle, as apart from transport protocols,
   most OAM functionality is generally configured in processes that are
   separated from the configuration of the monitored entities. Also,
   partitioning in a monitoring plane implemented through VNFs executed
   on compute resources does not necessarily mean that the dataplane of
   the monitored VNF was partitioned as well.





Meirosu, et al.        Expires August 27, 2015                 [Page 8]


Internet-Draft            DevOps Challenges               February 2015


6. Observability Challenges

   Monitoring algorithms need to operate in a scalable manner while
   providing the specified level of observability in the network, either
   for operation purposes (Ops part) or for debugging in a development
   phase (Dev part). We consider the following challenges:

   * Scalability - relates to the granularity of network observability,
   computational efficiency, communication overhead, and strategic
   placement of monitoring functions.

   * Distributed operation and information exchange between monitoring
   functions - monitoring functions supported by the nodes may perform
   specific operations (such as aggregation or filtering) locally on the
   collected data or within a defined data neighborhood and forward only
   the result to a management system. Such operation may require
   modifications of existing standards and development of protocols for
   efficient information exchange and messaging between monitoring
   functions. Different levels of granularity may need to be offered for
   the data exchanged through the interfaces, depending on the Dev or
   Ops role.

   * Configurability and conditional observability - monitoring
   functions that go beyond measuring simple metrics (such as delay, or
   packet loss) require expressive monitoring annotation languages for
   describing the functionality such that it can be programmed by a
   controller. Monitoring algorithms implementing self-adaptive
   monitoring behavior relative to local network situations may employ
   such annotation languages to receive high-level objectives (KPIs
   controlling tradeoffs between accuracy and measurement frequency, for
   example) and conditions for varying the measurement intensity.

   * Automation - includes mapping of monitoring functionality from a
   logical forwarding graph to virtual or physical instances executing
   in the infrastructure, as well as placement and re-placement of
   monitoring functionality for required observability coverage and
   configuration consistency upon updates in a dynamic network
   environment.



7. Verification Challenges

   Enabling ongoing verification of code is an important goal of
   continuous integration as part of the data center DevOps concept. In
   a software-defined telecom infrastructure, service definitions,
   decompositions and configurations need to be expressed in machine-


Meirosu, et al.        Expires August 27, 2015                 [Page 9]


Internet-Draft            DevOps Challenges               February 2015


   readable encodings. For example, configuration parameters could be
   expressed in terms of YANG models. However, the infrastructure
   management layers (such as Software-Defined Network Controllers and
   Orchestration functions) might not always export such machine-
   readable descriptions of the runtime configuration state. In this
   case, the management layer itself could be expected to include a
   verification process that has the same challenges as the stand-alone
   verification processes we outline further in this section. In that
   sense, verification can be considered as a set of features providing
   gatekeeper functions to verify both the abstract service models and
   the proposed resource configuration before or right after the actual
   instantiation on the infrastructure layer takes place.

   A verification process can involve different layers of the network
   and service architecture. Starting from a high-level verification of
   the customer input (for example, a Service Graph as defined in [I-
   D.draft-unify-nfvrg-challenges-00]), the verification process could
   go more in depth to reflect on the Service Function Chain
   configuration. At the lowest layer, the verification would handle the
   actual set of forwarding rules and other configuration parameters
   associated to a Service Function Chain instance. This enables the
   verification of more quantitative properties (e.g. compliance with
   resource availability), as well as a more detailed and precise
   verification of the abovementioned topological ones. Existing
   verification tools for the SDN scenario could be deployed in this
   context, but the majority of them only operate on flow space rules
   commonly expressed using OpenFlow syntax.

   Moreover, such verification tools were designed for networks where
   the flow rules are necessary and sufficient to determine the
   forwarding state. This assumption is valid in networks composed only
   by network functions that forward traffic by analyzing only the
   packet headers (e.g. simple routers, stateless firewalls, etc.).
   Unfortunately, most of the real networks contain active network
   functions, represented by middle-boxes that dynamically change the
   forwarding path of a flow according to function-local algorithms and
   an internal state (that is based on the received packets), e.g. load
   balancers, packet marking modules and intrusion detection systems.
   The existing verification tools do not consider active network
   functions because they do not account for the dynamic transformation
   of an internal state into the verification process.

   Defining a set of verification tools that can account for active
   network functions is a significant challenge. In order to perform
   verification based on formal properties of the system, the internal
   states of an active (virtual or not) network function would need to
   be represented. Although these states would cause an increasing of


Meirosu, et al.        Expires August 27, 2015                [Page 10]


Internet-Draft            DevOps Challenges               February 2015


   the verification process complexity (e.g., using simple model
   checking would not be feasible due to state explosion), they help to
   better represent the forwarding behavior in real networks. A way to
   address this challenge is by attempting to summarize the internal
   state of an active network function in a way that allows for the
   verification process to finish within a reasonable time interval.

8. Troubleshooting Challenges

   One of the problems brought up by the complexity introduced by NFV
   and SDN is pinpointing the cause of a failure in an infrastructure
   that is under continuous change. Developing an agile and low-
   maintenance debugging mechanism for an architecture that is comprised
   of multiple layers and discrete components is a particularly
   challenging task to carry out. Verification, observability, and
   probe-based tools are key to troubleshooting processes, regardless
   whether they are followed by Dev or Ops personnel.

   * Automated troubleshooting workflows

   Failure is a frequently occurring event in network operation.
   Therefore, it is crucial to monitor components of the system
   periodically. Moreover, the troubleshooting system should search for
   the cause automatically in the case of failure. If the system follows
   a multi-layered architecture, monitoring and debugging actions should
   be performed on components from the topmost layer to the bottom layer
   in a chain. Likewise, the result of operations should be notified in
   reverse order. In this regard, one should be able to define
   monitoring and debugging actions through a common interface that
   employs layer hopping logic. Besides, this interface should allow
   fine-grained and automatic on-demand control for the integration of
   other monitoring and verification mechanisms and tools.

   * Troubleshooting with active measurement methods

   Besides detecting network changes based on passively collected
   information, active probes into delay, network utilization, loss rate
   are important to debug errors and to evaluate the performance of
   network elements. While tools that are effective in determining such
   conditions for particular technologies were defined by IETF and other
   standardization organization, their use requires a significant amount
   of manual labor in terms of both configuration and interpretation of
   the results. In contrasts, methods that test and debug networks
   systematically based on models generated from the router
   configuration, router interface tables or forwarding tables, would
   significantly simplify management. They could be made usable by Dev
   personnel that have little expertise on diagnosing network defects.


Meirosu, et al.        Expires August 27, 2015                [Page 11]


Internet-Draft            DevOps Challenges               February 2015


   Such tools naturally lend themselves to integration into complex
   troubleshooting workflows that could be generated automatically based
   on the description of a particular service chain. However, there are
   scalability challenges associated with deploying such tools in a
   network. Some tools may poll each networking device for the
   forwarding table information to calculate the minimum number of test
   packets to be transmitted in the network. Therefore, as the network
   size and the forwarding table size increases, forwarding table
   updates for the tools may put a non-negligible load in the network.



9. DevOps Performance Metrics

   Defining a set of metrics that are used as performance indicators is
   important for service providers to ensure the successful deployment
   and operation of a service in the software-defined telecom
   infrastructure.

   We identify three types of considerations that are particularly
   relevant for these metrics: 1) technical considerations directly
   related to the service provided, 2) process-related considerations
   regarding the deployment, maintenance and troubleshooting of the
   service, i.e. concerning the operation of VNFs, and 3) cost-related
   considerations associated to the benefits from using a Software-
   Defined Telecom Infrastructure.

   First, technical performance metrics shall be service-dependent/-
   oriented and may address inter-alia service performance in terms of
   delay, throughput, congestion, energy consumption, availability, etc.
   Acceptable performance levels should be mapped to SLAs and the
   requirements of the service users. Metrics in this category were
   defined in IETF working groups and other standardization
   organizations with responsibility over particular service or
   infrastructure descriptions.

   Second, process-related metrics shall serve a wider perspective in
   the sense that they shall be applicable for multiple types of
   services. For instance, process-related metrics may include: number
   of probes for end-to-end QoS monitoring, number of on-site
   interventions, number of unused alarms, number of configuration
   mistakes, incident/trouble delay resolution, delay between service
   order and deliver, or number of self-care operations.

   Third, cost-related metrics shall be used to monitor and assess the
   benefit of employing Software-Defined Telecom Infrastructure compared
   to the usage of legacy hardware infrastructure with respect to


Meirosu, et al.        Expires August 27, 2015                [Page 12]


Internet-Draft            DevOps Challenges               February 2015


   operational costs, e.g. possible man-hours reductions, elimination of
   deployment and configuration mistakes, etc.

   Finally, identifying a number of highly relevant metrics for DevOps
   and especially monitoring and measuring them is highly challenging
   because of the amount and availability of data sources that could be
   aggregated within one such metric, e.g. calculation of human
   intervention, or secret aspects of costs.





10. Security Considerations

   TBD



11. IANA Considerations

   This memo includes no request to IANA.



12. References

12.1. Normative References

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

12.2. Informative References

   [NFVMANO] ETSI, "Network Function Virtualization (NFV) Management
             and Orchestration V0.6.1 (draft)", Jul. 2014

   [I-D.draft-aldrin-sfc-oam-framework]   S. Aldrin, R. Pignataro, N.
             Akiya. "Service Function Chaining Operations,
             Administration and Maintenance Framework", draft-aldrin-
             sfc-oam-framework-00, (work in progress), July 2014.

   [I-D.draft-lee-sfc-verification-00] S. Lee and M. Shin. "Service
             Function Chaining Verification", draft-lee-sfc-
             verification-00, (work in progress), February 2014.




Meirosu, et al.        Expires August 27, 2015                [Page 13]


Internet-Draft            DevOps Challenges               February 2015


   [I-D. draft-irtf-sdnrg-layer-terminology-04] E. Haleplidis (Ed.), K.
             Pentikousis (Ed.), S. Denazis, J. Hadi Salim, D. Meyer, and
             O. Koufopavlou, "SDN Layers and Architecture Terminology",
             Internet Draft, draft-haleplidis-sdnrg-layer-terminology-04
             (work in progress), October 2014

   [RFC7149] M. Boucadair, C Jaquenet. "Software-Defined Networking: A
             Perspective from within a Service Provider Environment",
             RFC 7149, March 2014.

   [TR228]   TMForum Gap Analysis Related to MANO Work. TR228, May 2014

   [I-D.draft-unify-nfvrg-challenges-00]  R. Szabo et al. "Unifying
             Carrier and Cloud Networks: Problem Statement and
             Challenges", draft-unify-nfvrg-challenges-00 (work in
             progress), October 2014

   [D4.1]    W. John et al. D4.1 Initial requirements for the SP-DevOps
             concept, universal node capabilities and proposed tools,
             August 2014.

   [SDNsurvey] D. Kreutz, F. M. V. Ramos, P. Verissimo, C. Esteve
             Rothenberg, S. Azodolmolky, S. Uhlig. "Software-Defined
             Networking: A Comprehensive Survey." To appear in
             proceedings of the IEEE, 2015.

   [DevOpsP] "DevOps, the IBM Approach" 2013. [Online].

   [Y1564]   ITU-R Recommendation Y.1564: Ethernet service activation
             test methodology, March 2011

   [CAP]     E. Brewer, "CAP twelve years later: How the "rules" have
             changed", IEEE Computer, vol.45, no.2, pp.23,29, Feb. 2012.



13. Acknowledgments

   The research leading to these results has received funding from the
   European Union Seventh Framework Programme FP7/2007-2013 under grant
   agreement no. 619609 - the UNIFY project. The views expressed here
   are those of the authors only. The European Commission is not liable
   for any use that may be made of the information in this document.

   We would like to thank in particular the UNIFY WP4 contributors, the
   internal reviewers of the UNIFY WP4 deliverables, Konstantinos



Meirosu, et al.        Expires August 27, 2015                [Page 14]


Internet-Draft            DevOps Challenges               February 2015


   Pentikousis from EICT, and Wolfgang John from Ericsson for the useful
   discussions and insightful comments.



   This document was prepared using 2-Word-v2.0.template.dot.











































Meirosu, et al.        Expires August 27, 2015                [Page 15]


Internet-Draft            DevOps Challenges               February 2015




Authors' Addresses

   Catalin Meirosu
   Ericsson Research
   S-16480 Stockholm, Sweden
   Email: catalin.meirosu@ericsson.com

   Antonio Manzalini
   Telecom Italia
   Via Reiss Romoli, 274
   10148 - Torino, Italy
   Email: antonio.manzalini@telecomitalia.it

   Juhoon Kim
   Deutsche Telekom AG
   Winterfeldtstr. 21
   10781 Berlin, Germany
   Email: J.Kim@telekom.de

   Rebecca Steinert
   SICS Swedish ICT AB
   Box 1263, SE-16429 Kista, Sweden
   Email: rebste@sics.se

   Sachin Sharma
   Ghent University-iMinds
   Research group IBCN - Department of Information Technology
   Zuiderpoort Office Park, Blok C0
   Gaston Crommenlaan 8 bus 201
   B-9050 Gent, Belgium
   Email: sachin.sharma@intec.ugent.be

   Guido Marchetto
   Politecnico di Torino
   Corso Duca degli Abruzzi 24
   10129 - Torino, Italy
   Email: guido.marchetto@polito.it

   Ioanna Papafili
   Hellenic Telecommunications Organization
   Measurements and Wireless Technologies Section
   Laboratories and New Technologies Division
   2, Spartis & Pelika str., Maroussi,
   GR-15122, Attica, Greece
   Buidling E, Office 102


Meirosu, et al.        Expires August 27, 2015                [Page 16]


Internet-Draft            DevOps Challenges               February 2015


   email: iopapafi@oteresearch.gr
















































Meirosu, et al.        Expires August 27, 2015                [Page 17]