Flow Aware Packet Sampling Techniques
draft-krishnan-ipfix-flow-aware-packet-sampling-00

IPFIX                                                       R. Krishnan
Internet Draft                                                 D. Meyer
Intended status: Experimental                    Brocade Communications
Expires: July 2013                                              Ning So
January 18, 2013                                    Tata Communications

                   Flow Aware Packet Sampling Techniques

          draft-krishnan-ipfix-flow-aware-packet-sampling-00.txt

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79. This document may not be modified,
   and derivative works of it may not be created, and it may not be
   published except as an Internet-Draft.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008. The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on July 18, 2013.

Krishnan                Expires July 18, 2013                  [Page 1]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.

Abstract

   The demands on the networking infrastructure and thus the
   switch/router bandwidths are growing exponentially; the drivers are
   bandwidth hungry rich media applications, inter data center
   communications etc. Using sampling techniques, for a given sampling
   rate, the amount of samples that need to be processed is increasing
   exponentially. This draft suggests flow aware sampling techniques for
   handling various scenarios with minimal sampling overhead.

Table of Contents

   1. Introduction...................................................3
      1.1. Conventions used..........................................3
   2. Flow Aware Packet Sampling.....................................4
      2.1. Long-lived Large Flow Identification......................4
         2.1.1. Automatic identification.............................5
            2.1.1.1. Programmable parameters in Switches and Routers for
            Automatic Identification.................................6
            2.1.1.2. Suggested Technique for Automatic Identification6
   3. Acknowledgements...............................................6
   4. IANA Considerations............................................7
   5. Security Considerations........................................7
   6. Data Model Considerations......................................7

Krishnan                Expires July 18, 2013                  [Page 2]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

   7. References.....................................................7
      7.1. Normative References......................................7
      7.2. Informative References....................................8
   Authors' Addresses................................................8

1. Introduction

   Packet sampling techniques in switches and routers provide an
   effective mechanism for approximate detection of various types of
   flows (long-lived and short-lived) with minimal packet replication
   bandwidth overhead. A large percentage of the packet samples comprise
   of long-lived large flows and a small percentage of the packet
   samples comprise of other flows. The long-lived large flows aka top-
   talkers consume a large percentage of the bandwidth and small
   percentage of the flow space. The other flows, which are the typical
   cause of security threats like Denial of Service (DOS) attacks,
   Scanning attacks etc., consume a small percentage of the bandwidth
   and a large percentage of the flow space. This draft explores light-
   weight techniques for automatically detecting the top-talkers in
   real-time with a high degree of accuracy and sampling only the other
   flows - this makes security threat detection more effective with
   minimal sampling overhead.

1.1. Conventions used

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Krishnan                Expires July 18, 2013                  [Page 3]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

   The following acronyms are used:

   DOS: Denial of Service

   MPLS: Multi Protocol Label Switching

   NVGRE: Network Virtualization using Generic Routing Encapsulation

   TCAM: Ternary Content Addressable Memory

   VXLAN: Virtual Extensible LAN

2. Flow Aware Packet Sampling

   The steps in flow aware packet sampling are described below

   1) Any flow which exceeds minimum flow duration and a minimum
     bandwidth would be characterized as a long-lived large flow. For
     identifying long-lived large flows, use the techniques described
     in Section 2.1. This helps in identifying the long-lived large
     flows aka top-talkers in real-time with a high degree of accuracy.

   2) The identified long-lived large flows can be broadly classified
     into 2 categories as detailed below; these flows can be sampled at
     a low rate or need not be sampled.

        a.  Well behaved (steady rate) long-lived large flows, e.g.
          video streams

        b.  Bursty (fluctuating rate) long-lived large flows e.g. Peer-
          to-Peer traffic

   3) The other flows (excluding the long-lived large flows) can be
     sampled at a normal rate. The other flows can be examined for
     determining security threats like DOS attacks, Scanning attacks
     etc. [LANCOPE]

   For packet sampling, it is recommended to use PSAMP [RFCs 5474-5477]
   or sFlow [RFC 3176].

 2.1. Long-lived Large Flow Identification

   A flow (long-lived large/small flow or short-lived large/small flow)
   can be defined using one or more of the following suggested formats
   as described below

Krishnan                Expires July 18, 2013                  [Page 4]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

     .  IP 5 tuple: IP Protocol, IP source address, IP destination
        address, TCP/UDP source port, TCP/UDP destination port

     .  IP 3 tuple: IP Protocol,  IP source address, IP destination
        address

     .  MPLS Labels

     .  VXLAN, NVGRE

     .  IP source address, IP destination address and IPv6 flow label
        (RFC 6437)

     .  Other formats

   The techniques described in this document are agnostic to the format
   of the flow.

2.1.1. Automatic identification

   Automatic identification of long-lived large flows can be implemented
   in ingress and/or egress processing elements of switches and routers.
   The characteristics of such an implementation would be

     .  Inline solution

     .  Minimal system resources

     .  Maintain line-rate performance

     .  Perform accounting of long-lived large flows with a high degree
        of accuracy

   The advantages and disadvantages of automatic identification are
   detailed below.

   Advantages of Automatic Identification

     .  Accurate identification of long-lived large flows

     .  Real-time identification of long-lived large flows

   Disadvantages of Automatic Identification

     .  Not supported in many switches and routers

Krishnan                Expires July 18, 2013                  [Page 5]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

   The implementation of automatic identification of long-lived large
   flows is vendor dependent. Below is a suggested technique.

   Note: Netflow learns all the flows (long-lived and short-lived) and
   has scalability issues in terms of flow-cache size and CPU
   utilization.

2.1.1.1. Programmable parameters in Switches and Routers for
     Automatic Identification

     .  Minimum measurement interval for determining a candidate long-
        lived large flow (for e.g. 60 seconds)

     .  Minimum bandwidth of long-lived large flow (for e.g. 100 Mbps)

     .  Policy specification (for e.g. flows from a given IP source
        and/or destination address)

2.1.1.2. Suggested Technique for Automatic Identification

   Step 1) If the long-lived large flow exists in a flow-table (e.g.
   TCAM), increment a per flow counter. Else, proceed to Step 2.

   Step 2) There are multiple hash tables, each with a different hash
   function. Each hash table entry has an associated counter. On packet
   arrival, a new flow is looked up in parallel in all the hash tables
   and the corresponding counter is incremented. If the counter exceeds
   a programmed threshold in a given time interval in all the hash table
   entries, a candidate long-lived-flow is learnt and programmed in a
   flow-table.

   There may be some false positives due to multiple other flows
   masquerading as a long-lived large flow; the amount of false
   positives is reduced by parallel hashing using different hash
   functions.

   This technique is also suggested in [draft-krishnan-opsawg-large-flow-
   load-balancing].

3. Acknowledgements

   The authors would like to thank Juergen Quittek for all the support
   and valuable input.

Krishnan                Expires July 18, 2013                  [Page 6]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

4. IANA Considerations

   This memo includes no request to IANA.

5. Security Considerations

   This document does not directly impact the security of the Internet
   infrastructure or its applications. In fact, it proposes techniques
   which could help in identifying a DOS attack pattern.

6. Data Model Considerations

   In Section 2, for exporting the identified long-lived large flows to
   an external entity, it is recommended to use one of the protocols
   recommended in evaluation of candidate protocols for IPFIX [RFC
   3955]. For any packet formats (for e.g. VXLAN, NVGRE) which are not
   covered by the above RFCs, a flow export data model needs to be
   defined - IETF could potentially consider a standards-based activity
   around this.

   Section 2.1.1.1 defines programmable parameters in switches and
   routers for automatic identification. IETF could potentially consider
   a standards-based activity around defining a data model for moving
   this information from a central management entity to the
   switch/router.

7. References

7.1. Normative References

   [1]   Bradner, S., "Key words for use in RFCs to Indicate Requirement
         Levels", BCP 14, RFC 2119, March 1997.

   [2]   Crocker, D. and Overell, P.(Editors), "Augmented BNF for Syntax
         Specifications: ABNF", RFC 2234, Internet Mail Consortium and
         Demon Internet Ltd., November 1997.

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for
             Syntax Specifications: ABNF", RFC 2234, Internet Mail
             Consortium and Demon Internet Ltd., November 1997.

Krishnan                Expires July 18, 2013                  [Page 7]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

7.2. Informative References

   N. Duffield et al., "A Framework for Packet Selection and Reporting",
   RFC 5474, March 2009.

   T. Zseby et al., "Sampling and Filtering Techniques for IP Packet
   Selection", RFC 5475, March 2009.

   B. Claise, Ed. et al., "Packet Sampling (PSAMP) Protocol
   Specifications", RFC 5476, March 2009.

   T. Dietz et al., "Information Model for Packet Sampling Exports", RFC
   5477, March 2009.

   S. Leinen "Evaluation of Candidate Protocols for IP Flow Information
   Export (IPFIX)", RFC 3955, October 2004

   P. Phaal et al. "InMon Corporation's sFlow: A Method for Monitoring
   Traffic in Switched and Routed Networks", RFC 3176, September 2001

   [LANCOPE] Benefits of Flow Analysis Using sFlow: Network Visibility,
   Security  and Integrity
   http://www.lancope.com/files/Lancope_Generic_sFlow_WP.pdf

   [draft-krishnan-opsawg-large-flow-load-balancing] R. Krishnan et
   al., "Best Practices for Optimal LAG/ECMP Component Link Utilization
   in Provider Backbone Networks", January 2013

   Authors' Addresses

   Ram Krishnan
   Brocade Communications
   San Jose, 95134, USA

   Phone: +001-408-406-7890
   Email: ramk@brocade.com

Krishnan                Expires July 18, 2013                  [Page 8]
Internet-Draft  Flow Aware Packet Sampling Techniques      January 2013

   David Meyer
   Brocade Communications
   San Jose, 95134, USA

   Phone: +001-408-333-4193
   Email: dmm@1-4-5.net

   Ning So
   Tata Communications
   Plano, TX 75082, USA

   Phone: +001-972-955-0914
   Email: ning.so@tatacommunications.com

Krishnan                Expires July 18, 2013                  [Page 9]