Internet Draft J. Quittek Document: draft-ietf-ipfix-reqs-05.txt NEC Europe Ltd. Expires: February 2003 T. Zseby FhI FOKUS B. Claise Cisco Systems S. Zander G. Carle FhI FOKUS K.C. Norseth Consultant August 2002 Requirements for IP Flow Information Export <draft-ietf-ipfix-reqs-05.txt> Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of [RFC 2026]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html Distribution of this document is unlimited. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Abstract This memo defines requirements for the export of measured IP flow information out of routers, traffic measurement probes and middleboxes. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 1]
Internet-Draft IPFIX Requirements August 2002 Table of Contents 1 Introduction ................................................. 2 2 Terminology .................................................. 2 2.1 IP Traffic Flow ............................................ 2 2.2 Observation Point .......................................... 3 2.3 Metering Process ........................................... 3 2.4 Flow Record ................................................ 4 2.5 Exporting Process .......................................... 4 2.6 Collecting Process ......................................... 4 3 Applications Requiring IP Flow Information Export ............ 4 3.1 Usage-based Accounting ..................................... 5 3.2 Traffic Profiling .......................................... 5 3.3 Attack/Intrusion Detection ................................. 6 3.4 QoS Monitoring ............................................. 6 4 Distinguishing Flows ......................................... 7 4.1 Interfaces ................................................. 7 4.2 IP Header Fields ........................................... 8 4.3 Transport Header Fields .................................... 8 4.4 MPLS Label ................................................. 8 4.5 DiffServ Code Point ........................................ 8 4.6 Header Compression and Encryption .......................... 8 5 Metering Process ............................................. 9 5.1 Reliability ................................................ 9 5.2 Sampling ................................................... 9 5.3 Overload Behavior .......................................... 10 5.4 Timestamps ................................................. 10 5.5 Time Synchronization ....................................... 10 5.6 Flow Expiration ............................................ 10 5.7 Multicast Flows ............................................ 11 5.8 Ignore Port Copy ........................................... 11 6 Data Export .................................................. 11 6.1 Information Model .......................................... 11 6.2 Data Model ................................................. 13 6.3 Data Transfer .............................................. 13 6.3.1 Congestion Awareness ..................................... 14 6.3.2 Reliability .............................................. 14 6.3.3 Security ................................................. 14 6.4 Regular Reporting Interval ................................. 14 6.5 Notification on Specific Events ............................ 14 6.6 Anonymization .............................................. 15 7 Configuration ................................................ 15 7.1 Configuration of the Metering Process ...................... 15 7.2 Configuration of the Exporting Process ..................... 15 8 General Requirements ......................................... 16 8.1 Openness ................................................... 16 ........................................................... 16 8.3 Several Collecting Processes ............................... 16 9 Special Device Considerations ................................ 16 10 Security Considerations ..................................... 18 Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 2]
Internet-Draft IPFIX Requirements August 2002 10.1 Disclosure of Flow Information Data ....................... 19 10.2 Forgery of Flow Records ................................... 19 10.3 Denial of Service (DoS) Attacks ........................... 20 11 Acknowledgments ............................................. 20 12 References .................................................. 20 13 Authors' Addresses .......................................... 21 14 Full Copyright Statement .................................... 22 Appendix: Derivation of Requirements from Applications ........ 23 1. Introduction There are several applications that require flow-based IP traffic measurements. Such measurements could be performed by a router while forwarding the traffic, by a middlebox [RFC3234], or by a traffic measurement probe attached to a line or a monitored port. This memo defines requirements for exporting traffic flow information out of these boxes for further processing by applications located on other devices. In section 3, a selection of such applications is presented. The following sections list requirements derived from these applications. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2. Terminology The following terminology is used within this document: 2.1. IP Traffic Flow There are several definitions of the term 'flow' being used by the Internet community. Within this document we use the following one: A flow is defined as a set of IP packets passing an observation point in the network during a certain time interval. All packets belonging to a particular flow have a set of common properties. Each property is defined as the result of applying a function to the values of: 1. one or more packet header field (e.g. destination IP address) 2. one or more properties of the packet itself (e.g. packet length) 3. one or more of fields derived from packet treatment (e.g. the BGP Autonomous System number [RFC1771]) Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 3]
Internet-Draft IPFIX Requirements August 2002 A packet is defined to belong to a flow if it completely satisfies all the defined properties of the flow. This definition covers the range from a flow containing all packets observed at a network interface to a flow consisting of just a single packet between two applications with a specific sequence number. Please note that the flow definition does not necessarily match a general application-level end-to-end stream. However, an application may derive properties of application-level streams by processing measured flow data. 2.2. Observation Point The observation point is a location in the network where IP packets can be observed. Examples are a line to which a probe is attached, a shared medium, such as an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) of a router. Note that one observation point may be a superset of several other observation points. For example one observation point can be an entire line card. This would be the superset of the individual observation points at the line card's interfaces. 2.3. Metering Process The metering process generates flow records. Input to the process are IP packet headers observed at an observation point. The metering process consists of a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining flow records. The maintenance of flow records may include creating new records, updating existing ones, computing flow statistics, deriving further flow properties, detecting flow expiration, passing flows record to the exporting process, and deleting flow records. The sampling function and the classifying function may be applied more than once with different parameters. Figure 1 shows the sequence in which the functions are applied. Sampling is not illustrated in the figure, it may be applied before any other function. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 4]
Internet-Draft IPFIX Requirements August 2002 packet header capturing | timestamping | v +----->+ | | | classifying | | +------+ | maintaining flow records | v Figure 1: Functions of the metering process 2.4. Flow Record A flow record contains information about a specific flow that was metered at an observation point. A flow record contains measured properties of the flow (e.g. the total number of bytes of all packets of the flow) and usually also characteristic properties of the flow (e.g. source IP address). 2.5. Exporting Process The exporting process sends flow records to one or more collecting processes. The flow records are generated by one or more metering processes. 2.6. Collecting Process The collecting process receives flow records from one or more exporting processes. The collecting process might store received flow records or further process them, but these actions are out of the scope of this document. 3. Applications Requiring IP Flow Information Export This section describes a selection of applications requiring IP flow information export. Because requirements for flow export listed in further sections below are derived from these applications, their selection is crucial. The goal of this requirements document is not to cover all possible applications with all their flow export requirements, but to cover applications which are considered to be of significant importance in today's or future IP networks, and for Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 5]
Internet-Draft IPFIX Requirements August 2002 which requirements can be met with reasonable technical effort. Please note, that the described applications can have a large number of differing implementations. Requirement details or the weighting of requirements could differ for specific implementations. Therefore we derive the requirements from the general functionality of the selected applications. The list of applications should lead to a better understanding of the requirements which is particularly important when designing or implementing traffic flow metering functions. A detailed overview of which requirement was derived from which application(s) is given in the appendix. 3.1. Usage-based Accounting Several new business models for selling IP service and IP-based services are currently under investigation. Beyond flat rate services which do not need accounting, accounting for these models can be based on time or volume. Accounting data can serve as input for billing systems. Accounting can be performed per user or per user group, it can be performed just for basic IP service or individually per high-level service and/or per content type delivered. For advanced/future services, accounting may also be performed per class of service, per application, per time of day, per used (label switched) path, etc. 3.2. Traffic Profiling Traffic profiling is the process of characterizing IP flows by using a model that represents key parameters of the flows such as flow duration, volume, time and burstiness. It is a prerequisite for network planning, network dimensioning, trend analysis, developing business models, and other activities. It heavily depends on the particular traffic profiling objective(s) what statistics and accuracy are required from the measurements. Typical information needed for traffic profiling is the distribution of used services and protocols in the network, the amount of packets of a specific type (e.g. percentage of IPv6 packets) and specific flow profiles. Since objectives for traffic profiling can vary, this application requires a high flexibility of the measurement infrastructure, especially regarding the options for measurement configuration and packet classification. Traffic Engineering (TE) comprises methods for measurement, modeling, characterization and control of a network. The goal of TE is the optimization of network resource utilization and traffic performance [RFC2702]. Since control and administrative reaction to measurement results requires access to the involved network nodes, TE mechanisms Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 6]
Internet-Draft IPFIX Requirements August 2002 and the required measurement function usually are performed within one administrative domain. Typical parameters required for TE are link utilization, load between specific network nodes, number, size and entry/exit points of the active flows and routing information. 3.3. Attack/Intrusion Detection Capturing of flow information plays an important role for network security, both for detection of security violation, and for subsequent defense. In case of a Denial of Service (DOS) attack, flow monitoring can allow detection of unusual load situations or suspicious flows. In a second step, flow analysis can be performed in order to gather information about the attacking flows, and for deriving a defense strategy. Intrusion detection is a potentially more demanding application which would not only look at specific characteristics of flows, but that may also use a stateful packet flow analysis for detecting specific, suspicious activities, or unusually frequent activities. Such activities may be characterized by specific communication patterns, detectable by characteristic sequences of certain packet types. 3.4. QoS Monitoring QoS monitoring is the passive measurement of quality parameters for IP flows. In contrast to active measurements, passive measurements utilize the existing traffic in the network for QoS analysis. Since no test traffic is sent, passive measurements can only be applied in situations where the traffic of interest is already present in the network. One example application is the validation of QoS parameters negotiated in a service level specification (SLS). Note that passive/active measurement is also referred to as non- intrusive/intrusive measurement or as measurement or observed/synthethic traffic. Passive measurements cannot provide the kind of controllable experiments that can be achieved with active measurements. On the other hand passive measurements do not suffer from undesired side effects caused by sending test traffic (e.g. additional load, potential differences in treatment of test traffic and real customer traffic). QoS monitoring often requires the correlation of data from multiple observation points (e.g. for measuring one-way metrics). This requires proper clock synchronization of the involved metering processes. For some measurements, flow records and/or notifications on specific events at the different observation points must be correlated, for example the arrival of a certain packet. For this, the provisioning of post-processing functions (e.g. the generation of packet IDs) at the metering processes would be useful. Furthermore, Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 7]
Internet-Draft IPFIX Requirements August 2002 QoS monitoring can lead to a huge amount of measurement result data. Therefore it would highly benefit from mechanisms to reduce the measurement data, like aggregation of results and sampling. Please note that not all requirements for QoS monitoring are covered by the IPFIX requirements specified in the following sections. The IPFIX requirements are targeted at per flow information including summaries of per-packet properties for packets within a flow, but not per-packet information itself. For example jitter measurement requires timestamping each packet and reporting of all timestamps of a flow, but the IPFIX requirements only cover timestamps of first and last packet of a flow. 4. Distinguishing Flows Packets are mapped to flows by evaluating their properties. Packets with common properties are considered to belong to the same flow. A packet showing at least one difference in the set of properties is considered to belong to a different flow. The following subsections list a set of properties which a metering process MUST, SHOULD, or MAY be able to evaluate for mapping packets to flows. Please note that requiring the ability to evaluate a certain property does not imply that this property must be evaluated for each packet. In other words, meeting the IPFIX requirements means that the metering process in general must be able, via its configuration, to somehow support to distinguish flows via all the MUST fields, even if in certain circumstance/for certain applications, only a subset of the MUST fields is needed and only a subset of the MUST fields is effectively used to distinguish flows. Which combination of properties is used for distinguishing flows and how these properties are evaluated depends on the configuration of the metering process. The configured choice of evaluated properties strongly depends on the environment and purpose of the measurement and on the information required by the collecting process. For specific deployments, only a subset of the REQUIRED properties listed below can be used to distinguish flows, for example in order to aggregate the flow records and reduce the number of flow records exported. On the other hand, some other deployments will require distinguishing flows by some extra parameters, like for example the TTL field of the IP header or the BGP Autonomous Systems. 4.1. Interfaces The metering process MUST be able to separate flows by the incoming interface or by the outgoing interface or by both of them. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 8]
Internet-Draft IPFIX Requirements August 2002 4.2. IP Header Fields The metering process MUST, SHOULD, or MAY be able to separate flows by the following fields of the IP header as indicated. 1. source IP address (MUST) 2. destination IP address (MUST) 3. protocol type (TCP,UDP,ICMP,...) (MUST) 4. IP version number (SHOULD) This requirement only applies if the the observation point is located at a device that is supporting more than one version of IP. For source address and destination address, separating by full match MUST be supported as well as separation by a partial match (for example subnet masking). 4.3. Transport Header Fields The metering process MUST be able to separate flows by the port numbers of the transport header in case of TCP or UDP being used as transport protocol. Both, source and destination port number MUST be supported for distinguishing flows, individually as well as in combination. 4.4. MPLS Label If the observation point is located at a device supporting Multiprotocol Label Switching (MPLS, see [RFC3031]) then the metering process MUST be able to separate flows by the MPLS label. 4.5. DiffServ Code Point If the observation point is located at a device supporting Differentiated Services (DiffServ) then the metering process MUST be able to separate flows by the DiffServ Code Point (DSCP, see [RFC2474]). 4.6. Header Compression and Encryption If header compression or encryption is used, the metering process might not be able to access all header fields. For packets with compressed or encrypted headers, the requirements stated in this section 4 MUST be met for observation points at end points of header compression or of packet encryption, but they do not need to be met for observation points between the end points. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 9]
Internet-Draft IPFIX Requirements August 2002 5. Metering Process The following are requirements for the metering process. All measurements MUST be conducted from the point of view of the observation point. 5.1. Reliability The metering process MUST either be reliable or missing reliability MUST be known and indicated. The metering process is reliable if each packet passing the observation point is metered according to the configuration of the metering process. If, e.g. due to some overload, not all passing packets can be included into the metering process, then the metering process MUST be able to detect this failure and to report about it. 5.2. Sampling Sampling describes the systematic or random selection of a subset of elements (the sample) out of a set of elements (the parent population). Usually the purpose of applying sampling techniques is to estimate a parameter of the parent population by using only the elements of the subset. Sampling techniques can be applied for instance to select a subset of packets out of all packets of a flow or to select a subset of flows out of all flows on a link. Sampling methods differ in their sampling strategy (e.g. systematic or random) and in the event that triggers the selection of an element. The selection of one packet can for instance be triggered by its arrival time (time-based sampling), by its position in the flow (count-based sampling) or by the packet content (content-based sampling). The metering process MAY support measuring traffic by packet sampling. If sampling is supported the sampling configuration MUST be well defined. The sampling configuration includes the sampling method and all its parameters. If the sampling configuration is changed during operation, the new sampling configuration with its parameters MUST be indicated to all collecting processes receiving the affected flow records. Changing the sampling configuration includes: start sampling, stop sampling, change sampling method, and change sampling parameter. In case of any change in the sampling configuration, all flow records metered by the previous sampling configuration MUST be terminated and exported according to the export configuration. The metering process MUST NOT merge the flow records generated with the new sampling configuration with the flow records generated with the previous sampling configuration. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 10]
Internet-Draft IPFIX Requirements August 2002 5.3. Overload Behavior In case of an overload, for example lack of memory or processing power, the metering process MAY change its behavior in order to cope with the lack of resources. Possible reactions include: - Reduce the number of flows to be metered. This can be achieved by more coarse-grained flow measurement or by a restriction of the flow records to a subset of the set of original ones. - Switch to sampling packets before they are processed by the metering process or - if sampling is already performed - reduce the sampling rate. - Stop metering. - Reducing the resource usage of competing processes on the same device. Example: reducing the packet forwarding throughput Overload behavior is not restricted to the four options listed above. But in case the overload behavior has an impact on the metering process or the exporting process, the overload behavior MUST be clearly defined and the collecting process MUST be able to distinguish the flow records exported before and after the metering process behavior change: in case of any change of the meter's behavior, all flow records metered by the previous behavior MUST be terminated and exported according to the configuration of the exporting process. The metering process MUST not merge the flow records generated with the new behavior with the flow records generated with the previous behavior. 5.4. Timestamps The metering process MUST be able to generate timestamps for the first and the last observed packet of a flow. The timestamp resolution MUST be at least the one of the sysUpTime [RFC1213], which is one centisecond. 5.5. Time Synchronization Metering processes and collecting processes SHOULD be time- synchronized with each other. Using NTP or GPS are possible ways of achieving this. However selecting a method for time synchronization is not in the scope of this document. 5.6. Flow Expiration The metering process MUST be able to detect flow expirations. A flow is considered to be expired if no packet of this flow has been observed for a given timeout interval. The metering process MAY Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 11]
Internet-Draft IPFIX Requirements August 2002 support means for detecting the expiration of a flow before a timeout occurs, for example by detecting the FIN or RST bits in a TCP connection. The procedure for detecting a flow expiration MUST be clearly defined. 5.7. Multicast Flows For multicast flows containing packets replicated to multiple output interfaces, the metering process SHOULD be able to maintain discrete flow records per different output interface. For example, the metering process SHOULD be able to report an incoming multicast packet that is replicated to four output interfaces in four different flow records that differ by the output interface. 5.8. Ignore Port Copy The metering process MAY be able to ignore packets which are generated by a port copy function acting at the device where the observation point of a flow is located. 6. Data Export The following are requirements for exporting flow records out of the exporting process. Beside requirements on the data transfer, we separate requirements concerning the information model from requirements concerning the data model. Furthermore, we list requirements on reporting times, notification on specific events, and on anonymization of flow records. 6.1. Information Model The information model for the flow information export is the list of attributes of a flow to be contained in the report (including the semantics of the attributes). This section lists attributes an exporting process MUST or MAY be able to report. This does not imply that each exported flow records MUST contain all REQUIRED attributes. But it implies that it MUST be possible to configure the exporting process in a way that the information contained in all of the REQUIRED attributes of each exported flow is transmitted from the exporting process to the receiving collecting process(es). In other words, meeting the IPFIX requirements means that the exporting process in general must be able, via its configuration, to somehow support to report all the MUST fields, even if in certain circumstance or for certain applications, only a subset of the MUST fields is needed and only a subset of the MUST fields is effectively reported. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 12]
Internet-Draft IPFIX Requirements August 2002 Beyond that, the exporting process might offer to report further attributes not mentioned here. A particular flow record may contain some of the "REQUIRED" attributes as well as some additional ones, for example covering future technologies. This document does not impose that the following attributes are reported for every single flow record, especially for repetitive attributes. For example, if the observation point is the incoming packet stream at the IP interface with the ifIndex value 3, then this observation point does not have to be exported as part of every single flow record. Exporting it just once might give sufficient information to the collecting process. The exporting process MUST be able to report the following attributes for each metered flow: 1. IP version number This requirement only applies if the observation point is located at a device supporting more than one version of IP. 2. source IP address 3. destination IP address 4. IP protocol type (TCP,UDP,ICMP,...) 5. source TCP/UDP port number 6. destination TCP/UDP port number 7. input interface (ifIndex) This requirement does not apply if the observation point is located at a probe device. 8. output interface (ifIndex) This requirement does not apply if the observation point is located at a probe device. 9. packet counter If a packet is fragmented, each fragment is counted as an individual packet. 10. byte counter Which bytes of a packet are counted MUST be defined exactly. 11. in case of IPv4: Type of Service 12. in case of IPv6: Flow Label 13. if BGP is supported at the observation point: BGP AS number 14. if MPLS is supported at the observation point: MPLS label 15. if DiffServ is supported at the observation point: DSCP 16. timestamp of the first packet of the flow 17. timestamp of the last packet of the flow 18. if sampling is used: sampling configuration 19. unique identifier of the observation point 20. unique identifier of the exporting process Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 13]
Internet-Draft IPFIX Requirements August 2002 The exporting process SHOULD be able to report the following attributes for each metered flow: 21. multicast replication factor the number of outgoing packets originating from a single incoming multicast packet. This is a dynamic property of multicast flows, that may change over time. For unicast flows it has the constant value 1. The exporting process MAY be able to report the following attributes for each metered flow: 22. Time To Live 23. IP header flags 24. TCP header flags 25. dropped packet counter at the observation point If a packet is fragmented, each fragment MUST be counted as an individual packet. 26. fragmented packet counter counter of all packets for which the fragmented bit is set in the IP header 6.2. Data Model The data model describes how information is represented in flow records. The data model MUST be extensible for future attributes to be added. Even if a set of attributes is fixed in the flow record, the data model MUST provide a way of extending the record by configuration or for certain implementations. The data model used for exporting flow information MUST be flexible concerning the flow attributes contained in flow records. A flexible record format would offer the possibility of defining records in a flexible (customizable) way regarding the number and type of contained attributes. The Data Model SHOULD be independent of the underlying transport protocol, i.e. the data transfer. 6.3. Data Transfer Requirements for the data transfer include reliability and security requirements. For meeting these requirements the exporting process can utilize existing security features provided by the device hosting the process and or provided by the transport network. For example it can use existing security technologies for authentication and encryption or it can rely on physical protection of a separated network for transferring flow information. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 14]
Internet-Draft IPFIX Requirements August 2002 6.3.1. Congestion Awareness For the data transfer, a congestion aware protocol MUST be supported. 6.3.2. Reliability Absence of reliability of the data transfer, for example caused by loss or reordering of packets containing flow information, MUST be indicated. Please note that if an unreliable transport protocol is used, reliability can be provided by higher layers. In such a case only lack of overall reliability MUST be indicated. For example reordering could be dealt with by adding a sequence number to each packet. 6.3.3. Security Confidentiality of flow specific data transferred from an exporting process to a collecting process SHOULD be ensured. Integrity of flow specific data transferred from an exporting process to a collecting process MUST be ensured. Authenticity of flow specific data transferred from an exporting process to a collecting process MUST be ensured. The security threats from which these 3 requirements are deducted are explained in the "Security Considerations" (Section 10). sh 2 "Push and Pull Mode Reporting" In general, there are two ways of deciding on reporting times: push mode and pull mode. In push mode, the exporting process decides without an external trigger when to send flow records. In pull mode, sending flow records is triggered by an explicit request from a collecting process. The exporting process MUST support push mode reporting, it MAY support pull mode reporting. 6.4. Regular Reporting Interval The exporting process SHOULD be capable of reporting measured traffic data regularly according to a given interval length. 6.5. Notification on Specific Events The exporting process MAY be capable of sending notifications to a collecting process, if a specific event occurs. Such an event can be for instance the arrival of the first packet of a new flow, or the termination of a flow after flow timeout. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 15]
Internet-Draft IPFIX Requirements August 2002 6.6. Anonymization The exporting process MAY be capable of anonymizing source and destination IP addresses in flow data before exporting them. It MAY support anonymization of port numbers and other fields. Please note that anonymization is not originally an application requirement, but derived from general requirements for treatment of traffic within a network. 7. Configuration 7.1. Configuration of the Metering Process The metering process MUST provide a way of configuring traffic measurement. The following parameters of the metering process SHOULD be configurable: 1. specification of the observation point e.g. an interface or a list of interfaces to be monitored. 2. specifications of flows to be metered 3. flow timeouts The following parameters MAY be configurable: 4. sampling method and parameters, if feature is supported 5. overload behavior, if feature is supported 7.2. Configuration of the Exporting Process The exporting process MUST provide a way of configuring the data export. The following parameters of the exporting process SHOULD be configurable: 1. reporting data format Specifying the reporting data format MUST include a selection of attributes to be reported for each flow. 2. the collecting process(es) to which flows are reported 3. notifications to be sent to the collecting process(es) 4. flow anonymization Only applicable if the exporting process supports flow anonymization. If configuration is done remotely, security SHOULD be provided for the configuration process covering confidentiality, integrity and authenticity. The means used for remote configuration are out of the scope of this document. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 16]
Internet-Draft IPFIX Requirements August 2002 8. General Requirements 8.1. Openness IPFIX specifications SHOULD be open to future technologies. This includes extensibility of configuration of measurement and reporting. Openness is also required concerning the extensibility of the data model, as stated in section 6.2. 8.2. Scalability Concerning the Number of Exporting Processes Data collection from hundreds of different exporting processes MUST be supported. The collecting process MUST be able to distinguish several hundred exporting processes by their identifiers. 8.3. Several Collecting Processes The exporting process MAY be able to export flow information to more than one collecting process. 9. Special Device Considerations This document intends to avoid constraining the architecture of probes, routers, and other devices hosting observation points, metering processes, exporting processes, and/or collecting processes. It can be expected that typically observation point, metering process, and exporting process are co-located at a single device. However, the requirements defined in this document do not exclude devices that derive from this configuration. Figure 2 shows some examples. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 17]
Internet-Draft IPFIX Requirements August 2002 +---+ +-----+ +---------+ +---------+ | E-+-> | E--+-> | E----+-> <-+--E E--+-> | | | | | | | / \ | | | | | | M | | M | | M M | | M M | | | | | /|\ | | /|\ /|\ | | /|\ /|\ | | O | | OOO | | OOO OOO | | OOO OOO | +---+ +-----+ +---------+ +---------+ Probe Basic Complex Multiple Router Router Exporting Processes +---+ +---+ +---+ | E-+-> | E-+-> | E-+------------->---+ | | | | | | | | | +---+ +-+-----+ +-+-+ | M | | M | | E-+------->-+-C-M-E-+-> | | | | | | | | | | +---+ +-+-----+ +-+-+ +-+-+ | O | | M | | E-+->---+ | | | | +---+ | | | | | | | M | +-+-+ | O | | M | | | | | | | +---+ | | | +-----+ | O | | O | | O | ->-+-C-E-+-> +---+ +---+ +---+ +-----+ Protocol Remote Concentrator Proxy Converter Observation Figure 2: IPFIX-related Devices All examples are composed of one or more of the following elements: observation point (O), metering process (M), exporting process (E), collecting process (C). The observation points shown in the figure are always the most fine-granular ones supported by the respective device. A very simple device is a probe. A typical probe contains a single observation point, a single metering process, and a single exporting process. A basic router extends this structure by multiple observation points. Here, the observation point of a particular flow may be one of the displayed most fine-granular observation points, but also it may be a set of them. A more complex router may host more than one metering process, for example one per line card. Please note that here, the observation point of a single flow cannot exceed the set of most fine-granular observation points linked to a single metering process, because only the metering process can merge packets observed at different fine- granular observation points to a joint flow. An observation point containing all most fine-granular observation points of this router Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 18]
Internet-Draft IPFIX Requirements August 2002 is not possible with this structure. Alternatively, a complex router may host different exporting processes for flow records generated by different metering processes. A protocol converter makes use of a metering process that can be accessed only by another protocol than the one defined for IPFIX, for example the SNMP and the Meter MIB module [RFC2720]. Then the exporting process receives flow record from a remote metering process and exports these records using the IPFIX protocol. Please note, that this document does not make any particular assumption on how metering processes and a export processes exchange information, as long as all individual requirements for these processes are met. Also the locations of metering processes are not of any relevance for this document (in contrast to the locations of observation points and the exporting processes). In the example of remote packet observation in Figure 2 the metering process and the observation point are not co-located. Packet header captured at an observation point may be exported as raw data to a device hosting metering process and exporting process. Again, this document does not make any particular assumption on how packet headers are transfered from observation points to metering processes, as long as all requirements for the metering processes are met. An intermediate structure between protocol converter and remote observation (not shown in the Figure) would be a split metering process, for example performing timestamping and sampling at the device hosting the observation point and performing packet classification at another device hosting the exporting process. A concentrator receives flow records via the IPFIX protocol, merges them into more aggregated flow records, and exports them again using the IPFIX protocol. Please note, that for the final flow records the resulting observation point may be a superset of the more fine- granular observation points at the first level devices. The metering process of the final flow records is composed by the (partial) metering processes at the first level devices and the partial metering process at the concentrator. Finally, a very simple IPFIX-related device is a proxy. It just receives flow records using the IPFIX protocol and sends them further using the same protocol. A proxy might be useful for traversing firewalls or other gateways. 10. Security Considerations An IPFIX protocol must be capable to transport data over the public Internet. Therefore it cannot be excluded that an attacker captures or modifies packets or inserts additional packets. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 19]
Internet-Draft IPFIX Requirements August 2002 This section describes security requirements for IPFIX. Like other requirements, the security requirements differ among the considered applications. The incentive to modify collected data for accounting or intrusion detection for instance is usually higher than the incentive to change data collected for traffic profiling. A detailed list of the required security features per application can be found in the appendix. The suggestion of concrete solutions for achieving the required security properties should be part of an IPFIX architecture and protocol is out of scope of this document. Also methods for remote configuration of the metering processes and exporting processes are out of scope. Therefore, threats that are caused by data exchange for remote configuration are not considered here. The following potential security hazards for an IPFIX protocol can be identified: disclosure of IP flow information, forgery of flow records, and Denial of Service (DoS) attacks. 10.1. Disclosure of Flow Information Data The content of data exchanged by an IPFIX protocol (for example IPFIX flow records) should be kept confidential between the involved parties (exporting process and collecting process). Observation of IPFIX flow records gives an attacker information about the active flows in the network, communication endpoints and traffic patterns. This information cannot only be used to spy out user behavior but also to plan and conceal future attacks. Therefore, the requirements specified in section 6.3.3. include confidentiality of the transferred data. This can be achieved for instance by encryption. 10.2. Forgery of Flow Records If flow records are used in accounting and security applications, there are potentially strong incentives to forge exported IPFIX flow records (for example to save money or prevent the detection of an attack). This can be done either by altering flow records on the path or by injecting forged flow records that pretend to be originated by the original exporting process. In order to make an IPFIX protocol resistant against such attacks, authentication and integrity must be provided, as specified in section 6.3.3. Special caution is required if security applications rely on flow measurements. With forged flow records it is possible to trick on security applications. It is for instance possible to pretend that a DoS attack happens without even launching a real attack. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 20]
Internet-Draft IPFIX Requirements August 2002 10.3. Denial of Service (DoS) Attacks DoS attacks on routers or other middleboxes that have the IPFIX protocol implemented would also affect the IPFIX protocol and impair the sending of IPFIX records. Nevertheless, since such hazards are not induced specifically by the IPFIX protocol the prevention of such attacks is out of scope of this document. Nevertheless IPFIX itself also causes potential hazards for DoS attacks. All processes that expect the reception of traffic can be target of a DoS attack. With the exporting process this is only the case if it supports the pull mode (which can be an optional feature of the future IPFIX protocol according to this document). The collecting process always expects data and therefore can be flooded by forged flow records. 11. Acknowledgments We like to thank all the people contributing to the requirements discussion on the mailing list for a lot of valuable comments. 12. References [RFC2026] S. Bradner, "The Internet Standards Process -- Revision 3", RFC 2026, October 1996. [RFC3234] B. Carpenter, "Middleboxes: taxonomy and issues", RFC 3234, February 2002. [RFC2119] S. Bradner "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997. [RFC1771] Y. Rekhter, T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995. [RFC2702] D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, J. McManus, "Requirements for Traffic Engineering Over MPLS", RFC 2702, September 1999. [RFC3031] E. Rosen, A. Viswanathan, R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, January 2001. [RFC2474] K. Nichols, S. Blake, F. Baker, D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 21]
Internet-Draft IPFIX Requirements August 2002 [RFC1213] K. McCloghrie, M. Rose. "Management Information Base for Network Management of TCP/IP-based internets: MIB-II", RFC 1213, March 1991. [RFC2720] N. Brownlee. "Traffic Flow Measurement: Meter MIB", RFC 2720, October 1999. 13. Authors' Addresses Juergen Quittek NEC Europe Ltd., Network Laboratories Adenauerplatz 6 69115 Heidelberg Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de Tanja Zseby Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7153 Email: zseby@fokus.fhg.de Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 Email: bclaise@cisco.com Sebastian Zander Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7287 Email: zander@fokus.fhg.de Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 22]
Internet-Draft IPFIX Requirements August 2002 Georg Carle Fraunhofer Institute for Open Communication Systems (FOKUS) Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7149 Email: carle@fokus.fhg.de K.C. Norseth Consultant 934 S. Palos Verdes Dr. Kaysville, Utah 84037 USA Phone: 801.546.3316 Email: kcn@norseth.com 14. Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 23]
Internet-Draft IPFIX Requirements August 2002 Appendix: Derivation of Requirements from Applications The following table documents, how the requirements stated in sections 3-7 are derived from requirements of the applications listed in section 2. Used abbreviations: M = MUST S = SHOULD O = MAY (OPTIONAL) - = DONT CARE -----------------------------------------------------------------------. IPFIX | ----------------------------------------------------------------. | E: QoS Monitoring | | ----------------------------------------------------------. | | D: Attack/Intrusion Detection | | | ----------------------------------------------------. | | | C: Traffic Engineering | | | | ----------------------------------------------. | | | | B: Traffic Profiling | | | | | ----------------------------------------. | | | | | A: Usage-based Accounting | | | | | | ----------------------------------. | | | | | | | | | | | | | | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4. | DISTINGUISHING FLOWS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4. | Combination of | M | M | M | M | M | M | | | required attributes | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.1. | in/out IF | S | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | src/dst address | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | Masking of IP adresses | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | transport protocol | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.2. | version field | - | S | S | O | O | S | | | | | | (b) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.3. | src/dst port | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 24]
Internet-Draft IPFIX Requirements August 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.4. | MPLS label (a) | S | S | M | O | S | M | | | | | | (c) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 4.5. | DSCP (a) | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5. | METERING PROCESS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.1. | Reliability | M | S | S | S | S | | |-------+-------------------------+-----+-----+-----+-----+-----+ M | | 5.1. | Indication of | - | M | M | M | M | | | | missing reliability | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.2. | Sampling (g) | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.3. | Overload Behavior | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.4. | Timestamps | M | O | O | S | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.5. | Time synchronization | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.6. | Flow timeout | M | S | - | O | O | M | | | | (d) | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.7. | Multicast flows | S | O | O | O | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 5.8. | Ignore port copy | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6. | DATA EXPORT | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | INFORMATION MODEL | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | IP Version | - | M | M | O | O | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | src/dst address | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | transport protocol | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | src/dst port | M | M | - | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Packet counter (h) | S | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Byte counter | M | M | M | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 25]
Internet-Draft IPFIX Requirements August 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Dropped Packet | O | M | M | S | M | M | | | Counter (h,i) | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | ToS Byte | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Flow Label | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | BGP AS# | - | S | M | - | - | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | MPLS label (a) | S | S | M | O | S | M | | | | | | (c) | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | DSCP (a) | M | S | M | O | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Timestamps for | M | O | O | S | S | M | | | first/last packet | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Sampling configuration | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | observation point | M | M | M | M | M | M | | | identifier | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | export process | M | M | M | M | M | M | | | identifier | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | TTL | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | IP header flags | - | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | TCP header flags | - | O | O | O | - | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Fragment counter | - | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.1. | Multicast | O | S | S | - | S | S | | | replication factor | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | DATA MODEL | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | Flexibility | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.2. | Extensibility | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 26]
Internet-Draft IPFIX Requirements August 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3. | DATA TRANSFER | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.1.| Congestion aware | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.2.| Reliability | M | S | S | S | S | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.3.| Confidentiality | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.4.| Integrity | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.3.5.| Authenticity | M | M | M | M | M | M | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | REPORTING TIMES | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | Push mode | M | O | O | M | S | M | | | | | (e) | (e) | |(e,f)| | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4. | Pull mode | O | O | O | O | O | O | | | | | (e) | (e) | | (e) | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.4.1.| Regular Interval | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.6. | Notifications | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 6.7. | Anonymization | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | CONFIGURATION | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Measurement | M | M | M | M | M | M | | | & Data Export | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Observation Point| S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Flow | S | S | S | S | S | S | | | Specifications | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Flow Timeouts | S | S | S | S | O | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Sampling | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 27]
Internet-Draft IPFIX Requirements August 2002 | Sect. | Requirement | A | B | C | D | E | IPFIX| |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Report | S | S | S | S | S | S | | | Data Format | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config | O | O | O | O | O | O | | | Notifications | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 7. | Config Anonymization | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8. | GENERAL REQUIREMENTS | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.1. | Openness | S | S | S | S | S | S | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.2. | Scalability: | | | | | | | | | data collection | M | S | M | O | S | M | | | from hundreds of | | | | | | | | | measurement devices | | | | | | | |-------+-------------------------+-----+-----+-----+-----+-----+------| | 8.3. | Several Collectors | O | O | O | O | O | O | |-------+-------------------------+-----+-----+-----+-----+-----+------| Remarks: (a) If feature is supported. (b) The differentiation of IPv4 and IPv6 is for TE of importance. So we tended to make this a MUST. Nevertheless, a SHOULD seems to be sufficient to perform most TE tasks and allows us to have a SHOULD for IPFIX instead of a MUST. (c) For TE in an MPLS network the label is essential. Therefore a MUST is given here leading to a MUST in IPFIX. (d) Precise time-based accounting requires reaction to a flow timeout. (e) Either push or pull has to be supported. (f) Required, in order to immediately report drop indications for SLA validation. (g) If sampling is supported the methods and parameters MUST be well defined. (h) If a packet is fragmented, each fragment is counted as an individual packet. (i) Only if measurement is done on data path i.e. has access to forwarding decision. Quittek et al. draft-ietf-ipfix-reqs-05.txt [Page 28]
