IPFIX Working Group A. Kobayashi
Internet-Draft K. Ishibashi
Intended status: Informational T. Kondoh
Expires: August 25, 2008 NTT PF Lab.
D. Matsubara
Hitachi
February 22, 2008
Reference Model for IPFIX Mediators
draft-kobayashi-ipfix-mediator-model-02.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 25, 2008.
Copyright Notice
Copyright (C) The IETF Trust (2008).
Kobayashi, et al. Expires August 25, 2008 [Page 1]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Abstract
An IPFIX Mediator is an intermediate device between IPFIX Exporting
Processes and IPFIX Collecting Processes. IPFIX Mediators act as an
IPFIX Proxy, and IPFIX Concentrator. IPFIX Mediators mediate IPFIX
protocol using several functions. That enables the flow-based
measurement system to become a high-capacity system and accommodate a
variety of monitoring methods. This document describes each function
that is provided by IPFIX Mediators and the method of handling the
Flow Records of each function. In addition, this document describes
a model of an applicable scenario using IPFIX Mediators.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Framework for IPFIX Mediators . . . . . . . . . . . . . . . . 6
3.1. Internal Components Model . . . . . . . . . . . . . . . . 6
3.1.1. Collecting Process . . . . . . . . . . . . . . . . . . 7
3.1.2. Metering Process . . . . . . . . . . . . . . . . . . . 7
3.1.3. Exporting Process . . . . . . . . . . . . . . . . . . 10
3.1.4. Storing Process . . . . . . . . . . . . . . . . . . . 11
3.2. IPFIX Protocol Considerations . . . . . . . . . . . . . . 12
3.2.1. Export Time Issue . . . . . . . . . . . . . . . . . . 12
3.2.2. Observation Domain ID Management . . . . . . . . . . . 12
3.2.3. Template Management . . . . . . . . . . . . . . . . . 12
3.2.4. Transport Session Management . . . . . . . . . . . . . 13
3.2.5. Option Template Management . . . . . . . . . . . . . . 13
3.2.6. Reporting of Exporter Information . . . . . . . . . . 13
4. Solution Scenarios with IPFIX Mediators . . . . . . . . . . . 15
4.1. Flexible Aggregation . . . . . . . . . . . . . . . . . . . 15
4.2. Distributed Aggregation . . . . . . . . . . . . . . . . . 15
4.3. Duplication of Flow Records . . . . . . . . . . . . . . . 16
4.4. Distribution of Flow Records . . . . . . . . . . . . . . . 17
4.5. Extraction of Suspicious Flow . . . . . . . . . . . . . . 18
5. Mediator Option Template Presentation . . . . . . . . . . . . 19
5.1. Exporter Information Option Template . . . . . . . . . . . 19
5.2. Usage of Scope Field . . . . . . . . . . . . . . . . . . . 21
6. Security Considerations . . . . . . . . . . . . . . . . . . . 23
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8.1. Normative References . . . . . . . . . . . . . . . . . . . 25
8.2. Informative References . . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26
Intellectual Property and Copyright Statements . . . . . . . . . . 27
Kobayashi, et al. Expires August 25, 2008 [Page 2]
Internet-Draft Reference Model for IPFIX Mediators February 2008
1. Introduction
An IPFIX Mediator is located between one or more Exporting Processes
and one or more Collecting Processes. An IPFIX Mediator acts as a
Collector by receiving Flow Records, and it acts as an Exporter by
sending Flow Records. This dual-role architecture enables cascading
IPFIX Mediators and building a combination of several solutions.
By defining IPFIX Mediators, network operators can take increasing
advantage of an extensive Template format, and handle Flow Records in
accordance with their preference. This document describes a model of
applicable scenarios by using IPFIX Mediator and its key component.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
Kobayashi, et al. Expires August 25, 2008 [Page 3]
Internet-Draft Reference Model for IPFIX Mediators February 2008
2. Terminology
The definitions of basic IPFIX and PSAMP terms are identical with
those in [I-D.ietf-psamp-framework], [RFC3917], [RFC5101], [RFC5102],
and [I-D.ietf-ipfix-architecture]. The terminology related to IPFIX
Mediation is described in [I-D.kobayashi-ipfix-large-ps]. Other than
the above terminology, the following terminology is used in this
document. Therefore, terms defined in the IPFIX terminology are
capitalized in this document.
Metering Process
The Metering Process in IPFIX Mediators can be considered as a
partial Metering Process separated from the Metering Process in
the Original Exporter. The Metering Process in IPFIX Mediators
consists of a set of subprocesses that include the Selecting
Process, Aggregating Process, and Modifying Process. The Metering
Process generates the final Flow Records that should be exported.
Selecting Process
The Selecting Process in an IPFIX Mediator is similar to that of
PSAMP Devices, which is described in [I-D.ietf-psamp-framework].
However, the Selecting Process in an IPFIX Mediator only has
field-match filtering functions. This filtering function blocks
Flow Records based on the values of specified Information Elements
to forward them to the next process. The Selecting Process is one
of the subprocesses in the Metering Process.
Aggregating Process
The Aggregating Process creates aggregated Flow Records from input
Flow Records in accordance with aggregation rules that are
described in [I-D.dressler-ipfix-aggregation]. The Aggregating
Process is one of the subprocesses in the Metering Process.
Modifying Process
The Modifying Process carries out two different kinds of
modification, as follows.
* The Modifying Process changes the Template and record structure
by adding/deleting specific Information Elements. For example,
the Modifying Process adds Information Elements like derived
packet properties that cannot be extracted in the Original
Exporters. Information Elements related to derived packet
properties are described in [RFC5102].
Kobayashi, et al. Expires August 25, 2008 [Page 4]
Internet-Draft Reference Model for IPFIX Mediators February 2008
* The Modifying Process changes the value of the specific
Information Elements. For example, the values of specific
Information Elements are anonymized to avoid violating privacy.
The Modifying Process is a key part of a IPFIX Masquerading Proxy.
The Modifying Process is one of the subprocesses in the Metering
Process.
Storing Process
The Storing Process stores the input Flow Records from any process
in a storage system such as a database or flat-file system.
Stored data may be retrieved from the Storing Process in different
ways, which are outside the scope of this document.
Distributing Function
The Distributing Function classifies input Flow Records based on
the value of specified Information Elements. The classified Flow
Records are exported to the specified Collectors. The
Distributing Function is carried out by the Exporting Process.
Observation Domain ID
An IPFIX Mediator does not host the Observation Points and
Observation Domain. The Observation Domain ID in the IPFIX header
sent by IPFIX Mediator also indicates the largest set of
Observation Points in the Original Exporter, but this value does
not indicate the physical entity of the Original Exporter. If an
IPFIX Mediator handles one Collecting session and one Exporting
session, the IPFIX Mediator does not need to change the value of
the Observation Domain ID. If an IPFIX Mediator handles multiple
sessions on the collecting and exporting side, the IPFIX Mediator
needs to assign a new value.
Transport Session Information
In SCTP, the Transport Session Information is the SCTP
association. In TCP and UDP, the Transport Session Information
corresponds to a 5-tuple {exporter IP address, collector IP
address, Exporter transport port, Collector transport port,
transport protocol}. In IPFIX Mediator, the Collecting Process
manages this information.
Kobayashi, et al. Expires August 25, 2008 [Page 5]
Internet-Draft Reference Model for IPFIX Mediators February 2008
3. Framework for IPFIX Mediators
The internal model of an IPFIX Mediator is composed of Collecting
Processes, Metering Processes, Storing Processes, and Exporting
Processes. An IPFIX Mediator may optionally have Metering Processes
and Storing Processes. Basically, the allocation of these processes
depends on the role of devices, such as IPFIX Proxy, IPFIX
Masquerading Proxy, IPFIX Distributor, and IPFIX Concentrator.
3.1. Internal Components Model
The following figure indicates four components (Collecting Process,
Metering Process, Exporting Process, and Storing Process) within the
IPFIX Mediator that are referred to in [RFC3917]. The Metering
Process can have one or multiple subprocesses. These subprocesses
are the Selecting Process, Aggregating Process, and Modifying Process
that can be connected to each other in any sequence defined by the
user. The Metering Process and Storing Process are options.
+--------------------------------------------------------------+
| IPFIX Mediator |
| .----------. .---------. |
| | | .-------------------------------. | | |
| |Collecting| | Metering Process | |Exporting| |
| |Process | |.-------. .-------. .-------.| |Process | |
| | |--->sub |-->sub |->|sub |-->| | |
IPFIX | ||process| |process| |process|| | IPFIX
--> | |.->|#1 | |#2 | |#3 || | |-->
| | || |'-------' '-------' '-------'| | | |
| | || '----|----------|----------|----' | | |
| | || | | | | | |
| | || .--\/---------\/---------\/--. | | |
| | |'---| Storing Process |<--| | |
| | |--->| |-->| | |
| | | '----------------------------' | | |
| | |------------------------------------>| | |
| '----------' '---------' |
+--------------------------------------------------------------+
Figure A: Key components within IPFIX Mediator.
Each process is associated with a common identifier in the IPFIX
Mediator. This method is similar to PSAMP associations in
[I-D.ietf-psamp-sample-tech].
Kobayashi, et al. Expires August 25, 2008 [Page 6]
Internet-Draft Reference Model for IPFIX Mediators February 2008
3.1.1. Collecting Process
The Collecting Process receives Flow Records from the previous
Exporter. An instance of the process is created according to the
IPFIX session. Functions of the process are described in [RFC5101].
The process also forwards received Flow Records with IPFIX header
information and Transport Session Information to multiple Metering
Processes or Storing Processes. In other words, Flow Records can be
duplicated by forwarding multiple Metering Processes or Exporting
Processes. In addition, the process can directly forward Flow
Records to the Exporting Process.
3.1.2. Metering Process
The Metering Process generates a new set of Flow Records from input
Flow Records with received IPFIX header information, such as "Export
Time" and "Observation Domain ID". The process hosts several
subprocesses. The processing order of these functions, which could
be configured by user definitions would create a different set of
Flow Records.
3.1.2.1. Selecting Process
The Selecting Process decides whether a Flow Record passes through to
the next process. The process has a filtering function and selects
Flow Records that are matched under given conditions. Prior to
receiving Flow Records, the user configures a filter pattern in
Selecting Process, which specifies how the Flow Records are treated
by the process. If the values of some Information Elements in the
Flow Record match the filter pattern, this process selects Flow
Records with all fields and forwards these Flow Records to the next
process. For example, the process selects Flow Records that are
included in the specified destination IP address.
3.1.2.2. Aggregating Process
The Aggregating Process gathers Flow Records within a given time
interval and then distinguishes Flow Records that have common
properties. If values of a given key field are the same, that means
those Flow Records have common properties. This process merges Flow
Records that have a common property and creates an aggregated Flow
Record. Therefore, for example, aggregated Flow Records have an
aggregation counter that indicates the number of packets. These
functions are defined in accordance with the IPFIX aggregation rule
in [I-D.dressler-ipfix-aggregation].
The process has aggregation rule defined by the user prior to
receiving Flow Records. The process indicates Information Elements
Kobayashi, et al. Expires August 25, 2008 [Page 7]
Internet-Draft Reference Model for IPFIX Mediators February 2008
that should become aggregated Flow Keys and other Information
Elements that should be kept or discarded. In addition, these
instruction rules include Information Elements that should be added
to aggregated Flow Records. Aggregated Flow Records may need to
complement information that is discarded during the Aggregating
Process. They help the Collector to analyze aggregated Flow Records.
For example, these Information Elements correspond to
"averageActiveTime", "synCount", and "flowCount" elements, as
follows.
o averageActiveTime
This Information Element indicates the average active time of an
input Flow Record in the aggregated Flow Records. This
Information Element is created from flow time stamp Information
Elements. There are "flowStartSeconds", "flowEndSeconds",
"flowStartMilliSeconds", "flowEndMilliSeconds",
"flowStartSysUpTime", and "flowEndSysUpTime". Moreover,
"minimumActiveTime" and "maxmumActiveTime" might be considered in
addition to the element.
o synCount
This Information Element indicates the number of input Flow
Records that have "tcpControlBits", which the SYN bit sets to 1 in
an aggregated Flow Record. Using this element, Collector can
determine the number of SYN packets throughout the network.
Moreover, "ackCount", "finCount", "pshCount", "urgCount", and
"rstCount" might be considered in addition to this element.
o flowCount
This Information Element is the number of input Flow Records
included in an aggregated Flow Record.
3.1.2.3. Modifying Process
The Modifying Process modifies the input Flow Records. The process
can add new Information Elements, delete included Information
Elements, or modify the value of included Information Elements, as
follows. If this process modifies the original Template, it SHOULD
revise the received "flowKeyIndicator".
Adding new Information Elements
This function adds specified Information Elements into the input
Flow Records. The values of Information Elements are extracted by
searching some database based on the input value of other
Kobayashi, et al. Expires August 25, 2008 [Page 8]
Internet-Draft Reference Model for IPFIX Mediators February 2008
specified Information Elements. The added Information Elements
and used Information Elements are configured according to
instructions by the user to obtain the value. The method to
obtain a value from some Information Elements is outside the scope
of this document.
This function, instead of the Original Exporter, adds a derived
packet property parameter, which is useful for the traffic-
monitoring technique. Doing that can compensate for the inability
of some Exporters to add a derived packet property parameter.
Therefore, the Collector does not need to recognize the difference
between implementations of routers from several vendors. For
example, the addition of "bgpNextHop{IPv4|IPv6}Address" and
"bgpCommunity" Information Elements is useful for making a traffic
matrix that covers the whole network domain. "bgpNextHop{IPv4|
IPv6}Address" can indicate the egress router of some network
domain. In addition, "bgpCommunity" can indicate the same group
of destination or source IP addresses. This value can be given by
looking for the BGP route database based on the destination or
source IP address. In addition, "mplsVpnRouteDistinguisher",
which cannot be extracted from the core router in MPLS networks,
indicates the customer's identification. Network Operators can
monitor the traffic behavior of each customer by adding
"mplsVpnRouteDistinguisher" to the Flow Records. This value can
be given by looking for the BGP route database based on the
"mplsTopLabelStackSection" and "mplsTopLabel{IPv4|IPv6}Address".
Deleting Information Elements
This function deletes specified Information Elements according to
instructions that are configured by the user, which indicate
whether an Information Element should be removed. Hiding network
topology information and private information by using this
function is possible.
In the case of exchanging Flow Records with different network
domains or customers, this function can avoid making a
vulnerability by deleting unnecessary Information Elements. By
deleting unnecessary Information Elements, this function can hide
the network topology and another customer's information. In
particular, "ipNextHopIP{v4|v6}Address", "bgpNextHopIP{v4|
v6}Address", and "bgp{Next|Prev}AdjacentAsNumber" correspond to
network topology information. In addition, MPLS-related
Information Elements, such as "mplsLabelStackSection", that are
useless for customers might be removed in the case of feeding Flow
Records to VPN customers.
Kobayashi, et al. Expires August 25, 2008 [Page 9]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Modifying the value of Information Elements
This function modifies the value of specified Information Elements
according to instructions configured by the user.
For example, this function enables IPFIX Mediator to overwrite
private information with zeros or the maximum value. In
particular, IP address and port number is sensitive private
information. In the case of monitoring traffic trends and traffic
engineering, these Information Elements are not essential factors
for those purposes. In that case, this function anonymizes the
relevant Information Elements to prevent a violation of privacy.
If modification can anonymize some Information Elements, the
function might need to report which Information Elements are
anonymized. For example, "anonymizationIndicator" indicates which
Information Elements have been anonymized as a bitmap, just like
the "flowKeyIndicator". The anonymization method is outside the
scope of this document.
3.1.3. Exporting Process
The Exporting Process forwards Flow Records to the next Collector.
The process manages the reporting Template and makes an IPFIX
datagram.
In addition, the process carries out the Distributing Function as an
option. If this function is enabled, the process classifies Flow
Records based on the value of specified Information Elements and then
exports each classified Flow Record to the individual Collector.
Kobayashi, et al. Expires August 25, 2008 [Page 10]
Internet-Draft Reference Model for IPFIX Mediators February 2008
For example, the Exporting Process classifies Flow Records on the
basis of the peering AS, as shown in the following figure. The set
of classified Flow Records is exported to a dedicated Collector on
the basis of the Peering AS.
+-------------------------------------------+
| IPFIX Mediator .----------. |
| |Exporting | |
| |Process | |
| .----------. .---------. | | | PeerAS#100
Flow -->|Collecting|->|Metering |----->/------------> Collector#A
Records |Process | |Process | | | | | PeerAS#200
| '----------' '---------' | /------------> Collector#B
| | | | | PeerAS#300
| | /------------> Collector#C
| | | | | PeerAS#400
| | /------------> Collector#D
| | | |
| '----------' |
+-------------------------------------------+
Figure B: Exporting each classified Flow Record to dedicated Collector.
3.1.4. Storing Process
The Storing Process stores the input Flow Records from any Processes
in a storage system, such as a database or flat-file system. If the
storing record structure that user wants is different from the
template of input Flow Records, this process selects specified
Information Elements from the input Flow Records using the
instruction rules. Instruction rules configured by user indicate
whether the Information Elements should be stored by using a field
modifier. The field modifier that indicates "keep" or "discard" is
applied to Information Elements within Flow Record and IPFIX header,
such as "Observation Domain ID" and "Export time". This header
information MAY be used when IPFIX datagrams are made of past Flow
Records. That procedure is similar to the instruction of the
Aggregating Process.
When another device retrieves past Flow Records on the basis of a
time period given by a user, the retrieving method can be considered
in many ways. One solution is that another device gets a specified
flat-file from a Mediator and decodes that flat-file by itself.
Other solutions are that another device sends out a query command to
the IPFIX Mediator through XML-RPC, SNMP, or NETCONF, and then, the
IPFIX Mediator exports the specified past Flow Records. This method
is outside the scope of this document.
Kobayashi, et al. Expires August 25, 2008 [Page 11]
Internet-Draft Reference Model for IPFIX Mediators February 2008
3.2. IPFIX Protocol Considerations
This section describes IPFIX Protocol considerations with regard to
IPFIX Mediator.
3.2.1. Export Time Issue
If the Exporting Process writes the "Export Time" of the IPFIX
message when an IPFIX message leaves, an IPFIX Mediator needs to
compensate for any delta time, which is the difference from "Export
Time" of Information Elements contained in each Flow Record by
performing calculations. An IPFIX Proxy MUST reuse the "Export Time"
of received IPFIX messages from the Original Exporter.
3.2.2. Observation Domain ID Management
The Observation Domain ID is locally unique to the Exporting Process
in IPFIX Mediator. To comply with the IPFIX Protocol, the
Observation Domain ID value is RECOMMENDED to be assigned a unique
value per IPFIX Mediator. If an IPFIX Mediator relays an IPFIX
datagram from a transport session to a transport session, IPFIX
Mediator does not need to overwrite the Observation Domain ID with
another value. If an IPFIX Mediator relays an IPFIX datagram from
multiple transport sessions to a single transport session, IPFIX
Mediator needs to overwrite the Observation Domain ID. In that case,
IPFIX Mediator assigns the Observation Domain ID based on received
Transport Session Information and the original Observation Domain ID.
The renewed Observation Domain ID SHOULD be managed using the
received Transport Session Information and original Observation
Domain ID. This linkage information is available for overwriting the
scope field of the Option Template.
Note
If the Metering Process aggregates input Flow Records, the value of
the Observation Domain ID should be 0 to comply with the description
in [RFC5101].
3.2.3. Template Management
The Template ID of a generated Template SHOULD be unique on the basis
of the Observation Domain ID assigned by an IPFIX Mediator. The
Template ID needs to be unique on the basis of IPFIX Mediator when
the Observation Domain ID is 0. If the IPFIX Mediator overwrites the
received Template ID to relay a received Template or modified
Template, the renewed Template ID SHOULD be managed using received
Transport Session Information and the received Observation Domain ID.
This linkage information is available for overwriting the scope field
of an Option Template and Template handling. If IPFIX Mediator
Kobayashi, et al. Expires August 25, 2008 [Page 12]
Internet-Draft Reference Model for IPFIX Mediators February 2008
receives a "Template Withdraw Message", it SHOULD modify this message
to indicate relevant Templates, and send a "Template Withdraw
Message".
3.2.4. Transport Session Management
Each session of the Collecting Process and Exporting Process should
operate independently. Even if one session is reset, the status of
the other session is kept current. However, Templates for resetting
the Collecting Session SHOULD be withdrawn for the Exporting Session.
3.2.5. Option Template Management
IPFIX Mediator MUST check whether the scope field is applicable, if
Data Records associated with Option Templates are exported. If an
IPFIX Mediator rewrites the Observation Domain ID or Template ID,
these values included in scope fields SHOULD be rewritten before
exporting. Instead of exporting Option Template Records and
associated Data Records, Information Elements, such as sampling rate
or sampling method, exported using the Option template Record from
the Original Exporter could be merged in a Flow Record in an IPFIX
Mediator. In that case, IPFIX Mediator MUST modify the relevant
Template Record. Several sorts of received Statistics Option
Template Records and associated Data Records could be exported in
different ways as other Templates. In IPFIX Mediator, the Data
Record associated by Statistics Option Template Records can be
exported after merging its counter. In addition, Statistics Option
Template Records and associated Data Records can be exported by
indicating the source of the statistics data as a scope field instead
of merging the counter. This method is described in Section 5. The
user policy determines whether IPFIX Mediator and the above methods
should export Option Templates Records and associated Data Records.
3.2.6. Reporting of Exporter Information
If IPFIX Mediator acts as an IPFIX Masquerading Proxy or IPFIX Proxy,
reporting the Original Exporter IP address increases the
vulnerability. On the other hand, if IPFIX Mediator acts as other
devices, such as IPFIX Concentrator or IPFIX Distributor, the
Exporter IP address is important information for traffic analysis
such as traffic engineering. In the case of making a traffic matrix,
the Exporter IP address can indicate the ingress router of a network
domain. Therefore, reporting of Exporter Information, such as
Exporter IP address, is useful to identify the Original Exporter.
There are various methods as follows.
An IPFIX Mediator can directly merge Exporter Information into Flow
Records or use Option Templates described in Section 5. If an IPFIX
Kobayashi, et al. Expires August 25, 2008 [Page 13]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Mediator receives Information Elements related to the Exporter
information, IPFIX Mediator SHOULD NOT rewrite its own previous
Exporter information. The IPFIX Mediator can append its own previous
Exporter Information instead of rewriting. In the Collecting
Process, the order of the Exporter information indicates the Original
Exporter and the route of IPFIX Mediator. These methods defined by
user policy determine whether IPFIX Mediator should report Exporter
Information.
Kobayashi, et al. Expires August 25, 2008 [Page 14]
Internet-Draft Reference Model for IPFIX Mediators February 2008
4. Solution Scenarios with IPFIX Mediators
4.1. Flexible Aggregation
An IPFIX Mediator can aggregate Flow Records in the same manner as
that of IPFIX Concentrator and reduce the number of Flow Records
received by a Collector.
The following figure indicates a cascade connection of IPFIX
Mediators. If a Collector measures a traffic matrix to obtain
traffic demand, the Collector needs Flow Records of the whole network
domain, but does not need detailed Flow Records. In the first step,
a first level Mediator receives Flow Records from IPFIX Devices and
then creates aggregated low-level Flow Records. For example, this
step is prefix mask aggregation. Next, a second level Mediator
receives aggregated Flow Records and aggregates them further. For
example, the second step is the aggregation of the BGP next-hop
address and Exporter address. After this, the Collector receives
high-level aggregated Flow Records and then stores them. This method
enables step-by-step aggregation of Flow Records without overloading
a single node.
.--------. .--------.
|IPFIX | |IPFIX |
|router#1|---->|Mediator|---.
| | |*1 | |
'--------' '--------' | .--------. .---------.
'--->|IPFIX | |Traffic |
.--------. .--------. .--->|Mediator|---->|Collector|
|IPFIX | |IPFIX | | |*2 | | |
|router#2|---->|Mediator|---' '--------' '---------'
| | |*1 |
'--------' '--------'
Figure C: Flexible Aggregation with cascading IPFIX Mediators.
4.2. Distributed Aggregation
When the network is used globally, the distances between PoPs become
longer, and the maintenance of a dedicated management network is very
expensive. Therefore, the huge number of Flow Records has burdened
management networks of global ISPs. If network operators place
Mediators at each PoP, the number of Flow Records exported from each
PoP can be reduced. Mediators can minimize the number of Flow
Records exported to the Collector. If the Collector needs detailed
information, it can retrieve Flow Records from Mediators that store
original Flow Records.
Kobayashi, et al. Expires August 25, 2008 [Page 15]
Internet-Draft Reference Model for IPFIX Mediators February 2008
A management network of a global ISP is shown in the following
figure. The Mediators are located at each PoP of the network, and
they collect Flow Records from routers in each PoP domain. The
Mediator reduces the number of Flow Records by aggregating or
filtering, so this system reduces the load of a management network.
POP#Asia
.--------.
.--------. | .---------.
.--------. | |----->|IPFIX |
|IPFIX | |------->|Mediator |----.
|router |---'----->|#1 | |
|#1 |-' '---------' |
'--------' |
|
POP#America |
.--------. |
.--------. | .---------. | .---------.
.--------. | |----->|IPFIX | '---->|Traffic |
|IPFIX | |------->|Mediator |--------->|Collector|
|router |---'----->|#2 | .---->| |
|#4 |-' '---------' | '---------'
'--------' |
|
POP#Europe |
.--------. |
.--------. | .---------. |
.--------. | |----->|IPFIX | |
|IPFIX | |------->|Mediator |----'
|router |---'----->|#3 |
|#7 |-' '---------'
'--------'
Figure D: Traffic monitoring architecture in global network.
4.3. Duplication of Flow Records
An IPFIX Mediator duplicates Flow Records to achieve redundant
storage or utilizes them for several purposes. The pair of
Collecting Process and Metering Process is similar to the pair of the
Observation Point and Metering Process. The Collecting Process
duplicates Flow Records by forwarding them to the multi-Metering
Process.
Kobayashi, et al. Expires August 25, 2008 [Page 16]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Several departments in an ISP want to use the same traffic
information for each intended purpose. The network design department
measures the traffic matrix to obtain traffic demand. The customer
service division uses traffic information for performing accounting
services for each customer while network operators use traffic
information for trouble shooting analysis. That situation is shown
in the following figure. An IPFIX Mediator distributes Flow Records
to several Collectors that have the appropriate aggregated
granularity. In addition, when network operators conduct
troubleshooting, past Flow Records from Mediators can be retrieved.
Measurement traffic matrix.
.--------. .---------.
|IPFIX | |Traffic |
|router#1|----. .---->|Collector|
| | | | |#1 |
'--------' | | '---------'
| | Using Accounting info.
.--------. | .---------. | .---------.
|IPFIX | '---->|IPFIX |----' |Traffic |
|router#2|--------->|Mediator |--------->|Collector|
| | .---->| |----. |#2 |
'--------' | '---------' | '---------'
| | Using Trouble shooting.
.--------. | | .---------.
|IPFIX | | | |Traffic |
|router#3|----' '---->|Collector|
| | |#3 |
'--------' '---------'
Figure E: Duplication of Flow Records for several purposes.
4.4. Distribution of Flow Records
An IPFIX Mediator MAY distribute Flow Records based on the value of
specified Information Elements. This function enables load balancing
of Collector and sorting Flow Records without extra Collector
functions. If Flow Records are used as accounting information,
Mediator can distribute Flow Records to the dedicated Collector of
each customer.
Kobayashi, et al. Expires August 25, 2008 [Page 17]
Internet-Draft Reference Model for IPFIX Mediators February 2008
When network operators disclose traffic information to each customer,
security or the privacy policy should be considered. In that case,
the IPFIX Mediator hides private information about each customer. In
addition, Mediator distributes traffic information based on RD (Route
Distinguisher), ingress IF, peering AS number, or BGP next hop, which
identify the customer. In the following figure, the IPFIX Mediator
distributes Flow Records based on RD. The system securely allows
each customer to access only their own records.
.--------. .---------.
|IPFIX | |Traffic |
|router#1|----. .---->|Collector|<===> Customer#A
| | | | |#1 |
'--------' | | '---------'
| RD=100:1
| .---------. |
.--------. '---->|IPFIX |----' .---------.
|IPFIX | |Mediator | RD=100:2 |Traffic |
|router#2|--------->| |--------->|Collector|<===> Customer#B
| | | | |#2 |
'--------' .---->| |----. '---------'
| '---------' |
| RD=100:3
.--------. | | .---------.
|IPFIX | | | |Traffic |
|router#3|----' '---->|Collector|<===> Customer#C
| | |#3 |
'--------' '---------'
Figure F: Distribution of Flow Records for each customer.
4.5. Extraction of Suspicious Flow
An IPFIX Mediator performs filtering based on the value of specified
Information Elements. Filter conditions are set depending on a
suspicious flow as follows. The Collector receives the specified
suspicious flow and detects an anomalous flow by simply monitoring
the traffic volume of each suspicious flow.
o TCP Flow Records whose "tcpControlBits" value is set to "null"
o TCP Flow Records whose "tcpControlBits" value is set to the SYN
bit only and the packet counter is only 1.
o ICMP Flow Records whose length is too long.
Kobayashi, et al. Expires August 25, 2008 [Page 18]
Internet-Draft Reference Model for IPFIX Mediators February 2008
5. Mediator Option Template Presentation
This section describes Option Templates that are used by IPFIX
Mediators.
5.1. Exporter Information Option Template
Each IPFIX Mediator and final destination Collector needs to know the
Original Exporter and route of IPFIX Mediators. Therefore, each
IPFIX Mediator informs the next Collector about previous Exporter
information, which is the Exporter Information Option Template that
specified the Original Exporter and the route of the IPFIX Mediator.
The final destination Collector can recognize them by receiving this
template. This template is composed of the following Information
Elements.
o exporter{IPv4|IPv6}Address
o collector{IPv4|IPv6}Address
o exporterTransportPort
o collectorTransportPort
o collectorTransportProtocol
o observationDomainId
The Observation Domain ID of the Original Exporter or IPFIX Mediator
is identified by specifying Exporter/Collector Information Elements,
such as "collector{IPv4|IPv6}Address", "collectorTransportPort",
"collectorTransportProtocol", "exporter{IPv4|IPv6}Address",
"exporterTransportPort", and "observationDomainId". The set of
"observationDomainId" and "templateId" or "observationDomainId" might
be used as a scope field. Not all Information Elements are
necessary. For example, the exporter{IPv4|IPv6}Address is necessary
to inform the next Collector about the Original Exporter that created
Flow Records. If the IPFIX Mediator receives this Template, it
SHOULD not overwrite each field. The IPFIX Mediator appends its own
previous Exporter information onto received Data Records specified by
the Exporter Option Template and sends that information to the
Collector. In this manner, the route is maintained until the final
destination Collector.
Kobayashi, et al. Expires August 25, 2008 [Page 19]
Internet-Draft Reference Model for IPFIX Mediators February 2008
The following example describes the cascade connection of IPFIX
Mediators. Each Mediator informs the next Collector about previous
Exporter information.
Session#a Session#b Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:10.1.1.1 IP:10.1.1.2 IP:10.1.1.3
SrcPort:6666 DstPort:4739 DstPort:4739
ODID:10 SrcPort:7777 SrcPort:8888
ODID:0 ODID:0
Figure G: Cascade connection of IPFIX Mediators.
Mediator#1 or Mediator#2 sends a Data Record specified by the
Exporter Option Template. The Data records are shown in Session#b or
Session#c, as follows.
Session#b Data Record:
Field Count = 7
Scope Count = 1
templateId = XXX
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10
Session#c Data Record:
Field Count = 13
Scope Count = 1
templateId = XXX
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10
exporterIPv4Address = 10.1.1.2
collectorIPv4Address = 10.1.1.3
collectorTransportProtocol = 132
exporterTransportPort = 7777
collectorTransportPort = 4739
observationDomainId = 0
Kobayashi, et al. Expires August 25, 2008 [Page 20]
Internet-Draft Reference Model for IPFIX Mediators February 2008
5.2. Usage of Scope Field
An IPFIX Mediator needs to send Options Template Records and
associated Data Records from the Original Exporter. However, IPFIX
Mediator cannot export original Option Template Records and
associated Data Records without modification because changing a
session from an Exporting Process to a Collecting Process causes the
scope fields to have a useless values. When an IPFIX Mediator relays
the Option Template Records that included Observation Domain ID as a
scope field and associated Data Records, an IPFIX Mediator uses the
Exporter Information Option Template. Option Template Records that
were created from an Original Exporter can use all fields of the
Exporter Information Option template as multiple scope fields.
Option Template Records that were created from an IPFIX Mediator can
use some fields of the Exporter Information Option Template as
multiple scope fields. An IPFIX Mediator needs to modify associated
Data Records according to the modified Options Template Record.
However, if each node uses another field, except for the Observation
Domain ID, as the scope, the scope field should be considered on a
case-by-case basis.
The following example describes the cascade connection of IPFIX
Mediators. Router#1 and Mediator#1 export the Metering Process
Statistics Option Template.
Session#a Session#b Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:10.1.1.1 IP:10.1.1.2 IP:10.1.1.3
SrcPort:6666 DstPort:4739 DstPort:4739
ODID:10 SrcPort:7777 SrcPort:8888
ODID:0 ODID:0
Figure H: Cascade connection of IPFIX Mediators.
Mediator#2 exports each Option Template and its Data Record with a
suitable scope.
Session#c Metering Process Statistics Data Records from the Original
Exporter:
Field Count = 15
Scope Count = 12
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10
Kobayashi, et al. Expires August 25, 2008 [Page 21]
Internet-Draft Reference Model for IPFIX Mediators February 2008
exporterIPv4Address = 10.1.1.2
collectorIPv4Address = 10.1.1.3
collectorTransportProtocol = 132
exporterTransportPort = 7777
collectorTransportPort = 4739
observationDomainId = 0
exportedMessageTotalCount
exportedFlowTotalCount
exportedOctetTotalCount
Kobayashi, et al. Expires August 25, 2008 [Page 22]
Internet-Draft Reference Model for IPFIX Mediators February 2008
6. Security Considerations
The IPFIX concentrator uses the IPFIX protocol. Security
considerations about flow information are described in [RFC5101].
Kobayashi, et al. Expires August 25, 2008 [Page 23]
Internet-Draft Reference Model for IPFIX Mediators February 2008
7. IANA Considerations
This document has no actions for IANA.
Kobayashi, et al. Expires August 25, 2008 [Page 24]
Internet-Draft Reference Model for IPFIX Mediators February 2008
8. References
8.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5101] Claise, B., "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic
Flow Information", January 2008.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
January 2008.
8.2. Informative References
[I-D.dressler-ipfix-aggregation]
Dressler, F., Sommer, C., Munz, G., and A. Kobayashi,
"IPFIX Aggregation",
draft-dressler-ipfix-aggregation-04.txt (work in
progress) , November 2007.
[I-D.ietf-ipfix-architecture]
Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export",
draft-ietf-ipfix-architecture-12.txt(work in progress) ,
September 2006.
[I-D.ietf-psamp-framework]
Duffield, N., "A Framework for Packet Selection and
Reporting", draft-ietf-psamp-framework-12.txt , June 2007.
[I-D.ietf-psamp-sample-tech]
Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", draft-ietf-psamp-sample-tech-10.txt ,
June 2007.
[I-D.kobayashi-ipfix-large-ps]
Kobayashi, A., Nishida, H., Sommer, C., Dressler, F., and
E. Stephan, "Problems with Flow Collection in Large-Scale
Networks", draft-kobayashi-ipfix-large-ps-01.txt(work in
progress) , February 2008.
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export(IPFIX)",
October 2004.
Kobayashi, et al. Expires August 25, 2008 [Page 25]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Authors' Addresses
Atsushi Kobayashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3978
Email: akoba@nttv6.net
Keisuke Ishibashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3407
Email: ishibashi.keisuke@lab.ntt.co.jp
Kondoh Tsuyoshi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-2419
Email: kondoh.tsuyoshi@lab.ntt.co.jp
Daisuke Matsubara
Hitachi, Ltd., Central Reseach Laboratory
1-280 Higashi-koigakubo
Kokubunji-shi, Tokyo 185-8601
Japan
Phone: +81-42-323-1111
Email: daisuke.matsubara.pj@hitachi.com
Kobayashi, et al. Expires August 25, 2008 [Page 26]
Internet-Draft Reference Model for IPFIX Mediators February 2008
Full Copyright Statement
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Kobayashi, et al. Expires August 25, 2008 [Page 27]