CoRE L. Braun
Internet-Draft C. Schmitt
Intended status: Standards Track TU Muenchen
Expires: September 2, 2010 B. Claise
Cisco Systems, Inc.
G. Carle
TU Muenchen
March 01, 2010
Compressed IPFIX for smart meters in constrained networks
<draft-braun-core-compressed-ipfix-00>
Abstract
This document specifies the Compressed IPFIX protocol that serves for
transmitting measurement data in 6LoWPAN networks [RFC4944].
Compressed IPFIX is derived from IPFIX [RFC5101] and adopted to the
needs of constrained networks. This documents defines how the
Compressed IPFIX Data and Template Records are transmitted in 6LoWPAN
networks and how Compressed IPFIX data can be converted into
uncompressed IPFIX data.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 2, 2010.
Copyright Notice
Braun, et al. Compressed IPFIX [Page 1]
Internet-Draft Compressed IPFIX March 2010
Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Document structure . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Hard- and Software constraints . . . . . . . . . . . . . . . . 6
3.1. Hardware constraints . . . . . . . . . . . . . . . . . . . 6
3.2. Energy constraints . . . . . . . . . . . . . . . . . . . . 6
3.3. Packet size constraints . . . . . . . . . . . . . . . . . 7
3.4. Transport protocol constraints . . . . . . . . . . . . . . 7
4. Application scenarios for Compressed IPFIX . . . . . . . . . . 8
4.1. Architectures for Compressed IPFIX . . . . . . . . . . . . 9
5. Compressed IPFIX Message Format . . . . . . . . . . . . . . . 11
5.1. Compressed IPFIX Message Header . . . . . . . . . . . . . 12
5.2. Compressed Set . . . . . . . . . . . . . . . . . . . . . . 14
5.3. Compressed Template Record Format . . . . . . . . . . . . 15
5.4. Field Specifier Format . . . . . . . . . . . . . . . . . . 16
5.5. Data Record Format . . . . . . . . . . . . . . . . . . . . 17
6. Compressed IPFIX Mediation . . . . . . . . . . . . . . . . . . 18
6.1. Expanding the Message header . . . . . . . . . . . . . . . 19
6.2. Expanding the Set headers . . . . . . . . . . . . . . . . 20
6.3. Expanding the Template Record Header . . . . . . . . . . . 21
7. Template Management . . . . . . . . . . . . . . . . . . . . . 21
7.1. TCP / SCTP . . . . . . . . . . . . . . . . . . . . . . . . 21
7.2. UDP . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
8. Security considerations . . . . . . . . . . . . . . . . . . . 22
Braun, et al. Compressed IPFIX [Page 2]
Internet-Draft Compressed IPFIX March 2010
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22
10.1. Norminative References . . . . . . . . . . . . . . . . . . 22
10.2. Informative References . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24
Braun, et al. Compressed IPFIX [Page 3]
Internet-Draft Compressed IPFIX March 2010
1. Introduction
Smart meters that form a constrained wireless network need an
application protocol that allows the efficient transmission of
metering data. The meters used to build such networks are usually
equipped with low-cost and low-power hardware. This leads to
constraints in computational capacities, available memory and
networking resources.
The devices are often battery powered and are expected to run for a
long time without having the possibility to re-charge themselves. In
order to save energy, smart meters often power off their wireless
network device. Hence, they don't have a steady network connection,
but are only part of the wireless network as needed when there is
data to transmit. A push protocol like Compressed IPFIX, where data
is transmitted from the meters to one or more collectors only, is
suitable for reporting metering data in such networks.
Compressed IPFIX is derived from IPFIX [RFC5101] and therefore
inherits most of its properties. One of them is the separation of
data and its data description by encoding the former in Data Sets and
the latter in Template Sets.
Transforming Compressed IPFIX to IPFIX as per [RFC5101] is very
simple and can be done on the constrained network border. The
transformation between one form of IPFIX data into another is known
as IPFIX Mediation [Kobayashi09]. Hence, smart metering networks
that are based on Compressed IPFIX can be easily integrated into an
existing IPFIX measurement infrastructure.
1.1. Document structure
Section 2 introduces the used terminology in this draft. Afterwards,
hardware and software constraints in constrained networks, which will
motivate our modifications to the IPFIX protocol, are discussed in
Section 3. Section 4 describes the application scenarios for
Compressed IPFIX and describes possible architectures for the
Compressed IPFIX infrastructure. Section 5 defines the Compressed
IPFIX protocol itself and shows the differences between Compressed
and IPFIX. The Mediation process from Compressed IPFIX to IPFIX is
described in Section 6. Section 7 defines the process of Template
Management on the Exporter and the Collector. Section 8 and
Section 9 discuss the security and IANA considerations for Compressed
IPFIX.
Braun, et al. Compressed IPFIX [Page 4]
Internet-Draft Compressed IPFIX March 2010
2. Terminology
Most of the terms used in this draft are defined in [RFC5101]. All
these terms are written with their first letter being capitalized.
Most of the terms that are defined for IPFIX can be used to describe
Compressed IPFIX. The term "Compressed" is used in front of the
IPFIX term to distinguish between the IPFIX version and the
Compressed IPFIX version, if necessary. This draft uses the
expression IPFIX to refer to IPFIX as per RFC 5101 and the expression
Compressed IPFIX for the IPFIX version defined in this draft.
The terms IPFIX Message, IPFIX Device, Set, Data Set, Template Set,
Data Record, Template Record, Collector and Exporter are defined as
in [RFC5101]. The term IPFIX Mediator is defined in [Kobayashi09].
Each of this terms has a correspondent term in Compressed IPFIX. The
objects behind these terms are also similar to the objects used for
IPFIX. The following list gives a brief overview on the changes to
the IPFIX objects. This brief overview is targeted for readers who
are familiar with IPFIX. A complete definition of these terms is
given throughout this document.
Compressed IPFIX Message
The Compressed IPFIX Message is similar to an IPFIX Message with
these exceptions: The Message Header is substituted with a
Compressed Message Header and the Sets which are contained in the
Compressed Messages are Compressed Sets. The Compressed IPFIX
Message Format is defined in Section 5.
Compressed Data Set
A Compressed Data Set is similar to an IPFIX Data Set. The Set
Header is substituted with a Compressed Set Header. The
Compressed Set Header is defined in Section 5.2.
Compressed Template Set
A Compressed Template Set is a Template Set whose Set Header is
replaced by a Compressed Set Header and whose Template Records are
replaced by Compressed Template Records.
Compressed Data Record
A Compressed Data Record equals an IPFIX Data Record.
Braun, et al. Compressed IPFIX [Page 5]
Internet-Draft Compressed IPFIX March 2010
Compressed Template Record
A Compressed Template Record is similar to a Template Record. The
Template Record Header is substituted with a Compressed Template
Record Header. The Compressed Template Record Format is defined
in Section 5.3.
Compressed IPFIX Mediator
A Compressed IPFIX Mediator is an IPFIX Mediator that is able to
receive and transform Compressed IPFIX Message and to export them
using IPFIX or Compressed IPFIX as shown in Section 6.
A Compressed IPFIX Transport Session is defined by the communication
between an Exporter (identified by an 6LowPAN-Address, the Transport
Protocol, and the Transport Port) and a Collector (identified by the
same properties).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
3. Hard- and Software constraints
3.1. Hardware constraints
The target devices for Compressed IPFIX are usually equipped with
low-cost hardware and therefore face several constraints concerning
CPU, memory and energy resources [Schmitt09]. For example, the IRIS
mote from Crossbow Technologies Inc. has a size of 58 x 32 x 7 mm
(without a battery pack) [Crossbow]. Thus, there is little space for
micro controller, flash memory (128 kb) and radio frequency
transceiver, which are located on the board.
Network protocols used on such hardware need to respect these
constraints. They must be simple to implement using little code and
little run time memory and should produce little overhead when
encoding the application payload.
3.2. Energy constraints
Smart meters that are battery powered have hard energy constraints.
If they run out of power, their battery has to be changed, which
means physical manipulation to the device is necessary. Using as
little energy as possible for network communication is therefore
desired.
Braun, et al. Compressed IPFIX [Page 6]
Internet-Draft Compressed IPFIX March 2010
A smart metering device can save a lot of energy, if it powers down
its radio frequency transceiver. Such devices do not have permanent
network connectivity but are only part of the network as needed. A
push protocol, where only one side is sending data, is suitable for
transmitting application data under such circumstances. As the
communication is unidirectional, a meter can completely power down
its radio frequency transceivers as long as it does not have any data
to sent.
3.3. Packet size constraints
Compressed IPFIX is supposed to be used in 6LoWPAN networks, which
are based on IEEE 802.15.4 [RFC4944]. IEEE 802.15.4 defines a
maximum frame size of 127 octets, which usually leaves 102 octets for
user data. IPv6 defines a minimum MTU of 1280 octets. Fragmentation
has to be implemented in order to transmit such large packets. While
fragmentation allows the transmission of large messages, its use is
problematic in networks with high packet loss because the complete
message has to be discarded if only a single fragment gets lost.
Compressed IPFIX enhances IPFIX by a header compression scheme, which
allows to reduce the overhead from header sizes significantly.
Additionally, the overall Message size is reduced which reduces the
need for fragmentation.
3.4. Transport protocol constraints
The IPFIX standard [RFC5101] defines several transport protocol
bindings for the transmission of IPFIX Messages. SCTP support is
required for any IPFIX Device to achieve standard conformance.
However, sending IPFIX over UDP and TCP may also be implemented.
SCTP is the recommended protocol.
This transport protocol recommendation is not suitable for Compressed
IPFIX. 6LoWPAN defines a compression scheme, which allows to compress
an IPv6 header from 40 octets down to 2 octets. There is a similar
compression scheme for UDP, but there is no such compression for TCP
or SCTP headers. If header compression can be employed, more space
for application payload is available.
Using UDP on the transport layer for transmitting IPFIX Messages is
therefore highly recommended. Furthermore, TCP or SCTP are currently
not supported on some platforms, like on TinyOS [Harvan08]. Hence,
UDP may be the only option.
Every Compressed IPFIX Exporter and Collector MUST implement UDP
transport layer support. It MAY also offer TCP or SCTP support.
However, using these protocols is NOT RECOMMENDED as their use will
Braun, et al. Compressed IPFIX [Page 7]
Internet-Draft Compressed IPFIX March 2010
consume more power and reduces the available size of application
payload compared to the use of UDP. If Compressed IPFIX is
transmitted over a non-constrained network, using SCTP as a transport
layer protocol is RECOMMENDED.
4. Application scenarios for Compressed IPFIX
Compressed IPFIX is derived from IPFIX [RFC5101] and is therefore a
push protocol. This means all communication that employs Compressed
IPFIX is unidirectional. Hence, Compressed IPFIX only fits for
application scenarios where meters transmit data to one or more
Collectors.
If Compressed IPFIX is used over UDP, as recommended, packet loss can
occur. Furthermore, if an initial Template Message gets lost, and is
therefore unknown to the Collector, all Data Sets that reference this
Template cannot be decoded. Hence, all these Messages are lost if
they are not cached by the Collector. It should be clear to an
application developer, that Compressed IPFIX can only be used over
UDP if these Message losses are not a problem.
Compressed IPFIX over UDP is especially not a suitable protocol for
applications where sensor data trigger policy decisions or
configuration updates where packet loss is not tolerable.
Applications that use smart sensors for accounting purposes for long
time measurements can benefit from the use of Compressed IPFIX. One
application for IPFIX can be long term monitoring of large physical
volumes. In [Tolle05], Tolle et al. built a system for monitoring a
"70-meter tall redwood tree, at a density interval of 5 minutes in
time and 2 meters in space". The sensor node infrastructure was
deployed to measure the air temperature, relative humidity and
photosynthetically active solar radiation over a long time period.
Deploying Compressed IPFIX in such scenarios seems to be a good fit.
The sensors can be queried over several 5 minute time intervals and
the query results can be aggregated into a single Compressed IPFIX
Message. As soon as enough query results are stored in a Message,
e.g. if the Message size fills the available payload in a single IEEE
802.15.4 packet, the wireless transceiver can be activated and the
Message can be transmitted to a Compressed IPFIX Collector.
Similar sensor networks have been built to monitor the habitat of
animals, e.g. in the "Great Duck Island Project" [GreatDuck],
[SMPC04]. The purpose of the sensor network was to monitor the birds
by deploying sensors in and around their burrows. The measured
sensor data was collected and stored in a database for offline
Braun, et al. Compressed IPFIX [Page 8]
Internet-Draft Compressed IPFIX March 2010
analysis and visualization. Again, the sensors can perform their
measurements periodically, aggregate the sensor data and export them
to a Compressed IPFIX Collector.
Other application scenarios for Compressed IPFIX could be
applications where sensor networks are used for long term structural
health monitoring in order to investigate long term weather
conditions on the structure of a building. For example, a smart
metering network has been built to monitor the structural health of
the Golden Gate Bridge [Kim07]. If a sensor network is deployed to
perform a long term measurement of the structural integrity,
Compressed IPFIX can be used to collect the sensor measurement data.
If an application developer wants to decide whether to use Compressed
IPFIX for transmitting data from smart meters, he must take the
following considerations into account:
1. The application MUST require a push protocol. It is not possible
to request data from a smart meter. The smart meter decides for
itself when to send its measurement data.
2. The property above allows a smart meter to turn off its wireless
device in order to save energy, as it does not have to receive
any data.
3. The application is allows to accumulate several measurements into
a single packet. Compressed IPFIX easily allows the aggregation
of several measurements into a single Compressed IPFIX Message
(or a single packet). This aggregation can happen on the smart
meter that aggregates several of its own measurements. Or it can
happen within a multi-hop wireless network where one smart meter
aggregates several Compressed IPFIX Messages into a single
Message before forwarding them.
4. The application MUST accept packet loss. Compressed IPFIX only
fits for applications where metering data is stored for
accounting purposes and not for applications where the sensor
data triggers configuration changes or policy decisions (except:
if Message loss is acceptable for some reason).
4.1. Architectures for Compressed IPFIX
Compressed IPFIX Devices can be deployed in different architectures,
which are similar to the ones described in [RFC5470]. The
architecture of these deployment possibilities are described in this
section. One possible architecture is described in figure Figure 1.
Braun, et al. Compressed IPFIX [Page 9]
Internet-Draft Compressed IPFIX March 2010
+----------------+ +----------------+
|[*Application 1]| ... |[*Application n]|
+--------+-------+ +-------+--------+
^ ^
| |
+ = = = = -+- = = = = +
^
|
+------------------------+ +-------+------------------+
| Sensor | Compressed IPFIX | Collector |
|[Exporting Process(es)] |----------------->| [Collecting Process(es)] |
+------------------------+ +--------------------------+
Figure 1: Direct transmission between sensors and applications
A smart meter queries its sensors, encodes the results into a
Compressed IPFIX Message and sends that Message to one or more
Collectors. These Collectors run one or more applications which
process the collected sensor data. Such Collectors can be non-
constrained devices at the constraint network border.
A second architecture could employ aggregation on Compressed IPFIX
Messages during their journey through the constrained network
(Figure 2). This aggregation can be performed by special aggregator
nodes, which must have enough resources to perform the aggregation.
+------------------------+ +-----------------------+
| Sensor | Compressed IPFIX | Aggregator |
|[Exporting Process] |------------------>| [Collecting Process] |
+------------------------+ +--------->| [Exporting Process] |
| +-----------------------+
+------------------------+ | |
| Sensor | | Compressed IPFIX|
|[Exporting Process] |--------+ |
+------------------------+ v
+-------+------------------+
| Collector(1) |
| [Collecting Process(es)] |
+--------------------------+
Figure 2: Aggregation on Compressed IPFIX
Several smart meters send their data to one aggregator which needs to
have enough storage space to store the incoming data. It may also
aggregate the incoming data with its own measurement data. The
Braun, et al. Compressed IPFIX [Page 10]
Internet-Draft Compressed IPFIX March 2010
aggregated data can then be re-exported again to one or more
Collectors. The aggregator is then one form of a Compressed IPFIX
Mediator.
The last scenario, shown in Figure 3, employs another Compressed
IPFIX Mediation process.
+------------------------+ +-----------------------+
| Sensors | Compressed IPFIX | IPFIX mediator |
|[Exporting Processes] |----------------->| [Collecting Process] |
+------------------------+ | [Exporting Process] |
+-----------------------+
|
IPFIX |
|
v
+-------+------------------+
| Collector(1) |
| [Collecting Process(es)] |
+--------------------------+
Figure 3: Aggregation on Compressed IPFIX
The smart meters transmit their Compressed IPFIX Messages to one
node, e.g. the base station, which translates the Compressed IPFIX
Messages to IPFIX. The IPFIX Messages can then be exported into the
existing IPFIX infrastructure. The Mediation process from Compressed
IPFIX to IPFIX is described in Section 6.
Compressed IPFIX fits especially into those scenarios where sensors
report their measurement data for accounting purposes and where
packet loss is acceptable.
5. Compressed IPFIX Message Format
A Compressed IFPIX Message starts with a Message header, followed by
one or more Sets. The Sets can be any of the possible two types:
Template Set and Data Set. An IPFIX Message MUST only contain one
type of Set. The structure of the Compressed IPFIX message equals the
structure of the IPFIX Messages with the following exceptions:
1. The Message header, the Set Header and the Template Header format
use compression to reduce the header sizes. This compression
leads to the fact that only a subset of possible IPFIX Messages
can be encoded in Compressed IPFIX message. However, each
Compressed IPFIX Message can be transformed into an IPFIX
Braun, et al. Compressed IPFIX [Page 11]
Internet-Draft Compressed IPFIX March 2010
Message.
2. There are no Option Sets in Compressed IPFIX.
The format of the Compressed IPFIX Message is shown in Figure 4
+----------------------------------------------------+
| Compressed Message Header |
+----------------------------------------------------+
| Compressed Set |
+----------------------------------------------------+
| Compressed Set |
+----------------------------------------------------+
...
+----------------------------------------------------+
| Compressed Set |
+----------------------------------------------------+
Figure 4: Compressed IPFIX Message Format
5.1. Compressed IPFIX Message Header
The Compressed IPFIX Message header is derived from the IPFIX Message
header. This is done by using compression on some of the header
fields. The original Message Header is shown in Figure 5. Its
length is 16 octets and every IPFIX Message has to be started with
this header.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: IPFIX Message Header
In order to reduce the header overhead from prepending a 16 octet
message header, Compressed IPFIX introduces a Compressed IPFIX
Braun, et al. Compressed IPFIX [Page 12]
Internet-Draft Compressed IPFIX March 2010
Message Header that can reduce the header length to two octets. The
Compressed header consists of a fixed part of two octets and a
variable length "Remaining Header" as shown in Figure 6.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version|ETC|SNC| Length |
|Number | | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remaining Header |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: Format of the Compressed IPFIX Message header
The first part has a fixed length of two octets and consists of the
"Version Field" (4 bit), the "Export Time Compression" (ETC) field (2
bit), the "Sequence Number Compression" (SNC) field (2 bit) and the
"Length" field (8 bit). The second part (the "Remaining Header") has
a variable length. Its length is defined by the ETC and SNC fields
in the fixed header.
The fixed header has a length of two octets which equals the length
of the version field of the IPFIX Header. Hence, Compressed IPFIX
messages can be read and identified by an IPFIX Collector. This is
important for building an IPFIX Mediator by extending an IPFIX
Collector (Section 6).
The fixed header fields are defined as follows:
Version number
The Compressed IPFIX version field MUST have the most significant
bit set to one and the other bits set to zero. The remaining bits
of the version field are reserved for future versions of
Compressed IPFIX. Note that IPFIX has the version 0x000a, hence
an IPFIX Collector can distinguish between IPFIX and Compressed
IPFIX by checking the first bit of the version field.
ETC
The ETC field defines the compression level of the "Export Time"
field of the IPFIX Messages Header. Its value defines the length
as follows. A bit sequence of "00" denotes that the "Export Time"
field is omitted. A sequence of "01" denotes that the "Export
Braun, et al. Compressed IPFIX [Page 13]
Internet-Draft Compressed IPFIX March 2010
Time" field has a size of one octet. A sequence of "10" denotes
that the "Export Time" field has a size of two octets. Finally, a
sequence of "11" denotes that the "Export Time" field has the
original length of four octets.
SNC
The SNC field defines the compression level of the "Sequence
Number" field of the IPFIX Messages Header. Its value defines the
length as follows. A bit sequence of "00" denotes that the
"Sequence Number" field is omitted. A sequence of "01" denotes
that the "Sequence Number" field has a size of one octet. A
sequence of "10" denotes that the "Sequence Number" field has a
size of two octets. Finally, a sequence of "11" denotes that the
"Sequence Number" field has the original length of four octets.
Length
The length field has a fixed length of one octet. Compressed
IPFIX messages therefore have a maximum length of 255 octets. An
application SHOULD never send a Compressed IPFIX that is bigger
than 102 octets to avoid fragmentation.
If the "Export Time" field is not omitted, it is placed directly
behind the length field. If the Export Time field has a size of four
octets, it MUST contain the time in seconds since 0000 UTC Jan 1,
1970, at which the IPFIX Message Header leaves the Exporter. This
complies with the "Export Time" field in IPFIX.
Afterwards, the "Sequence Number" field is attached (if not omitted).
If the field has a length of four bytes, it must contain the number
of records sent since the start of the Exporter module 2^32 at the
end of this message. If the field is Compressed to one or two bytes,
it must contain the number of IPFIX messages sent by the Exporter
since its start modulo 2^8 or 2^16.
5.2. Compressed Set
The IPFIX Set Header consists of an two octet "Set ID" field and a
two octet "Length" field. These two fields are compressed to one
octet each for the Compressed Set Header. The format of the
Compressed Set Header is shown in Figure 7.
Braun, et al. Compressed IPFIX [Page 14]
Internet-Draft Compressed IPFIX March 2010
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: Compressed Set Header
The two fields are defined as follows:
Set ID
The "Set ID" identifies the type of data that is transported in
the Set. A Template Set is identified by Set ID 2. This
corresponds to the Set IDs that are used by IPFIX. ID number 3
MUST NOT be used. All values from 4 to 127 are reserved for
future use. Values above 127 are used for Data Sets.
Length
The "Length" Field contains the total length of the set, including
the Compressed Set Header. This length MUST be set to the correct
value. It is necessary for an IPFIX Mediator as it only has to
parse and expand the Set headers and needs the length information
to proceed to the next Set header.
5.3. Compressed Template Record Format
The format of the Compressed Template Records is shown in Figure 8.
It equals the format of IPFIX Template Records. The Compressed
Template Record starts with an Compressed Template Record Header and
is followed by one or more Field Specifiers. The Field Specifier
format is defined as in Section 5.4 and is identical to the Field
specifier definition in [RFC5101].
+--------------------------------------------------+
| Compressed Template Record Header |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
...
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
Braun, et al. Compressed IPFIX [Page 15]
Internet-Draft Compressed IPFIX March 2010
Figure 8: Compressed Template Format
The format of the Template Record Header is shown in Figure 9
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Temp ID(> 127) | Field Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9: Compressed Template Header
Temp ID
Each Template Record must have a unique Template ID between 128
and 255. The Template ID must be unique for the given Compressed
Transport Session.
Field Count
The number of fields placed in the Template Record.
5.4. Field Specifier Format
The type and length of the transmitted data is encoded in Field
Specifiers within Template Records. The Field Specifier is shown in
Figure 10 and is identical with the Field Specifier that was defined
for IPFIX.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|E| Information Element ident. | Field Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10: Compressed Template Header
Where:
Braun, et al. Compressed IPFIX [Page 16]
Internet-Draft Compressed IPFIX March 2010
E
Enterprise bit. This is the first bit of the Field Specifier. If
this bit is zero, the Information Element Identifier identifies an
IETF-specified Information Element, and the four-octet Enterprise
Number field MUST NOT be present. If this bit is one, the
Information Element Identifier identifies an enterprise-specific
Information Element, and the Enterprise Number field MUST be
present.
Information Element Identifier
A numeric value that represents the type of Information Element.
Field Length
The length of the corresponding encoded Information Element, in
octets. Refer to [RFC5102]. The value 65535 is illegal as there
are no variable size encoded elements as they are defined in
IPFIX.
Enterprise Number
IANA [IANA] enterprise number of the authority defining the
Information Element identifier in this Template Record.
Vendors can easily define their own data model by registering a
Enterprise ID with IANA. Using their own Enterprise ID, they can use
any ID in the way they want them to use.
5.5. Data Record Format
The Data Records are sent in Compressed Data Sets. The format of the
Data Records is shown in Figure 11 and matches the Data Record format
from IPFIX.
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
...
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
Braun, et al. Compressed IPFIX [Page 17]
Internet-Draft Compressed IPFIX March 2010
Figure 11: Data Record Format
6. Compressed IPFIX Mediation
There are two types of Compressed IPFIX mediation processes. The
first one can occur on the transition between a constraint 6LoWPAN
and the non-constrained network. This mediation changes the network
and transport protocol from 6LowPAN/UDP to IP/(SCTP|TCP|UDP) and is
shown in Figure 12.
+------------------------+ Compressed IPFIX +-----------------------+
| Sensors | 6LoWPAN/UDP | IPFIX mediator |
|[Exporting Processes] |----------------->| [Collecting Process] |
+------------------------+ | [Exporting Process] |
+-----------------------+
|
Compressed IPFIX |
IP/(UDP/SCTP|TCP) |
v
+-------+------------------+
| Collector(1) |
| [Collecting Process(es)] |
+--------------------------+
Figure 12: Transformation from compressed IPFIX over 6LowPAN/UDP to
IP/(SCTP|TCP|UDP)
The mediator removes the Compressed IPFIX Messages from the 6LowPAN/
UDP packets and wraps them into the new network and transport
protocols. Templates MUST be managed the same way as in the
constraint environment after the translation to IP/(SCTP|UDP|TCP)
(see Section 7).
The second type of mediation transforms Compressed IPFIX into IPFIX.
This process MUST be combined with the transport protocol mediation
as shown in Figure 13.
Braun, et al. Compressed IPFIX [Page 18]
Internet-Draft Compressed IPFIX March 2010
+------------------------+ Compressed IPFIX +-----------------------+
| Sensors | 6LoWPAN/UDP | IPFIX mediator |
|[Exporting Processes] |----------------->| [Collecting Process] |
+------------------------+ | [Exporting Process] |
+-----------------------+
|
IPFIX |
IP/(UDP/SCTP|TCP) |
v
+-------+------------------+
| Collector(1) |
| [Collecting Process(es)] |
+--------------------------+
Figure 13: Transformation from Compressed IPFIX to IPFIX
This mediation can also be performed by an IPFIX Collector before
parsing the IPFIX message as shown in Figure 14. There is no need
for a Compressed IPFIX parser if such a mediation process can be
employed in front of an already existing IPFIX collector.
+------------------------+ Compressed IPFIX +-----------------------------+
| Sensors | 6LoWPAN/UDP | IPFIX mediator |
|[Exporting Processes] |----------------->| [Collecting Process] |
+------------------------+ | [Exporting Process] |
| | |
| |IPFIX |
| | |
| v |
| Collector(1) |
| [Collecting Process] |
+-----------------------------+
Figure 14: Transformation from Compressed IPFIX to IPFIX
The mediation process has to uncompress the IPFIX Message header, the
Set Headers and the Template Record Header. Afterwards, the new
Message Length needs to be calculated and inserted into the Message
header.
6.1. Expanding the Message header
The fields of the IPFIX Message Header that are shown in Figure 5 can
be determined as follows:
Braun, et al. Compressed IPFIX [Page 19]
Internet-Draft Compressed IPFIX March 2010
Version
This is always 0x000a.
Length
The IPFIX Message length can only be calculated after the complete
message has been expanded. The new length can be calculated by
adding the length of the IPFIX message header, which is 16 octets,
and the length of all Sets that are contained in the IPFIX
Message.
Export Time
If the "Export Time" in the Compressed IPFIX Message Header has a
length of 4 octets, the original value MUST be used for the IPFIX
Message. If it was omitted, the "Export Time" MUST be generated
by the Mediator. If the IPFIX Message is exported again, the
"Export Time" field MUST contain the time in seconds since 0000
UTC Jan 1, 1970, at which the IPFIX Message leaves the Exporter.
If the Message is passed to an IPFIX Collector for decoding
directly, the "Export Time" field is the time in seconds since
0000 UTC Jan 1 1970 at which the Compressed IPFIX Message has been
received.
Sequence Number
If the "Sequence Number" is not compressed, the original value
MUST be used for the IPFIX message. If the number was compressed
to one or two octets, the IPFIX Mediator MUST expand the
Compressed Sequence Number into a four octet field. If the
Sequence Number was omitted, the Mediator needs to calculate the
Sequence Number as per RFC 5101 [RFC5101].
Observation Domain ID
This is always 0 indicating to the IPFIX Collector, that the
Observation Domain ID is not relevant.
6.2. Expanding the Set headers
Both fields in the Compressed Set header are compressed and need to
be expanded:
Braun, et al. Compressed IPFIX [Page 20]
Internet-Draft Compressed IPFIX March 2010
Set ID
The field needs to be expanded from one octet to two octets. If
the Set ID is below 128, no recalculation needs to be performed.
This is because all IDs below 128 are reserved for special
messages and match the IDs used in IPFIX. The Compressed Set IDs
starting with 128 identify Data Sets. Therefore, every Compressed
Set ID above 127 needs to be incremented by 128 because IPFIX Data
Set IDs are located above 255.
Set Length
The field needs to be expanded from one octet to two octets. It
needs to be recalculated by adding a value of 2 octets to match
the additional size of the expanded Set Header. For each Template
Record that is contained in the Set, 2 more octets need to be
added to the length.
6.3. Expanding the Template Record Header
Both fields in the Compressed Template Record Header are Compressed
and therefore need expansion:
Template ID
The field needs to be expanded from one octet to two octets. The
Template ID needs to be increased by a value of 128.
Field Count
The field needs to be expanded from one octet to two octets.
7. Template Management
The way Templates have to be managed depends on the transport
protocol in use. If TCP or SCTP is used, it can be ensured that
Templates are delivered reliably. Template loss can occur on UDP on
the other hand. If a Template is lost on its way to the Collector,
all following Data Records that refer to this Template cannot be
decoded.
7.1. TCP / SCTP
If TCP or SCTP is an option and can be used for the transmission of
Compressed IPFIX, Template Management MUST be performed as
standardized in [RFC5101] for IPFIX.
Braun, et al. Compressed IPFIX [Page 21]
Internet-Draft Compressed IPFIX March 2010
7.2. UDP
Compressed IPFIX Templates MUST be sent by an Exporter before any
Data that refers to the Template is transmitted. Templates are not
expected to change over time in Compressed IPFIX. Hence, a Template
that has been sent once MAY NOT be withdrawn and MUST NOT expire. If
a Sensor Node wants to use another Template it MUST use a new
Template ID for this different Template.
As UDP is used, reliable transport of Templates cannot be guaranteed
and Templates can be lost and a Compressed Exporter MUST expect
Template loss. It MUST therefore re-send its template periodically.
A Template MUST be re-send after a fixed number of N IPFIX Messages
that contained Data Sets that referred to this Template. The number
N may be chosen by the application developer.
8. Security considerations
There are no security considerations defined in this draft (yet).
There should be though ...
9. IANA Considerations
The Compressed IPFIX version number needs to be registered with IANA.
The Set ID numbers used in this draft are already registered and
their meaning is not changed.
New assignments in either IPFIX Version Number or IPFIX Set ID
assignments require a Standards Action [RFC2434], i.e., they are to
be made via Standards Track RFCs approved by the IESG.
10. References
10.1. Norminative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2434] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 2434,
October 1998.
[RFC4944] Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler,
"Transmission of IPv6 Packets over IEEE 802.15.4
Networks", RFC 4944, September 2007.
Braun, et al. Compressed IPFIX [Page 22]
Internet-Draft Compressed IPFIX March 2010
[RFC5101] Claise, B., "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic
Flow Information", RFC 5101, January 2008.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
RFC 5102, January 2008.
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export", RFC 5470,
March 2009.
[Kobayashi09]
Kobayashi, A. and B. Claise, "IPFIX Mediation: Problem
Statement",
draft-ietf-ipfix-mediators-problem-statement-07 ,
December 2009.
[Shelby09]
Shelby, Z., Garrison Stuber, N., Sturek, D., Frank, B.,
and R. Kelsey, "CoAP Feature Analysis",
draft-shelby-6-lowapp-coap-00 , December 2009.
10.2. Informative References
[IANA] "IANA Private Enterprise Numbers registry
http://www.iana.org/assignments/enterprise-numbers.".
[Schmitt09]
Schmitt, C. and G. Carle, "Applications for Wireless
Sensor Networks", In Handbook of Research on P2P and Grid
Systems for Service-Oriented Computing: Models,
Methodologies and Applications, Antonopoulos N.;
Exarchakos G.; Li M.; Liotta A. (Eds.), Information
Science Publishing. , 2010.
[Tolle05] Tolle, G., Polastre, J., Szewczyk, R., Turner, N., Tu, K.,
Buonadonna, P., Burgess, S., Gay, D., Hong, W., Dawnson,
T., and D. Culler, "A macroscope in the redwoods", In the
Proceedings of the 3rd ACM Conference on Embedded
Networked Sensor Systems (Sensys 05), San Diego, ACM
Press , November 2005.
[Kim07] Kim, S., Pakzad, S., Culler, D., Demmel, J., Fenves, G.,
Glaser, S., and M. Turon, "Health Monitoring of Civil
Infrastructure Using Wireless Sensor Networks", In the
Proceedings of the 6th International Conference on
Information Processing in Sensor Networks (IPSN 2007),
Braun, et al. Compressed IPFIX [Page 23]
Internet-Draft Compressed IPFIX March 2010
Cambridge, MA, ACM Press, pp. 254-263 , April 2007.
[SMPC04] Szewczyk, R., Mainwaring, A., Polastre, J., and D. Culler,
"An analysis of a large scale habitat monitoring
application", The Proceedings of the Second ACM Conference
on Embedded Networked Sensor Systems (SenSys 04) ,
November 2004.
[GreatDuck]
Habitat Monitoring on Great Duck Island,
"http://www.greatduckisland.net", The Proceedings of the
Second ACM Conference on Embedded Networked Sensor Systems
(SenSys 04) , November 2004.
[Harvan08]
Harvan, M. and J. Schoenwaelder, "TinyOS Motes on the
Internet: IPv6 over 802.15.4 (6lowpan)", 2008.
[Crossbow]
Crossbow Technologies Inc., "http://www.xbow.com", 2010.
Authors' Addresses
Lothar Braun
Technische Universitaet Muenchen
Department of Informatics
Chair for Network Architectures and Services (I8)
Boltzmannstr. 3
Garching 85748
Germany
Email: braun@net.in.tum.de
URI: http://www.net.in.tum.de/~braun
Corinna Schmitt
Technische Universitaet Muenchen
Department of Informatics
Chair for Network Architectures and Services (I8)
Boltzmannstr. 3
Garching 85748
Germany
Email: schmitt@net.in.tum.de
URI: http://www.net.in.tum.de/~schmitt
Braun, et al. Compressed IPFIX [Page 24]
Internet-Draft Compressed IPFIX March 2010
Benoit Claise
Cisco Systems, Inc.
De Kleetlaan 6a b1
Diegem 1831
Belgium
Email: bclaise@cisco.com
Georg Carle
Technische Universitaet Muenchen
Department of Informatics
Chair for Network Architectures and Services (I8)
Boltzmannstr. 3
Garching 85748
Germany
Email: carle@net.in.tum.de
URI: http://www.net.in.tum.de/~carle
Braun, et al. Compressed IPFIX [Page 25]