IPFIX Working Group A. Kobayashi
Internet-Draft K. Ishibashi
Intended status: Informational T. Kondoh
Expires: April 26, 2007 NTT PF Lab.
D. Matsubara
Hitachi
October 23, 2006
IPFIX Mediators
draft-kobayashi-ipfix-mediator-01.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on April 26, 2007.
Copyright Notice
Copyright (C) The Internet Society (2006).
Kobayashi, et al. Expires April 26, 2007 [Page 1]
Internet-Draft IPFIX Mediators October 2006
Abstract
An IPFIX Mediator is an intermediate node between IPFIX Devices and
traffic Collectors. This node acts as a IPFIX proxy or IPFIX
concentrator. Therefore, IPFIX Mediators enables cascade connection.
This document defines the Option Template required by cascade
connection and a reference internal component model and describes
several solution scenarios that use IPFIX Mediators. IPFIX Mediator
has several capabilities such as storage function of original Flow
Records, flexible aggregation function, and proper distribution
function for collector or analyzer. The combination of these
capabilities can solve several problems related to traffic
monitoring.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Mediator Option Template Presentation . . . . . . . . . . . . 5
3.1. Route Option Templates . . . . . . . . . . . . . . . . . . 5
3.2. Consideration of Aggregation Pattern . . . . . . . . . . . 7
3.2.1. Aggregation beyond Several Routers . . . . . . . . . . 7
3.2.2. Aggregation beyond Several Observation Domain IDs . . 8
3.3. Usage of Scope Field . . . . . . . . . . . . . . . . . . . 9
4. Internal Components Model . . . . . . . . . . . . . . . . . . 11
4.1. Collecting Process . . . . . . . . . . . . . . . . . . . . 11
4.2. Selection Process . . . . . . . . . . . . . . . . . . . . 11
4.3. Aggregation Process . . . . . . . . . . . . . . . . . . . 12
4.4. Exporting Process . . . . . . . . . . . . . . . . . . . . 12
4.5. Storing Process . . . . . . . . . . . . . . . . . . . . . 13
5. Solution Scenarios with IPFIX Mediators . . . . . . . . . . . 14
5.1. Flexible Aggregation . . . . . . . . . . . . . . . . . . . 14
5.2. Distributed Aggregation . . . . . . . . . . . . . . . . . 14
5.3. Duplication of Flow Records . . . . . . . . . . . . . . . 15
5.4. Distribution of Flow Records . . . . . . . . . . . . . . . 16
5.5. Adding Information Elements . . . . . . . . . . . . . . . 17
5.6. Extraction of Suspicious Flow . . . . . . . . . . . . . . 18
6. Security Considerations . . . . . . . . . . . . . . . . . . . 20
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23
9.1. Normative References . . . . . . . . . . . . . . . . . . . 23
9.2. Informative References . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25
Intellectual Property and Copyright Statements . . . . . . . . . . 26
Kobayashi, et al. Expires April 26, 2007 [Page 2]
Internet-Draft IPFIX Mediators October 2006
1. Introduction
Several problems regarding traffic monitoring have occurred in each
network domain. As the sizes of networks become larger, the number
of Flow Records becomes greater. Large numbers of Flow Records have
been burdening management networks, and the Collecting Process.
Maintaining scalability is difficult as a given network grows. On
the other hand, networks such as IPv4, IPv6, and VPN on MPLS have
recently become more multifaceted. These sorts of Flow Records need
to be analyzed separately from a different perspective. However,
handling them separately without improving the capability of
Collector is difficult.
First, IPFIX protocol has an extensive Template format and flexible
functions. By defining IPFIX Mediators, we can increasingly take
advantage of an extensive Template format, and handle Flow Records in
accordance with our preference. This document describes some
solutions using IPFIX Mediator that solve the above problem and
components that are needed in the internal function process.
An IPFIX Mediator is located between one or more Exporting Processes
and one or more Collecting Processes. IPFIX Mediator has a main
function such as storing original Flow Records, aggregating them
flexibly just like IPFIX concentrator, and distributing them to an
appropriate Collector or traffic analyzer in accordance with flow
contents.
The internal model of a Mediator is composed of Collecting Processes,
Metering Processes, and Exporting Processes. IPFIX Mediator acts as
a Collector by receiving Flow Records, and it acts as an Exporter by
sending Flow Records. This dual-role architecture enables cascading
Mediators and buiilding a combination of several solutions.
This document proposes a solution by using IPFIX Mediator and a
reference model for IPFIX Mediators and describes the key component
of this architecture.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
Kobayashi, et al. Expires April 26, 2007 [Page 3]
Internet-Draft IPFIX Mediators October 2006
2. Terminology
The definitions of basic IPFIX and PSAMP terms are identical with
those in the [I-D.ietf-psamp-framework] and [RFC3917]. Other than
the above terminology, the following terminology is used in this
document.
IPFIX Mediator:
An IPFIX Mediator hosts at least one pair of Exporting Process and
Collecting Process. It may have Metering Process and Storing
Process as option. It may host multiple combinations of each
process. An IPFIX proxy and an IPFIX concentrator are one node of
IPFIX Mediators.
Selection Process:
An IPFIX Mediator's Selection Process is similar to that of PSAMP
Devices, which is described in [I-D.ietf-psamp-framework].
However, its Selection Process differs from the following
functions. This process in PSAMP Devices has two types of
selection functions: Filtering and Sampling. In addition, the
filtering function has two types of filtering methods: field-match
filtering and hash-based selection. This process in IPFIX
Mediator has only the field-match filtering function. This filter
selects Flow Records based on Flow Record content.
Aggregation Process:
This process creates aggregated Flow Records, in accordance with
aggregation rules that are described in
[I-D.dressler-ipfix-aggregation], from Flow Records that are the
output of the selection process.
Storing Process:
This process selects specified Information Elements by the storing
rules from the output of Collecting Process and stores these Flow
Records in a storage system such as a database or flat-file
system. Information Elements that are described in
[I-D.ietf-ipfix-info] are specified in storing rules. In
addition, the Observation Domain ID and Export Time in the header
of IPFIX messages are specified in the storing rules.
Distribute Function:
This function distributes final Flow Records based on the Flow
content. The final Flow Records are handled by Exporting Process
to export it to Collector. Each classified Flow Record is
exported to each Collector.
Kobayashi, et al. Expires April 26, 2007 [Page 4]
Internet-Draft IPFIX Mediators October 2006
3. Mediator Option Template Presentation
This section describes Option Templates that are used by IPFIX
Mediators.
3.1. Route Option Templates
The cascade connection of IPFIX Mediator enables several solutions
that are described in the section 5. Each IPFIX Mediator and final
destination Collector needs to know the source IPFIX Device of an
original Flow Record and route of aggregated Flow records. An
original Flow Record is created by a IPFIX Device that has
Observation Point. Therefore, each IPFIX Mediator should inform the
next Collector about previous Exporter information. The final
destination Collector can recognize the source IPFIX device, and
route of each Mediator. Route Option Template specifies the source
of the original Flow and the route of the Aggregated Flow. This
template is composed of the following information elements.
o templateId
o observationDomainId
o exporterIPv4Address or exporterIPv6Address
o exporterTransportPort
The IPFIX Router that creates original Flow Records is identified by
specifying "exporter{IPv4|IPv6}Address", "exporterTransportPort", and
"observationDomainId". On the other hands, the intermediate Mediator
is identified by specifying "exporter{IPv4|IPv6}Address" and
"exporterTransportPort". The "templateId" is used as a Scope field.
The route of received flow records is identified by an Exporter
information in the route. In that case, the order dependency that is
described in [I-D.ietf-ipfix-implementation-guidelines] is crucial.
The IPFIX Mediator inserts its own Exporter information into received
Data Records specified by the Route Option Template, and sends that
information to the next Collector. In this way, the route is
maintained until the final destination Collector.
Kobayashi, et al. Expires April 26, 2007 [Page 5]
Internet-Draft IPFIX Mediators October 2006
The following example describes the cascade connection of IPFIX
Mediators. Each Mediator informs the next node about previous
Exporter information.
Session#a Session#b Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3
Port:10 Port:20 Port:30
ODID:10
Figure A: Cascade connection of IPFIX Mediators.
The Mediator#1 or Mediator#2 sends a Data Record specified by the
Route Option Template. The Option templates and Data records are
shown in Session#b or Session#c, as follows.
Session#b OptionTemplate:
Set ID = 3
Template ID = 259
Field Count = 4
Scope Count = 1
templateId
exporterIPv4Address
exporterTransportPort
observationDomainId
Session#b Data Record:
Set ID = 259
templateId = XXX
exporterIPv4Address =1.1.1.1
exporterTransportPort = 10
observationDomainId = 10
Session#c OptionTemplate:
Set ID = 3
Template ID = 259
Field Count = 6
Scope Count = 1
templateId
exporterIPv4Address
exporterTransportPort
exporterIPv4Address
exporterTransportPort
observationDomainId
Kobayashi, et al. Expires April 26, 2007 [Page 6]
Internet-Draft IPFIX Mediators October 2006
Session#c Data Record:
Set ID = 259
templateId = XXX
exporterIPv4Address =2.2.2.2
exporterTransportPort = 20
exporterIPv4Address =1.1.1.1
exporterTransportPort = 10
observationDomainId = 10
3.2. Consideration of Aggregation Pattern
The Route Option Template should be used in consideration of the
aggregation pattern in Mediators. In two aggregation patterns, the
Route Option Template SHOULD be modified to handle properly.
3.2.1. Aggregation beyond Several Routers
If the number of source IPFIX devices in the largest set of Flow
information aggregated by a Metering Process becomes greater than
that of the source IPFIX devices, information about source IPFIX
devices is meaningless. In that case, this Mediator does not send a
Route Option Template, and next Mediator informs about received
Observation Domain ID. As shown in the Figure B, Mediator#1 does not
send Route Option Template and Mediator#2 exports the received
Observation Domain ID by the Route Option Template. From the
Collector viewpoint, the information seems to have come from the
source IPFIX Device, which is Mediator#1.
The following example describes cascade connection of IPFIX
Mediators. By the aggregation in Mediator#1, the largest set of Flow
information aggregated becomes whole routers that are Router#1 and
Router#2.
Session#a-1 Session#b Session#c
Router#1 -------+--> Mediator#1 --------> Mediator#2 -------->Collector
IP:1.1.1.1 | IP:2.2.2.2 IP:3.3.3.3
Port:10 | Port:20 Port:30
ODID:10 |
|
Session#a-2
Router#2 -------+
IP:4.4.4.4
Port:40
ODID:40
Figure B: Largest set of aggregated Flow information becomes
several routers.
Kobayashi, et al. Expires April 26, 2007 [Page 7]
Internet-Draft IPFIX Mediators October 2006
Note: In an actual aggregation pattern, this pattern does not seem to
be considered. The aggregation beyond several routers causes packet
properties derived from IPFIX Routers to become meaningless values.
The Mediator#1 does not send a Data Record of a specified Route
Option Template. The Mediator#2 sends a Data Record of a specified
Route Option Template. The Option templates and Data records are
shown in Session#c, as follows.
Session#c OptionTemplate:
Set ID = 3
Template ID = 259
Field Count = 4
Scope Count = 1
templateId
exporterIPv4Address
exporterTransportPort
observationDomainId
Session#c Data Record:
Set ID = 259
templateId = XXX
exporterIPv4Address =2.2.2.2
exporterTransportPort = 20
observationDomainId = 0
3.2.2. Aggregation beyond Several Observation Domain IDs
If the number of Observation Domains in the largest set of Flow
information aggregated by a Metering Process becomes greater than
that of the Observation Domains of source IPFIX device, information
about original Observation Domain ID of source IPFIX devices is
meaningless. In that case, the Mediator replaces the original
Observation Domain ID with 0.
The following example describes a cascade connection of IPFIX
Mediators. By the aggregation in Mediator#1, the largest set of
aggregated Flow information becomes Router#1, in other words, the
whole system.
Session#a Session#b Session#c
Router#1 --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3
Port:10 Port:20 Port:30
ODID:10
ODID:40
Figure C: Largest set of aggregated Flow information becomes
Kobayashi, et al. Expires April 26, 2007 [Page 8]
Internet-Draft IPFIX Mediators October 2006
whole system.
Router#1 exports Flow Records from two Observation Domains through
Session#a. Mediator#1 sends a Data Record of a specified Route
Option Template. The Option templates and Data records are shown in
Session#b, as follows.
Session#b OptionTemplate:
Set ID = 3
Template ID = 259
Field Count = 4
Scope Count = 1
templateId
exporterIPv4Address
exporterTransportPort
observationDomainId
Session#b Data Record:
Set ID = 259
templateId = XXX
exporterIPv4Address =1.1.1.1
exporterTransportPort = 10
observationDomainId = 0
3.3. Usage of Scope Field
An IPFIX Mediator needs to forward Option Template from source IPFIX
devices. However, it can not export an original Option Template
without modification, because changing a session from an Exporting
Process to a Collecting Process causes the scope fields to become a
meaningless value. In that case, an IPFIX Mediator can use the Route
Option Template. The Option Template that was created from a source
IPFIX device as a scope of an Observation Domain ID uses the entire
field of the Route Option template as the scope. The Option Template
that was created from a previous Mediator as a scope of the
Observation Domain ID uses some fields of the Route Option template
as a scope. These fields are given previous Mediator information
directly because the Mediator does not have Observation Domain, and
the Exporting Process of the Mediator uses the Observation Domain ID
with a value of 0. However, if each node uses another field except
for the Observation Domain ID as the scope, the scope field should be
considered on a case-by-case basis. If there is no Metering Process
in intermediate Mediators, the scope field with the template id can
be maintained untill the final destination Collector. If there is a
Metering process, the possibility of exporting this Option Template
depends on aggregation method.
Kobayashi, et al. Expires April 26, 2007 [Page 9]
Internet-Draft IPFIX Mediators October 2006
The following example describes the cascade connection of IPFIX
Mediators. Router#1 and Mediator#1 export the Metering Process
Statistics Option Template.
Session#a Session#b Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:1.1.1.1 IP:2.2.2.2 IP:3.3.3.3
Port:10 Port:20 Port:30
ODID:10
Figure D: Cascade connection of IPFIX Mediators.
Mediator#2 exports each Option Template and its Data Record with a
suitable scope.
Session#c Metering Process Statistics Data Records from Router:
Set ID = 3
Template ID = 259
Field Count = 8
Scope Count = 5
exporterIPv4Address = 2.2.2.2
exporterTransportPort = 20
exporterIPv4Address = 1.1.1.1
exporterTransportPort = 10
observationDomainId = 10
exportedMessageTotalCount
exportedFlowTotalCount
exportedOctetTotalCount
Session#c Metering Process Reliability Statistics Data Records from
Mediator#1:
Set ID = 3
Template ID = 259
Field Count = 5
Scope Count = 2
exporterIPv4Address = 2.2.2.2
exporterTransportPort = 20
exportedMessageTotalCount
exportedFlowTotalCount
exportedOctetTotalCount
Kobayashi, et al. Expires April 26, 2007 [Page 10]
Internet-Draft IPFIX Mediators October 2006
4. Internal Components Model
The following figure indicates five components (Collecting,
Selection, Aggregation, Exporting, and Storing) within the IPFIX
Mediator. These components are mapped to three processes as the
IPFIX architecture in the [RFC3917]. The Collecting Process matches
the IPFIX Collecting Process and the chain of Selection Process; the
Aggregation Process matches the IPFIX Metering Process. The
Exporting Process matches the IPFIX Exporting Process.
+----------------------------------------------------------+
| IPFIX Mediator |
| <----Metering Process-----> |
| .----------. .---------. .-----------. .---------. |
Flow -->|Collecting|->|Selection|->|Aggregation|->|Exporting|--> Flow
Records |Process | |Process | |Process | |Process | Records
| '----------' '---------' '-----------' '---------' |
| | |
| V |
| .----------. .---------. |
| |Storing |->|Database | |
| |Process | | | |
| '----------' '---------' |
| |
+----------------------------------------------------------+
Figure E: Key components within IPFIX Mediator.
Each process is associated with a common identifier in the IPFIX
Mediator. This method is similar to PSAMP associations in
[I-D.ietf-psamp-sample-tech].
4.1. Collecting Process
This process receives Flow Records from IPFIX devices or a previous
IPFIX Mediator. The instance of this process is created according to
the IPFIX session. This process has functions that are described in
[I-D.ietf-ipfix-protocol]. The Collecting Process also forwards
received Flow Records with IPFIX header information to multiple
Selection Processes or Storing Processes. In other words, Flow
Records can be duplicated by forwarding several Selecting Process.
4.2. Selection Process
The Selection Process receives Flow Records from Collecting
Processes. This process has a filtering function and selects Flow
Records that are matched under given conditions. Prior to receiving
Flow Records, this process has instruction pattern data that is
Kobayashi, et al. Expires April 26, 2007 [Page 11]
Internet-Draft IPFIX Mediators October 2006
defined by the user, which specifies how the Flow Records are treated
by this process. If some fields in the Flow Record match the
instruction pattern, this process selects Flow Records that have all
fields and forwards these to the Aggregation Process. For example,
this process selects Flow Records to prevent the next stage node from
being counted twice within the same flow.
4.3. Aggregation Process
This process gathers Flow Records within a given time interval and
then distinguishes Flow Records that have common properties. If
values of a given key field are the same, that means these Flow
Records have common properties. This process merges Flow Records
that have a common property and creates an aggregated Flow Record.
Therefore, for example, aggregated Flow Records have an aggregated
counter that indicates the number of packets. These functions are
defined in accordance with the IPFIX aggregation rule in
[I-D.dressler-ipfix-aggregation]. This process has instructions that
are user defined prior to receiving Flow Records. The process
indicates fields that should become aggregated flow keys and other
fields that should be kept or discarded. In addition, these
instruction rules include Information Elements that should be added
to aggregated Flow Records. The aggregated flow MAY need to
complement information that is discarded during the aggregation
process. For example, the aggregated flow has "averageActiveTime" or
"flowCount" elements.
4.4. Exporting Process
This process forwards Flow Records to the next IPFIX Mediator or
Collector. These processes manage the reporting template and make an
IPFIX datagram. The "Export time" and "Observation Domain Id" of
IPFIX header information can be either of the following methods. The
time of "Export time" field depends on whether the IPFIX Mediator is
a proxy or not. From the point of view of a Collector, if this IPFIX
Mediator is a proxy, it becomes the most recent time of the
originally received data. If it is not a proxy, "Export time"
becomes the export time of the Mediator. In the first case, this
process needs to obtain the originally received data from the
Aggregating Process. If IPFIX Mediator is not a proxy, its
"Observation domain ID" should be zero. These IPFIX Header
information values are forwarded from the Aggregation Process.
In addition, this process has the Distribution Function as option.
If this function is enabled, this process distributes Flow Records
based on Flow content, and then it exports each classified Flow
Records to each Collector.
Kobayashi, et al. Expires April 26, 2007 [Page 12]
Internet-Draft IPFIX Mediators October 2006
The Exporting Process distributes Flow Records based on Peering AS,
as shown in the following figure. Each classified Flow Record is
exported to a dedicated Collector per Peering AS.
+-------------------------------------------+
| IPFIX Mediator .----------. |
| |Exporting | |
| |Process | |
| .----------. .---------. | | | PeerAS#100
Flow -->|Collecting|->|Metering |----->/------------> Collector#A
Records |Process | |Process | | | | | PeerAS#200
| '----------' '---------' | /------------> Collector#B
| | | | | PeerAS#300
| | /------------> Collector#C
| | | | | PeerAS#400
| | /------------> Collector#D
| | | |
| '----------' |
+-------------------------------------------+
Figure F: Exporting each classified Flow Record to dedicated Collector.
4.5. Storing Process
The storing process selects specified Information Elements by the
storing instruction from the input Flow Records. Prior to receiving
Flow Records, this process has instructions that are configured by
the user, which indicate whether each field should be stored or
discarded. The field modifier indicates "keep" or "discard", which
is similar to the instruction of the aggregation process. The field
modifier specifies how these Information Elements are treated by the
process. This field modifier is applied to Information Elements
within Flow Record and header information of IPFIX messages. The
header information of IPFIX messages are "Observation Domain ID" and
"Export time". This header information MAY be used when IPFIX
datagrams are made of past Flow Records.
When another node retrieves past Flow Records, we can consider
several specifications. One solution is that another node gets a
specified flat-file from a Mediator, and decodes that flat-file by
itself. Other solutions are that another node sends out the query
command to the Mediator through the XML-RPC, SNMP, or NETCONF, and
then the Mediator exports the specified past Flow Records. This
function is out of the scope of this document.
Kobayashi, et al. Expires April 26, 2007 [Page 13]
Internet-Draft IPFIX Mediators October 2006
5. Solution Scenarios with IPFIX Mediators
5.1. Flexible Aggregation
An IPFIX Mediator can aggregate Flow Records as IPFIX concentrator
and reduce the number of Flow Records received by a traffic
collector.
The following figure indicates a cascade connection of IPFIX
Mediators. If a Collector measures a traffic matrix to obtain
traffic demand, it needs Flow Records of the whole network domain,
but does not need detailed Flow Records. In the first step, a
Mediator receives Flow Records from IPFIX Devices and then creates
aggregated low-level Flow Records. For example, this step is prefix
mask aggregation. Next, the Mediator receives aggregated Flow
Records and aggregates them further. For example, the second step is
the aggregation of the BGP next-hop address and exporter address.
After this, the collector receives high-level aggregated Flow Records
and then stores them. This method enables step-by-step aggregation
of Flow Records without overloading a single node.
.--------. .--------.
|IPFIX | |IPFIX |
|router#1|---->|Mediator|---.
| | |*1 | |
'--------' '--------' | .--------. .---------.
'--->|IPFIX | |Traffic |
.--------. .--------. .--->|Mediator|---->|Collector|
|IPFIX | |IPFIX | | |*2 | | |
|router#2|---->|Mediator|---' '--------' '---------'
| | |*1 |
'--------' '--------'
Figure G: Flexible Aggregation with cascading IPFIX Mediators.
5.2. Distributed Aggregation
As the network becomes widely global, the distances between PoPs
become longer, and the maintenance of a dedicated management network
is very expensive. Therefore, the huge number of Flow Records has
been burdening the management networks of global ISPs. If we place
Mediators at each PoP, the number of Flow Records exported from each
PoP can be reduced. Mediators can minimize the number of Flow
Records exported to the Collector. If the Collector needs detailed
information, it can retrieve Flow Records from Mediators that store
original Flow Records.
Kobayashi, et al. Expires April 26, 2007 [Page 14]
Internet-Draft IPFIX Mediators October 2006
A management networks of a global ISP is shown in the following
figure. The Mediators are located at each PoP of the network, and
they collect Flow Records from routers in each PoP domain. The
Mediator can reduces the number of Flow Records by aggregating or
filtering, so this system can reduces load of a management network.
POP#Asia
.--------.
.--------. | .---------.
.--------. | |----->|IPFIX |
|IPFIX | |------->|Mediator |----.
|router |---'----->|#1 | |
|#1 |-' '---------' |
'--------' |
|
POP#America |
.--------. |
.--------. | .---------. | .---------.
.--------. | |----->|IPFIX | '---->|Traffic |
|IPFIX | |------->|Mediator |--------->|Collector|
|router |---'----->|#2 | .---->| |
|#4 |-' '---------' | '---------'
'--------' |
|
POP#Europe |
.--------. |
.--------. | .---------. |
.--------. | |----->|IPFIX | |
|IPFIX | |------->|Mediator |----'
|router |---'----->|#3 |
|#7 |-' '---------'
'--------'
Figure H: Traffic monitoring architecture in global network.
5.3. Duplication of Flow Records
An IPFIX Mediator can duplicates Flow Records to achieve redundant
storage or utilizes them for several purposes. The pair of
Collecting Process and Metering Processes is similar to the pair of
the Observation Point and Metering Process. The Collecting process
duplicates Flow records by forwarding them to the multi-Metering
Process.
Kobayashi, et al. Expires April 26, 2007 [Page 15]
Internet-Draft IPFIX Mediators October 2006
Several departments in an ISP want to use the same traffic
information for each intended purpose. For example, the network
design department measures the traffic matrix as traffic demand, and
the customer service division uses traffic information for the
accounting information of each customer while the network operation
center uses traffic information for trouble shooting analysis. That
case is shown in the following figure. A Mediator can distribute
Flow Records to several Collectors that have appropriate aggregated
granularity. In addition, when NOC conducts trouble shooting, they
can retrieve past Flow Records from Mediators.
Measurement traffic matrix.
.--------. .---------.
|IPFIX | |Traffic |
|router#1|----. .---->|Collector|
| | | | |#1 |
'--------' | | '---------'
| | Using Accounting info.
.--------. | .---------. | .---------.
|IPFIX | '---->|IPFIX |----' |Traffic |
|router#2|--------->|Mediator |--------->|Collector|
| | .---->| |----. |#2 |
'--------' | '---------' | '---------'
| | Using Trouble shooting.
.--------. | | .---------.
|IPFIX | | | |Traffic |
|router#1|----' '---->|Collector|
| | |#3 |
'--------' '---------'
Figure I: Duplication of Flow Records for several purposes.
5.4. Distribution of Flow Records
An IPFIX Mediator can distribute Flow Records based on flow content.
This function enables load-balancing of Collector and sorting out
Flow Records without extra Collector functions. If the Flow Records
are used as accounting information, this solution is useful.
Kobayashi, et al. Expires April 26, 2007 [Page 16]
Internet-Draft IPFIX Mediators October 2006
When we disclose traffic information to each customer, security or
privacy policy should be considered. In that case, Mediator can hide
private information about each customer. For example, Mediator can
distribute them based on RD(Route Distinguisher), egress IF, Peering
AS number, or BGP next-hop which identifies the customer. In the
following figure, the Mediator distributes based on RD. It securely
allows for each customer to access only their own records.
.--------. .---------.
|IPFIX | |Traffic |
|router#1|----. .---->|Collector|<===> Customer#A
| | | | |#1 |
'--------' | | '---------'
| RD=100:1
| .---------. |
.--------. '---->|IPFIX |----' .---------.
|IPFIX | |Mediator | RD=100:2 |Traffic |
|router#2|--------->| |--------->|Collector|<===> Customer#B
| | | | |#2 |
'--------' .---->| |----. '---------'
| '---------' |
| RD=100:3
.--------. | | .---------.
|IPFIX | | | |Traffic |
|router#1|----' '---->|Collector|<===> Customer#C
| | |#3 |
'--------' '---------'
Figure J: Distribution of Flow Records for each customer.
5.5. Adding Information Elements
An IPFIX Mediator can add the following kinds of Information
Elements.
Complementing lost information by aggregation:
A Mediator can add complementary information to replace lost
information by aggregation. A main problem of aggregation is the
loss of some Information Elements. This problem could be
alleviated by adding complementary Information Elements. They are
added by the aggregation process and they help traffic collector
to analyze aggregated flow.
* averageActiveTime:
This Information Element indicates average time in milliseconds
of difference between flow start time and end time of each flow
included in the aggregated flow. It is created from flow time
stamp Information Elements. There are "flowStartSeconds",
Kobayashi, et al. Expires April 26, 2007 [Page 17]
Internet-Draft IPFIX Mediators October 2006
"flowEndSeconds", "flowStartMilliSeconds",
"flowEndMilliSeconds", "flowStartSysUpTime", and
"flowEndSysUpTime". In addition, "minimumActiveTime" and
"maxmumActiveTime" might be considered as well as this element.
* synCount:
It is the total number of Flow Records that has
"tcpControlBits" which SYN bit set to 1 in aggregated flow. By
this elements, we can grasp the number of SYN packets in whole
network. And also, "ackCount", "finCount", "pshCount",
"urgCount" and "rstCount" might be considered as well as this
element.
* flowCount:
It is the number of Flow Records that is included in the
aggregated flow.
Adding information instead of router:
The Information Elements that ca not be put in a original Flow
Record by a router are added to Flow Records by Mediator. For
example, those information elements are BGP next-hop, and RD. A
Mediator can add these elements by referring to BGP route
information and SNMP MIB. Hereby, traffic collectors do not need
to recognize the difference between implementations of routers
from several vendors.
Adding information that becomes easy to analyze:
These Information Elements enable nodes in the next stage to
become easy to analyze. For example, "Biflow ID" could be quoted.
"Biflow" is defined in [I-D.ietf-ipfix-biflow]. This indicates a
common ID that each UniFlow Record has in both directions. These
4-tuple elements that are in "Directional Key Field", have this
value as follows. There is a case by case consideration of which
fields will be included in the Hash function.
Biflow ID= Hash[src.IP+Dst.IP+Src.Port+Dst.Port]
Hereby, the next stage node can easily be used to gather UniFlow
Records. In an asymmetric routing condition, it is useful for the
accounting or analysis of anomalies in a session.
5.6. Extraction of Suspicious Flow
An IPFIX Mediator filters based on flow content. If filter
conditions are set depending on the suspicious flow as follows, the
next Collector receives the specified suspicious flow and detects an
anomalous flow by simply monitoring the traffic volume of each
suspicious flow.
Kobayashi, et al. Expires April 26, 2007 [Page 18]
Internet-Draft IPFIX Mediators October 2006
o TCP Flow Records whose "tcpControlBits" is set to "null"
o TCP Flow Records whose "tcpControlBits" is set to SYN bit only and
the packet counter is only 1.
o ICMP Flow Records whose length is too long.
Kobayashi, et al. Expires April 26, 2007 [Page 19]
Internet-Draft IPFIX Mediators October 2006
6. Security Considerations
The IPFIX concentrator uses the IPFIX protocol. Security
considerations about flow information are described in
[I-D.ietf-ipfix-protocol].
Kobayashi, et al. Expires April 26, 2007 [Page 20]
Internet-Draft IPFIX Mediators October 2006
7. IANA Considerations
This document has no actions for IANA.
Kobayashi, et al. Expires April 26, 2007 [Page 21]
Internet-Draft IPFIX Mediators October 2006
8. Acknowledgments
Many thanks to J. Quittek and B. Trammel for providing valuable
comments.
Kobayashi, et al. Expires April 26, 2007 [Page 22]
Internet-Draft IPFIX Mediators October 2006
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
9.2. Informative References
[I-D.dressler-ipfix-aggregation]
Dressler, F., Sommer, C., and G. Munz, "IPFIX
Aggregation", draft-dressler-ipfix-aggregation-03.txt
(work in progress) , June 2006.
[I-D.ietf-ipfix-biflow]
Trammell, B. and E. Boschi, "Bidirectional Flow Export
using IPFIX", draft-ietf-ipfix-biflow-00.txt(work in
progress) , August 2006.
[I-D.ietf-ipfix-implementation-guidelines]
Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P.
Aitken, "IPFIX Implementation Guidelines",
draft-ietf-ipfix-implementation-guideline-00.txt(work in
progress) , August 2006.
[I-D.ietf-ipfix-info]
Quittek, J., Bryant, S., Claise, B., and J. Meyer,
"Information Model for IP Flow Information Export",
draft-ietf-ipfix-info-13.txt(work in progress) ,
June 2006.
[I-D.ietf-ipfix-protocol]
Claise, B., "IPFIX Protocol Specification",
draft-ietf-ipfix-protocol-23 (work in progress) ,
October 2006.
[I-D.ietf-psamp-framework]
Duffield, N., "A Framework for Packet Selection and
Reporting", draft-ietf-psamp-framework-10.txt ,
January 2005.
[I-D.ietf-psamp-sample-tech]
Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", draft-ietf-psamp-sample-tech-07.txt ,
July 2005.
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
Kobayashi, et al. Expires April 26, 2007 [Page 23]
Internet-Draft IPFIX Mediators October 2006
"Requirements for IP Flow Information Export(IPFIX)",
October 2004.
Kobayashi, et al. Expires April 26, 2007 [Page 24]
Internet-Draft IPFIX Mediators October 2006
Authors' Addresses
Atsushi Kobayashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3978
Email: akoba@nttv6.net
Keisuke Ishibashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-3407
Email: ishibashi.keisuke@lab.ntt.co.jp
Kondoh Tsuyoshi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81-422-59-2419
Email: kondoh.tsuyoshi@lab.ntt.co.jp
Daisuke Matsubara
Hitachi, Ltd., Central Reseach Laboratory
1-280 Higashi-koigakubo
Kokubunji-shi, Tokyo 185-8601
Japan
Phone: +81-42-323-1111
Email: d-matuba@crl.hitachi.co.jp
Kobayashi, et al. Expires April 26, 2007 [Page 25]
Internet-Draft IPFIX Mediators October 2006
Full Copyright Statement
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Kobayashi, et al. Expires April 26, 2007 [Page 26]