NVO3 P. Jain
Internet-Draft K. Singh
Intended status: Standards Track F. Balus
Expires: August 16, 2014 Nuage Networks
W. Henderickx
Alcatel-Lucent
V. Bannai
PayPal
R. Shekhar
A. Lohiya
Juniper Networks
February 12, 2014
Generic Overlay OAM and Datapath Failure Detection
draft-jain-nvo3-overlay-oam-00
Abstract
This proposal describes a mechanism that can be used to detect Data
Path Failures of various overlay technologies as VXLAN, NVGRE,
MPLSoGRE and MPLSoUDP and verifying/sanity of their Control and Data
Plane for given Overlay Segment. This document defines the following
for each of the above Overlay Technologies:
o Encapsulation of OAM Packet, such that it has same Outer and
Overlay Header as any End-System's data going over the same
Overlay Segment.
o The mechanism to trace the Underlay that is exercised by any
Overlay Segment.
o Procedure to verify presence of any given Tenant VM or End-System
within a given Overlay Segment at Overlay End-Point.
Even though the present proposal addresses Overlay OAM for VXLAN,
NVGRE, MPLSoGRE and MPLSoUDP, but the procedures described are
generic enough to accommodate OAM for any other Overlay Technology.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Jain, et al. Expires August 16, 2014 [Page 1]
Internet-Draft Detecting Overlay Segment Failure February 2014
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 16, 2014.
Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Jain, et al. Expires August 16, 2014 [Page 2]
Internet-Draft Detecting Overlay Segment Failure February 2014
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 8
3. Motivation for Overlay OAM . . . . . . . . . . . . . . . . . . 9
4. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5. Packet Format . . . . . . . . . . . . . . . . . . . . . . . . 11
5.1. Overlay OAM Encapsulation in Layer 2 Context . . . . . . . 11
5.2. Overlay OAM Encapsulation in Layer 3 Context . . . . . . . 11
5.3. Generic Overlay OAM Packet Format . . . . . . . . . . . . 11
5.3.1. TLV Types for various Overlay Ping Models . . . . . . 14
5.3.1.1. TLV for VXLAN Ping . . . . . . . . . . . . . . . . 15
5.3.1.2. TLV for NVGRE Ping . . . . . . . . . . . . . . . . 16
5.3.1.3. TLV for MPLSoGRE Ping . . . . . . . . . . . . . . 17
5.3.1.4. TLV for MPLSoUDP Ping . . . . . . . . . . . . . . 18
6. Return Codes . . . . . . . . . . . . . . . . . . . . . . . . . 19
7. Procedure for Overlay Segment Ping . . . . . . . . . . . . . . 20
7.1. Encoding of Inner Header for Echo Request in Layer 2
Context . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2. Encoding of Inner Header for Echo Request in Layer 3
Context . . . . . . . . . . . . . . . . . . . . . . . . . 21
7.3. VXLAN Procedures . . . . . . . . . . . . . . . . . . . . . 21
7.3.1. Sending VXLAN Echo Request . . . . . . . . . . . . . . 21
7.3.2. Receiving VXLAN Echo Request . . . . . . . . . . . . . 22
7.3.3. Sending VXLAN Echo Reply . . . . . . . . . . . . . . . 23
7.3.4. Receiving VXLAN Echo Reply . . . . . . . . . . . . . . 23
7.4. NVGRE Procedures . . . . . . . . . . . . . . . . . . . . . 23
7.4.1. Sending NVGRE Echo Request . . . . . . . . . . . . . . 23
7.4.2. Receiving NVGRE Echo Request . . . . . . . . . . . . . 24
7.4.3. Sending NVGRE Echo Reply . . . . . . . . . . . . . . . 25
7.4.4. Receiving NVGRE Echo Reply . . . . . . . . . . . . . . 25
7.5. MPLSoGRE Procedures . . . . . . . . . . . . . . . . . . . 25
7.5.1. Sending MPLSoGRE Echo Request . . . . . . . . . . . . 25
7.5.2. Receiving MPLSoGRE Echo Request . . . . . . . . . . . 25
7.5.3. Sending MPLSoGRE Echo Reply . . . . . . . . . . . . . 26
7.5.4. Receiving MPLSoGRE Echo Reply . . . . . . . . . . . . 26
7.6. MPLSoUDP Procedures . . . . . . . . . . . . . . . . . . . 26
7.6.1. Sending MPLSoUDP Echo Request . . . . . . . . . . . . 26
7.6.2. Receiving MPLSoUDP Echo Request . . . . . . . . . . . 27
7.6.3. Sending MPLSoUDP Echo Reply . . . . . . . . . . . . . 28
7.6.4. Receiving MPLSoUDP Echo Reply . . . . . . . . . . . . 28
Jain, et al. Expires August 16, 2014 [Page 3]
Internet-Draft Detecting Overlay Segment Failure February 2014
8. Procedure for Trace . . . . . . . . . . . . . . . . . . . . . 29
9. Procedure for End-System Ping . . . . . . . . . . . . . . . . 30
9.1. Sub-TLV for End-System Ping . . . . . . . . . . . . . . . 30
9.1.1. Sub-TLV for Validating End-System MAC Address . . . . 31
9.1.2. Sub-TLV for Validating End-System IP Address . . . . . 32
9.1.3. Sub-TLV for Validating End-System MAC and IP
Address . . . . . . . . . . . . . . . . . . . . . . . 33
9.2. Sending End-System Ping Request . . . . . . . . . . . . . 34
9.3. Receiving End-System Ping Request . . . . . . . . . . . . 34
9.4. Sending End-System Ping Reply . . . . . . . . . . . . . . 35
9.5. Receiving End-System Ping Reply . . . . . . . . . . . . . 35
10. Security Considerations . . . . . . . . . . . . . . . . . . . 37
11. Management Considerations . . . . . . . . . . . . . . . . . . 38
12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 39
13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 40
14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 41
14.1. Normative References . . . . . . . . . . . . . . . . . . . 41
14.2. Informative References . . . . . . . . . . . . . . . . . . 42
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43
Jain, et al. Expires August 16, 2014 [Page 4]
Internet-Draft Detecting Overlay Segment Failure February 2014
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC2119 [RFC2119].
When used in lower case, these words convey their typical use in
common language, and are not to be interpreted as described in
RFC2119 [RFC2119].
Jain, et al. Expires August 16, 2014 [Page 5]
Internet-Draft Detecting Overlay Segment Failure February 2014
1. Introduction
VXLAN [I-D.draft-mahalingam-dutt-dcops-vxlan], NVGRE
[I-D.draft-sridharan-virtualization-nvgre], MPLSoGRE [RFC4023] and
MPLSoUDP [I-D.draft-ietf-mpls-in-udp] are well known technologies and
are used as tunneling mechanism to Overlay either Layer 2 networks or
Layer 3 networks on top of Layer 3 Underlay networks. For all above
Overlay Models there are two Tunnel End Points for a given Overlay
Segment. One End Point is where the Overlay Originates, and other
where Overlay Terminates. In most cases the Tunnel End Point is
intended to be at the edge of the network, typically connecting an
access switch to an IP transport network. The access switch could be
a physical or a virtual switch located within the hypervisor on the
server which is connected to End System which is a VM.
This document describes a mechanism that can be used to detect Data
Plane failures and sanity of Overlay Control and Data Plane for a
given Overlay Segment, and the method to trace the Underlay path that
is exercised by any given Overlay Segment.
The document also defines procedures for validating the presence of
any given Tenant VM/End-System/End-System or Flow representing the
End-System System within a given Overlay Segment.
The proposal describes:
o The mechanism to verify Overlay Control Plane and Data Plane
consistency at the Overlay End Point(s), by encapsulating the OAM
Packet in exact the same way as that of any End System Traffic
that is transported over the Overlay Segment.
o The mechanism to trace the Underlay that is exercised by any
Overlay Segment.
o The mechanism to verify presence of any "End-System" in a given
Overlay Segment.
The proposal defines the information to check correct operation of
the Data Plane, as well as a mechanism to verify the Data Plane
against the Control Plane for a given Overlay Segment.
It is important consideration in this proposal to carry Echo Request
along same Data Path that any End System's data using the given
Overlay Segment takes.
The tenants VM(s) or End System(s) are not aware of the Overlays and
as such the need for the verification of the Data Path MUST solely
rest with the Cloud Provider. The use cases where the Tenant VM(s)
Jain, et al. Expires August 16, 2014 [Page 6]
Internet-Draft Detecting Overlay Segment Failure February 2014
need to be aware of the Data Plane failures is beyond the scope of
this document.
Jain, et al. Expires August 16, 2014 [Page 7]
Internet-Draft Detecting Overlay Segment Failure February 2014
2. Terminology
Terminology used in this document:
OAM: Operations, Administration, and Management
VXLAN: Virtual eXtensible Local Area Network.
NVGRE: Network Virtualization using GRE.
MPLSoGRE: Encapsulating MPLS in IP or Generic Routing Encapsulation
(GRE)
MPLSoUDP: Encapsulating MPLS in UDP.
Originating End Point: Overlay Segment's Head End or Starting Point
of Overlay Tunnel.
Terminating End Point: Overlay Segment's Tail End or Terminating
Point of Overlay Tunnel.
VM: Virtual Machine.
VNI: VXLAN Network Identifier (or VXLAN Segment ID)
VSID: Virtual Subnet ID. (for NVGRE)
NVE: Network Virtualized Edge
End System: Could be Tenant VM, Host, Bridge etc. - System whose data
is expected to go over Overlay Segment.
Echo Request: Throughout this document, Echo Request packet is
expected to be transmitted by Originator Overlay End Point and
destined to Overlay Terminating End Point.
Echo Reply: Throughout this document, Echo Reply packet is expected
to be transmitted by Terminating Overlay End Point and destined to
Overlay Originating End Point.
Other terminologies are as defined in
[I-D.draft-mahalingam-dutt-dcops-vxlan],
[I-D.draft-sridharan-virtualization-nvgre], [RFC4023] and
[I-D.draft-ietf-mpls-in-udp]
Jain, et al. Expires August 16, 2014 [Page 8]
Internet-Draft Detecting Overlay Segment Failure February 2014
3. Motivation for Overlay OAM
When any Overlay Segment fails to deliver user traffic, there is a
need to provide a tool that would enable users, as Cloud Providers to
detect such failures, and a mechanism to isolate faults. It may also
be desirable to test the data path before mapping End System traffic
to the Overlay Segment.
The basic idea is to facilitate following verifications:-
o End-System's data that are expected to go over a particular
Overlay Segment actually ends up using the Data-Path represented
by given Overlay Segment between the two End-Points.
o To verify the correct value of Overlay Segment Identifier is
programmed at Originating and Terminating End Point(s) for a given
Overlay Segment. Segment Identifier will be VNI for VXLAN, VSID
for NVGRE, MPLS Label for MPLSoGRE and MPLSoUDP.
o The facilitate mechanism to trace the Underlay that is exercised
by any Overlay Segment.
o The mechanism to verify presence of any "End-System" in a given
Overlay Segment.
To facilitate verification of Overlay Segment or any End-System using
the Overlay, this document proposes sending of a Packet (called an
"Echo Request") along the same data path as other Packets belonging
to this Segment. Echo Request also carries information about the
Overlay Segment whose Data Path is to be verified. This Echo Request
is forwarded just like any other End System Data Packet belonging to
that Overlay Segment, as it contains the same Overlay Encapsulation
as regular End System's data.
On receiving Echo Request at the end of the Overlay Segment, it is
sent to the Control Plane of the Terminating Overlay End Point, which
in-turn would respond with Echo Reply.
To facilitate tracing of the Underlay used by any given Overlay
Segment, the document proposes Echo Request/Reply encapsulation in
"trace mode", which would allow the user or Cloud Provider to gather
information of the Underlay network.
Jain, et al. Expires August 16, 2014 [Page 9]
Internet-Draft Detecting Overlay Segment Failure February 2014
4. Approach
The proposal aims at validating Data Plane and its view of Control
Plane for a particular Overlay Segment. To achieve this aim, the
draft proposes creating an Overlay OAM Packet which MUST be
encapsulated with the Overlay Header as that of any End-Point data
going over the same Overlay Segment. This would guarantee the data-
path for OAM Packet follows the same path as that for any End User
data going over the same Overlay Segment.
The draft outlines procedures to encode Overlay Header and Inner
Ethernet or IP Header based on the type of payload that Overlay is
expected to carry.
Jain, et al. Expires August 16, 2014 [Page 10]
Internet-Draft Detecting Overlay Segment Failure February 2014
5. Packet Format
Generic Overlay Echo Request/Reply is a UDP Packet identified by well
known UDP Port XXXX. The payload carried by Overlay typically could
be either be Layer 2 / Ethernet Frame, or it could be Layer 3 / IP
Packet.
5.1. Overlay OAM Encapsulation in Layer 2 Context
If the encapsulated payload carried by Overlay is of type Ethernet,
then the OAM Echo Request packet would have inner Ethernet Header,
followed by IP and UDP Header. The payload of inner UDP would be as
described in below section "Generic Overlay OAM Packet Format".
5.2. Overlay OAM Encapsulation in Layer 3 Context
If the encapsulated payload carried by Overlay is of type IP, then
the OAM Echo Request packet would have inner IP Header, followed by
UDP Header. The payload of inner UDP would be as described in below
section "Generic Overlay OAM Packet Format".
5.3. Generic Overlay OAM Packet Format
Following is the format of UDP payload of Generic Overlay OAM Packet:
Jain, et al. Expires August 16, 2014 [Page 11]
Internet-Draft Detecting Overlay Segment Failure February 2014
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Message Type | Reply mode | Return Code | Return Subcode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Originator Handle |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TimeStamp Sent (seconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TimeStamp Sent (microseconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TimeStamp Received (seconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TimeStamp Received (microseconds) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TLVs ... |
. .
. .
. .
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Generic Overlay OAM Packet
The Message Type is one of the following:-
Value What it means
----- -------------
1 Echo Request
2 Echo Reply
Reply Mode Values:-
Value What it means
----- ---------------------------------
1 Do not reply
2 Reply via an IPv4/IPv6 UDP Packet
3 Reply via Overlay Segment
Echo Request with 1 (Do not reply) in the Reply Mode field may be
used for one-way connectivity tests. The receiving node may log gaps
in the Sequence Numbers and/or maintain delay/jitter statistics. For
normal operation Echo Request would have 2 (Reply via an IPv4 UDP
Jain, et al. Expires August 16, 2014 [Page 12]
Internet-Draft Detecting Overlay Segment Failure February 2014
Packet) in the Reply Mode field.
If it is desired that the reply also comes back via Overlay Segment
i.e. encapsulated with the Overlay Header, then the Reply Mode filed
needs to be set to 3 (Reply via Overlay Segment).
The Originator's Handle is filled in by the Originator, and returned
unchanged by the receiver in the Echo Reply (if any). The value used
for this field can be implementation dependent, this MAY be used by
the Originator for matching up requests with replies.
The Sequence Number is assigned by the Originator of Echo Request and
can be (for example) used to detect missed replies.
The TimeStamp Sent is the time-of-day (in seconds and microseconds,
according to the sender's clock) in NTP format [NTP] when the VXLAN
Echo Request is sent. The TimeStamp Received in an Echo Reply is the
time-of-day (according to the receiver's clock) in NTP format that
the corresponding Echo Request was received.
TLVs (Type-Length-Value tuples) have the following format:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Value |
. .
. .
. .
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sub-TLV Type | Length | Variable Length Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Variable Length Value |
| " |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Types are defined below; Length is the length of the Value field in
octets. The Value field depends on the Type; it is zero padded to
align to a 4-octet boundary. There could be one or many optional
Sub-TLV that could be encoded under the TLV.
Jain, et al. Expires August 16, 2014 [Page 13]
Internet-Draft Detecting Overlay Segment Failure February 2014
5.3.1. TLV Types for various Overlay Ping Models
TLV Types:-
Value What it means
----- ------------------------------
1 VXLAN Segment Ping for IPv4
2 VXLAN Segment Ping for IPv6
3 NVGRE Segment Ping for IPv4
4 NVGRE Segment Ping for IPv6
5 MPLSoGRE Segment Ping for IPv4
6 MPLSoGRE Segment Ping for IPv6
7 MPLSoUDP Segment Ping for IPv4
8 MPLSoUDP Segment Ping for IPv6
Jain, et al. Expires August 16, 2014 [Page 14]
Internet-Draft Detecting Overlay Segment Failure February 2014
5.3.1.1. TLV for VXLAN Ping
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 1(VXLAN ping IPv4)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VXLAN VNI | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Sender Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv4
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 2 (VXLAN ping IPv6)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VXLAN VNI | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPv6 Sender Address |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv6
Jain, et al. Expires August 16, 2014 [Page 15]
Internet-Draft Detecting Overlay Segment Failure February 2014
5.3.1.2. TLV for NVGRE Ping
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 3 (NVGRE ping IPv4)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NVGRE VSID | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Sender Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv4
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 4 (NVGRE ping IPv6)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NVGRE VSID | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPv6 Sender Address |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv6
Jain, et al. Expires August 16, 2014 [Page 16]
Internet-Draft Detecting Overlay Segment Failure February 2014
5.3.1.3. TLV for MPLSoGRE Ping
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 5 (MPLSoGRE ping IPv4)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
| (8 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Sender Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv4
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 6 (MPLSoGRE ping IPv6)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
| (8 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv6 Sender Address |
| |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv6
Route Distinguisher is defined as part of [RFC4365]
Jain, et al. Expires August 16, 2014 [Page 17]
Internet-Draft Detecting Overlay Segment Failure February 2014
5.3.1.4. TLV for MPLSoUDP Ping
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 7 (MPLSoUDP ping IPv4)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
| (8 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sender IPv4 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv4
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type = 8 (MPLSoUDP ping IPv6)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
| (8 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sender IPv6 Address |
| |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TLV if Sender Address is IPv6
Route Distinguisher is defined as part of [RFC4365]
Jain, et al. Expires August 16, 2014 [Page 18]
Internet-Draft Detecting Overlay Segment Failure February 2014
6. Return Codes
Sender MUST always set the Return Code set to zero. The receiver can
set it to one of the values listed below when replying back to Echo-
Request.
Following are the Return Codes (Suggested):-
Value What it means
----- -------------------------------
0 No return code
1 Malformed Echo Request Received
2 Overlay Segment Not Present
3 Overlay Segment Not Operational
4 Return-Code-OK
Jain, et al. Expires August 16, 2014 [Page 19]
Internet-Draft Detecting Overlay Segment Failure February 2014
7. Procedure for Overlay Segment Ping
Echo Request is used to test Data Plane and its view of Control Plane
for particular Overlay Segment. The Overlay Segment to be verified
is identified differently for various Overlay Technologies. For
VXLAN, VNI is used to identify given Overlay Segment. For NVGRE,
VSID is used. For MPLSoGRE and MPLSoUDP the MPLS Stack is used to
identify a given Overlay Segment.
For the Data Plane verification, the Overlay Echo Request Packet MUST
be encapsulated within the Overlay Header, which is same as that of
any End-Point data going over the same Overlay Segment. This would
guarantee the data-path for OAM Packet follows the same path as that
for any End User data going over the same Overlay Segment.
The payload carried by Overlay typically could be either be Layer 2
or Ethernet Frame, or it could be Layer 3 or IP Packet. Based on the
type of payload following is the way inner Header(s) of Echo Request
would be encoded.
7.1. Encoding of Inner Header for Echo Request in Layer 2 Context
If the encapsulated payload carried by Overlay is of type Ethernet,
then the OAM Echo Request packet would have inner Ethernet Header,
followed by IP and UDP Header. The payload of inner UDP would be as
described in below section "Generic Overlay OAM Packet Format".
Inner Ethernet Header for the Echo Request Packet MUST have the
Destination Mac set to 00-00-5E-90-XX-XX (to be assigned IANA). The
Source Mac should be set to Mac Address of the Originating VTEP.
However, it is desired that the Inner Source Mac SHOULD not be learnt
in the MAC-Table as this represent Control Packet in context of
Overlay OAM.
Inner IP header is set with the Source IP Address which is a routable
Address of the sender; the Destination IP Address is a (randomly
chosen) IPv4 Address from the range 127/8, IPv6 addresses are chosen
from the range 0:0:0:0:0:FFFF:127/104. The IP TTL is set to 255.
The inner Destination UDP port is set to xxxx (assigned by IANA for
Overlay OAM).
The "Generic Overlay OAM Packet" will now be encoded, with following
information.
The sender chooses a Originator's Handle and a Sequence Number. When
sending subsequent Overlay Echo Requests, the sender SHOULD increment
the Sequence Number by 1.
Jain, et al. Expires August 16, 2014 [Page 20]
Internet-Draft Detecting Overlay Segment Failure February 2014
The TimeStamp Sent is set to the time-of-day (in seconds and
microseconds) that the Echo Request is sent. The TimeStamp Received
is set to zero. Also, the Reply Mode must be set to the desired
reply mode. The Return Code and Subcode are set to zero.
Next, the TLV is Encoded for desired Overlay Type, as per Section
"Types of TLVs defined for various Overlay Ping Models"
7.2. Encoding of Inner Header for Echo Request in Layer 3 Context
If the encapsulated payload carried by Overlay is of type IP, then
the Encoding of the Echo Request would be same as above Section
"Encoding of Inner Header for Echo Request in Layer 2 Context", but
without the presence of Inner Ethernet Header.
7.3. VXLAN Procedures
7.3.1. Sending VXLAN Echo Request
The Outer VxLAN header for the Echo Request packet follows the
encapsulation as defined in [I-D.draft-mahalingam-dutt-dcops-vxlan].
The VNI is same as that of the VXLAN Segment that is being verified.
This would make sure that OAM Packet takes the same datapath as any
other End System data going over this VXLAN Segment.
The VXLAN Router Alert option
[I-D.draft-singh-nvo3-vxlan-router-alert] MUST be set in the VXLAN
header as shown below.
VXLAN Header:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|R|R|R|I|R|R|RA| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VXLAN Network Identifier (VNI) | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
RA: Router Alter Bit (Proposed)
Originating VTEP MAY set the I Bit to 1 in VXLAN Header when sending
OAM Frame. This would cause dropping of such VXLAN frames on any
Terminating VTEP that does not understand Overlay OAM framework, and
prevent sending those frames to End-Systems or VMs.
It is desired to choose the Source UDP port (in the outer header), so
as to exercise the same Data-Path as that of the traffic carried over
the VXLAN Segment and is left to the implementation.
Jain, et al. Expires August 16, 2014 [Page 21]
Internet-Draft Detecting Overlay Segment Failure February 2014
The Encoding of Inner Header(s) and UDP payload of Generic Overlay
OAM Packet is as described in above Sub-Section i.e. "Encoding of
Inner Header for Echo Request in Layer 2/Layer 3 Context".
7.3.2. Receiving VXLAN Echo Request
At the Terminating Overlay End Point or VTEP, since the Overlay OAM
Packet is exactly same as that of End-System Packet(s). It is
important to send OAM packet to Control Plane and prevent it from
sending to the End System. The trapping and sending VXLAN Echo
Request to the Control Plane is triggered by one of the following
Packet processing exceptions: VXLAN Router Alert option,
[I-D.draft-singh-nvo3-vxlan-router-alert] the Inner Destination MAC
Address of 00-00-5E-90-XX-XX as defined in above section, and the
Destination IP Address in the 127/8 Address range for IPv4 Address,
or 0:0:0:0:0:FFFF:127/104 for IPv6 Address.
The Control Plane further identifies the Overlay OAM Application by
UDP well know destination port xxxx.
Since the VxLAN Router Alert bit is set in VxLAN Header, which
signifies the presence of Control Packet. The terminating VTEP
SHOULD not learn the Mac address set in the Inner Mac Header of VxLAN
Echo Request Packet.
Once the VXLAN Echo Request Packet is identified at Control Plane, it
is processed as follows:-
o General Packet sanity is verified. If the Packet is not well-
formed, VTEP SHOULD send VXLAN Echo Reply with the Return Code set
to "Malformed Echo Request received" and the Subcode to zero. The
header fields Originator's Handle, Sequence Number, and Timestamp
Sent are not examined, but are included in the VXLAN Echo Reply
message
o VNI Validation: If there is no entry for VNI, it indicates that
there could be a transient or permanent disconnect between Control
Plane and data Plane and VTEP needs to report an error with Return
Code of "Overlay Segment Not Present" and a Return Subcode of
Zero. If the mapping for VNI Exists, but the state is not
Operational, VTEP needs to report an error with Return Code of
"Overlay Segment Not Operational" If the mapping exists then send
VXLAN Echo Reply with a Return Code of "Return-Code-OK", and a
Return Subcode of Zero. The procedures for sending the Echo Reply
are found in subsection below section.
Jain, et al. Expires August 16, 2014 [Page 22]
Internet-Draft Detecting Overlay Segment Failure February 2014
7.3.3. Sending VXLAN Echo Reply
If the Reply Mode is set to "Reply via an IPv4/IPv6 UDP Packet", the
Echo Reply is a UDP Packet. It MUST ONLY be sent in response to Echo
Request. The Source IP Address in the Header should be Routable
Address of the replier; The Destination IP Address should be IP
Address of the Echo Request's Originating End Point or the requester.
The destination UDP Port is set to XXXX (assigned by IANA for
identifying VXLAN OAM application). The IP TTL is set to 255.
The format of the Echo Reply is the same as the Echo Request. The
Originator Handle, the Sequence Number, and TimeStamp Sent are copied
from the Echo Request; the TimeStamp Received is set to the time-of-
day that the Echo Request is received (note that this information is
most useful if the time-of-day clocks on the requester and the
replier are synchronized). The replier MUST fill in the Return Code
and Subcode, as determined in the previous subsection.
If the Reply Mode is set to "Reply via Overlay Segment", then the
Replying Overlay End Point is expected to place Echo Reply packet in-
band in the Overlay Segment destined to the Originating Overlay End
Point. The detailed encapsulation for this would be covered in next
revision of the draft.
7.3.4. Receiving VXLAN Echo Reply
An Originating Overlay End Point should only receive Echo Reply in
response to an Echo Request that it sent. When the Reply Mode is
"Reply via an IPv4/IPv6 UDP Packet", the Echo Reply would be and IP
Packet/UDP Packet, and is identified by the destination UDP Port
XXXX. The Originating Overlay End Point should parse the Packet to
ensure that it is well-formed, then attempt to match up the Echo
Reply with an Echo Request that it had previously sent, and the
Originator Handle. If no match is found, then it should drop the
Echo Reply Packet; otherwise, it checks the Sequence Number to see if
it matches.
7.4. NVGRE Procedures
7.4.1. Sending NVGRE Echo Request
The Outer NVGRE header for the Echo Request packet follows the
encapsulation as defined in
[I-D.draft-sridharan-virtualization-nvgre]. The VSID is same as that
of the NVGRE Segment that is being verified. This would make sure
that OAM Packet takes the same datapath as any other End System data
going over this NVGRE Segment.
Jain, et al. Expires August 16, 2014 [Page 23]
Internet-Draft Detecting Overlay Segment Failure February 2014
The NVGRE Router Alert option
[I-D.draft-singh-nvo3-nvgre-router-alert] MUST be set in the NVGRE
header as shown below.
GRE Header:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| |1|0| Reserved0 RA| Ver | Protocol Type 0x6558 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Virtual Subnet ID (VSID) | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
RA: Router Alter Bit (Proposed)
The Encoding of Inner Header(s) and UDP payload of Generic Overlay
OAM Packet is as described in above Sub-Section i.e. "Encoding of
Inner Header for Echo Request in Layer 2/Layer 3 Context".
7.4.2. Receiving NVGRE Echo Request
At the Terminating Overlay End Point, since the Overlay OAM Packet is
exactly same as that of End-System Packet(s). It is important to
send OAM packet to Control Plane and prevent it from sending to the
End System. The trapping and sending NVGRE Echo Request to the
Control Plane is triggered by one of the following Packet processing
exceptions: NVGRE Router Alert option,
[I-D.draft-singh-nvo3-nvgre-router-alert] the Inner Destination MAC
Address of 00-00-5E-90-XX-XX as defined in above section, and the
Destination IP Address in the 127/8 Address range for IPv4 Address,
or 0:0:0:0:0:FFFF:127/104 for IPv6 Address.
The Control Plane further identifies the Overlay OAM Application by
UDP well know destination port xxxx.
Since the NVGRE Router Alert bit is set in NVGRE Header, which
signifies the presence of Control Packet. The Terminating Overlay
End Point SHOULD not learn the Mac address set in the Inner Mac
Header of NVGRE Echo Request Packet.
Once the NVGRE Echo Request Packet is identified at Control Plane, it
is processed as follows:-
o General Packet sanity is verified. If the Packet is not well-
formed, NVGRE End Point SHOULD send NVGRE Echo Reply with the
Return Code set to "Malformed Echo Request received" and the
Subcode to zero. The header fields Originator's Handle, Sequence
Number, and Timestamp Sent are not examined, but are included in
the NVGRE Echo Reply message
Jain, et al. Expires August 16, 2014 [Page 24]
Internet-Draft Detecting Overlay Segment Failure February 2014
o VSID Validation: If there is no entry for VSID, it indicates that
there could be a transient or permanent disconnect between Control
Plane and data Plane and NVGRE End Point needs to report an error
with Return Code of "Overlay Segment Not Present" and a Return
Subcode of Zero. If the mapping for VSID Exists, but the state is
not Operational, NVGRE End Point needs to report an error with
Return Code of "Overlay Segment Not Operational" If the mapping
exists then send NVGRE Echo Reply with a Return Code of "Return-
Code-OK", and a Return Subcode of Zero. The procedures for
sending the Echo Reply are found in subsection below section.
7.4.3. Sending NVGRE Echo Reply
The procedure for sending NVGRE Echo Reply are exactly same as
defined in above section "Sending VXLAN Echo Reply".
7.4.4. Receiving NVGRE Echo Reply
The procedure for Receiving NVGRE Echo Reply are exactly same as
defined in above section "Receiving VXLAN Echo Reply".
7.5. MPLSoGRE Procedures
7.5.1. Sending MPLSoGRE Echo Request
The Outer header of MPLSoGRE for the Echo Request packet follows the
encapsulation as defined in [RFC4023]. The MPLS Stack is same as
that of the MPLSoGRE Segment that is being verified. This would make
sure that OAM Packet takes the same datapath as any other End System
data going over this MPLSoGRE Segment.
However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert
Label [RFC3032]. This would indicate the Overlay Terminating End
Point that the payload is a Control Packet and needs to be delivered
to Control Plane.
The Encoding of Inner Header(s) and UDP payload of Generic Overlay
OAM Packet is as described in above Sub-Section i.e. "Encoding of
Inner Header for Echo Request in Layer 2/Layer 3 Context".
7.5.2. Receiving MPLSoGRE Echo Request
At the Terminating Overlay End Point, since the Overlay OAM Packet is
exactly same as that of End-System Packet(s). It is important to
send OAM packet to Control Plane and prevent it from sending to the
End System. The trapping and sending MPLSoGRE Echo Request to the
Control Plane is triggered by one of the following Packet processing
exceptions: MPLS Router Alert Label, and the Destination IP Address
Jain, et al. Expires August 16, 2014 [Page 25]
Internet-Draft Detecting Overlay Segment Failure February 2014
in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/
104 for IPv6 Address.
The Control Plane further identifies the Overlay OAM Application by
UDP well know destination port xxxx.
Once the MPLSoGRE Echo Request Packet is identified at Control Plane,
it is processed as follows:-
o General Packet sanity is verified. If the Packet is not well-
formed, MPLSoGRE End Point SHOULD send MPLSoGRE Echo Reply with
the Return Code set to "Malformed Echo Request received" and the
Subcode to zero. The header fields Originator's Handle, Sequence
Number, and Timestamp Sent are not examined, but are included in
the MPLSoGRE Echo Reply message
o Segment Validation: If there is no entry for service represented
by given Route Distinguisher for the MPLSoGRE Segment, it
indicates that there could be a transient or permanent disconnect
between Control Plane and Data Plane and MPLSoGRE End Point needs
to report an error with Return Code of "Overlay Segment Not
Present" and a Return Subcode of Zero. If the entry for service
represented by given Route Distinguisher for the MPLSoGRE Segment
is present, but is Operationally Down. The End Point needs to
report an error with Return Code of "Overlay Segment Not
Operational" If the mapping of service represented by given Route
Distinguisher for the MPLSoGRE Segment is present and Active, then
send MPLSoGRE Echo Reply with a Return Code of "Return-Code-OK".
7.5.3. Sending MPLSoGRE Echo Reply
The procedure for sending MPLSoGRE Echo Reply are exactly same as
defined in above section "Sending VXLAN Echo Reply".
7.5.4. Receiving MPLSoGRE Echo Reply
The procedure for Receiving MPLSoGRE Echo Reply are exactly same as
defined in above section "Receiving VXLAN Echo Reply".
7.6. MPLSoUDP Procedures
7.6.1. Sending MPLSoUDP Echo Request
The Outer header of MPLSoUDP for the Echo Request packet follows the
encapsulation as defined in [I-D.draft-ietf-mpls-in-udp]. The MPLS
Stack is same as that of the MPLSoUDP Segment that is being verified.
This would make sure that OAM Packet takes the same datapath as any
other End System data going over this MPLSoUDP Segment.
Jain, et al. Expires August 16, 2014 [Page 26]
Internet-Draft Detecting Overlay Segment Failure February 2014
However, the bottommost Label in MPLS Stack MUST be MPLS Router Alert
Label [RFC3032]. This would indicate the Overlay Terminating End
Point that the payload is a Control Packet and needs to be delivered
to Control Plane.
It is desired to choose the Source UDP port (in the outer header), so
as to exercise the same Data-Path as that of the traffic carried over
the MPLSoUDP Segment and is left to the implementation.
The Encoding of Inner Header(s) and UDP payload of Generic Overlay
OAM Packet is as described in above Sub-Section i.e. "Encoding of
Inner Header for Echo Request in Layer 2/Layer 3 Context".
7.6.2. Receiving MPLSoUDP Echo Request
At the Terminating Overlay End Point, since the Overlay OAM Packet is
exactly same as that of End-System Packet(s). It is important to
send OAM packet to Control Plane and prevent it from sending to the
End System. The trapping and sending MPLSoGRE Echo Request to the
Control Plane is triggered by one of the following Packet processing
exceptions: MPLS Router Alert Label, and the Destination IP Address
in the 127/8 Address range for IPv4 Address, or 0:0:0:0:0:FFFF:127/
104 for IPv6 Address.
The Control Plane further identifies the Overlay OAM Application by
UDP well know destination port xxxx.
Once the MPLSoUDP Echo Request Packet is identified at Control Plane,
it is processed as follows:-
o General Packet sanity is verified. If the Packet is not well-
formed, MPLSoUDP End Point SHOULD send MPLSoUDP Echo Reply with
the Return Code set to "Malformed Echo Request received" and the
Subcode to zero. The header fields Originator's Handle, Sequence
Number, and Timestamp Sent are not examined, but are included in
the MPLSoUDP Echo Reply message
o Segment Validation: If there is no entry for service represented
by given Route Distinguisher for the MPLSoUDP Segment, it
indicates that there could be a transient or permanent disconnect
between Control Plane and data Plane and MPLSoUDP End Point needs
to report an error with Return Code of "Overlay Segment Not
Present" and a Return Subcode of Zero. If the entry for service
represented by given Route Distinguisher for the MPLSoUDP Segment
is present, but is Operationally Down. The End Point needs to
report an error with Return Code of "Overlay Segment Not
Operational" If the mapping of service represented by given Route
Distinguisher for the MPLSoUDP Segment is present and Active, then
Jain, et al. Expires August 16, 2014 [Page 27]
Internet-Draft Detecting Overlay Segment Failure February 2014
send MPLSoUDP Echo Reply with a Return Code of "Return-Code-OK".
7.6.3. Sending MPLSoUDP Echo Reply
The procedure for sending MPLSoGRE Echo Reply are exactly same as
defined in above section "Sending VXLAN Echo Reply".
7.6.4. Receiving MPLSoUDP Echo Reply
The procedure for Receiving MPLSoGRE Echo Reply are exactly same as
defined in above section "Receiving VXLAN Echo Reply".
Jain, et al. Expires August 16, 2014 [Page 28]
Internet-Draft Detecting Overlay Segment Failure February 2014
8. Procedure for Trace
In order to be able to trace the Path that a particular flow in the
Overlay takes through the Underlay Network, following mechanism can
be used - An overlay Echo Request packet is built and sent using the
mechanisms described in the Section "Procedure for Overlay Segment
Ping" so that the overlay traceroute follows the same path as the
data packet for the overlay segment being traced.
The Echo Request packet in the traceroute mode is sent with the
initial TTL set to 1 in the Outer IP header and thereafter
incremented by 1 in each successive request. At each transit hop
where the TTL expires, an exception is created. Because of this
exception, the packet gets delivered to the Control Plane. Control
plane can further deliver the packet to the OAM application based on
the TTL exception and the specific UDP port XXXX in the incoming
overlay echo request packet. If the transit node has the IP
reachability to the destination IP address in the outer IP header, it
sends back an overlay echo reply response otherwise the Overlay Echo
Request is discarded by the Overlay OAM module on the transit nodes.
If the transit node does not support overlay OAM functionality, it
will simply generate a regular ICMP TTL exceeded response. This
could result into "false negatives". The originating Overlay node
that generated the OAM echo request SHOULD try sending the echo
request with TTL=n+1, n+2, ... to probe the nodes further down the
path to the terminating overlay End-point.
At the originating node, when the Echo Reply from the transit node
corresponding to the traceroute query is received, it can correlate
the incoming Echo Reply with the traceroute query by matching on the
sequence numbers in the Overlay Echo Request/Reply packets.
Current revision of this draft limits overlay traceroute capability
to fault isolation only. A subsequent version of the draft will
include mechanisms to trace all possible paths in the underlay that
can be used to carry overlay tunnel traffic.
Jain, et al. Expires August 16, 2014 [Page 29]
Internet-Draft Detecting Overlay Segment Failure February 2014
9. Procedure for End-System Ping
In typical Overlay deployment scenarios there is a desired to check
the presence of any given Tenant VM/End-System or Flow representing
the End-System System within a given Overlay Segment. This draft
proposes the way to achieve it via End-System Ping.
The End-System can be identified at Overlay End Point by either its
IP Address, Ethernet MAC Address or combination of IP/MAC Address.
In that case, it would be important to verify the End-System
connectivity by procedure which goes over the Overlay Segment from
Originating Overlay End-Point and verifies the presence of the End-
System at the Terminating Overlay End-Point.
The scope of End-System Ping is solely with the Cloud Provider which
owns control of the Overlay End Point(s). It is expected that the
Overlay End Point traps this request and checks the Presence of the
End-System via its MAC Address, Route or Flow information and replies
back. There SHOULD not be a case where the End-System Ping is
delivered to the actual End-Point.
9.1. Sub-TLV for End-System Ping
Sub-TLV Types:-
Value What it means
----- ---------------------------
1 End-System MAC Sub-TLV
2 End-System IPv4 Sub-TLV
3 End-System IPv6 Sub-TLV
4 End-System MAC/IPv4 Sub-TLV
6 End-System MAC/IPv6 Sub-TLV
End-System Return Code:-
Value What it means
----- ----------------------
1 End-System Present
2 End-System Not Present
Jain, et al. Expires August 16, 2014 [Page 30]
Internet-Draft Detecting Overlay Segment Failure February 2014
9.1.1. Sub-TLV for Validating End-System MAC Address
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| End-System MAC Sub-TLV (1) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 | Return Code#1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #2 | Return Code#2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n | Return Code #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
MAC Address: MAC Address of the End-System, that user is interested
to validate.
Return Code: Return Code specifying status of End-System at Overlay End Point
Jain, et al. Expires August 16, 2014 [Page 31]
Internet-Draft Detecting Overlay Segment Failure February 2014
9.1.2. Sub-TLV for Validating End-System IP Address
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| End-System IPv4 Sub-TLV (2) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Return Code #1 | IP address #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #2 | Return Code #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Return Code #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| End-System IPv6 Sub-TLV (3) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPv6 Address #1 |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Return Code #1 | IPv6 Address #2... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPv6 Address #n |
| |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Return Code #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IP Address : IP Address of the End-System, that user is interested to
validate.
Return Code: Return Code specifying status of End-System at Overlay End Point
Jain, et al. Expires August 16, 2014 [Page 32]
Internet-Draft Detecting Overlay Segment Failure February 2014
9.1.3. Sub-TLV for Validating End-System MAC and IP Address
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| End-System IPv4/MAC Sub-TLV (4)| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 | IP address #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #1 | Return Code #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #2 | IP address #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #2 | Return Code #2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n | IP address #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP address #n | Return Code #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| End-System IPv6/MAC Sub-TLV(5) | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #1 | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
| |
+ +
| IPv6 address #1 |
+ +
| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | Return Code #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ... ~
Jain, et al. Expires August 16, 2014 [Page 33]
Internet-Draft Detecting Overlay Segment Failure February 2014
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MAC address #n | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
| |
+
| IPv6 address #1 |
+ +
| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | Return Code #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IP Address : IP Address of the End-System, that user is interested to
validate.
MAC Address: MAC Address of the End-System, that user is interested to
validate.
Return Code: Return Code specifying status of End-System at Overlay End Point
9.2. Sending End-System Ping Request
When it is desired to check presence of a given End-System, the Echo
Request Message is prepared as described in above Section "Procedure
for Overlay Segment Ping". This packet should compose of Outer
Header, Overlay Header, Inner Header, Generic Overlay Header with TLV
representing desired Overlay Type (VXLAN, NVGRE, MPLSoGRE or
MPLSoUDP). Apart form this the packet should also have one of the
Sub-TLV's as defined in above section "Sub-TLV for End-System Ping"
to identify the type of End-System Ping that user is interested in.
Because of the above mentioned encapsulation, it would be guaranteed
that the packet follows the same Data Path as that of any End-User
data going over the given Overlay Segment.
User need to fill in MAC, IP or MAC/IP combination for the End-
System(s) that needs to be validated at the Overlay End Point in the
respective Sub-TLV for End-System Ping.
9.3. Receiving End-System Ping Request
On receiving the End-System Ping Request the processing to trap this
Packet, and sent it to Control Plane is done by Overlay Terminating
End-System as define in above Section "Procedure for Overlay Segment
Ping". Once the OAM Packet reaches OAM Application, it is identified
as End-System Ping Request by virtue of presence any of the Sub-TLV's
as defined in Section "Sub-TLV for End-System Ping".
Jain, et al. Expires August 16, 2014 [Page 34]
Internet-Draft Detecting Overlay Segment Failure February 2014
If the Sub-TLV is of Type "End-System MAC Sub-TLV", the Overlay End
Point should iterate through the list of MAC Addresses and verify the
presence of individual MAC Address in its Flow Table or MAC Table for
the given Overlay Segment.
If the MAC Address is present, it should set the respective End-
System's Return Code field in the Sub-TLV to 1 "End-System-Present".
If the MAC Address is not present, it should set respective the End-
System's Return Code filed in the Sub-TLV to 2 "End-System-Not-
Present".
If the Sub-TLV is of Type "End-System IP Sub-TLV", the Overlay End
Point should iterate through the list of IP Addresses and verify the
presence of individual IP Address in its Flow Table or Route Table
for the given Overlay Segment.
If the IP Address is present, it should set the respective End-
System's Return Code field in the Sub-TLV to 1 "End-System-Present".
If the IPC Address is not present, it should set respective the End-
System's Return Code filed in the Sub-TLV to 2 "End-System-Not-
Present".
If the Sub-TLV is of Type "End-System MAC and IP Sub-TLV", the
Overlay End Point should iterate through the list of MAC/IP Addresses
and verify the presence of individual MAC/IP Combination in its Flow
Table or MAC and IP Table for the given Overlay Segment.
If the IP and MAC Address is present, it should set the respective
End-System's Return Code field in the Sub-TLV to 1 "End-System-
Present".
If the IP and MAC Address is not present, it should set respective
the End-System's Return Code filed in the Sub-TLV to 2 "End-System-
Not-Present".
9.4. Sending End-System Ping Reply
The procedure for sending End-System Echo Reply is same as defined in
above section "Sending VXLAN Echo Reply". The replier MUST fill Sub-
TLV with proper Return Code for each element in the End-System Sub-
TLV.
9.5. Receiving End-System Ping Reply
An Originating Overlay End Point should only receive Echo Reply for
End-System Ping, in response to an Echo Request that it sent. By
Jain, et al. Expires August 16, 2014 [Page 35]
Internet-Draft Detecting Overlay Segment Failure February 2014
virtue of presence of End-System Sub-TLV it would identify the status
of respective End-System, and report it to the user. The other part
of the handling is similar to section "Receiving VXLAN Echo Reply"
Jain, et al. Expires August 16, 2014 [Page 36]
Internet-Draft Detecting Overlay Segment Failure February 2014
10. Security Considerations
TBD
Jain, et al. Expires August 16, 2014 [Page 37]
Internet-Draft Detecting Overlay Segment Failure February 2014
11. Management Considerations
None
Jain, et al. Expires August 16, 2014 [Page 38]
Internet-Draft Detecting Overlay Segment Failure February 2014
12. Acknowledgements
This document is the outcome of many discussions among many people,
including Saurabh Shrivastava, Krishna Ram Kuttuva Jeyaram and Suresh
Boddapati of Nuage Networks, Jorge Rabadan of Alcatel-Lucent, Inc and
Rahul Kasralikar of Juniper Networks, Inc.
Jain, et al. Expires August 16, 2014 [Page 39]
Internet-Draft Detecting Overlay Segment Failure February 2014
13. IANA Considerations
Action-1: This specification reserves a IANA UDP Port Number to be
used when sending the Overlay OAM Packet
Action-2: This specification reserves a IANA Ethernet unicast Address
for VXLAN/NVGRE Exception handling. This Address needs to be
reserved from the block. "IANA Ethernet Address block - Unicast Use"
Jain, et al. Expires August 16, 2014 [Page 40]
Internet-Draft Detecting Overlay Segment Failure February 2014
14. References
14.1. Normative References
[I-D.draft-ietf-mpls-in-udp]
Xu, Sheth, Yong, Pignataro, Yongbing , and Li,
"Encapsulating MPLS in UDP", May 2013.
[I-D.draft-lasserre-nvo3-framework]
Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for DC Network Virtualization",
September 2011.
[I-D.draft-mahalingam-dutt-dcops-vxlan]
Mahalingam, M., Dutt, D., Agarwal, P., Kreeger, L.,
Sridhar, T., Bursell, M., and C. Wright, "VXLAN: A
Framework for Overlaying Virtualized Layer 2 Networks
over Layer 3 Networks", May 2013.
[I-D.draft-singh-nvo3-nvgre-router-alert]
Singh, K., Jain, P., Balus, F., and W. Henderickx, "NVGRE
Router Alert Option", May 2013.
[I-D.draft-singh-nvo3-vxlan-router-alert]
Singh, K., Jain, P., Balus, F., and W. Henderickx, "VxLAN
Router Alert Option", May 2013.
[I-D.draft-sridharan-virtualization-nvgre]
Sridharan, M., Duda, K., Greenberg, A., Lin, G., Pearson,
M., Thaler, P., Tumuluri, C., Venkataramiah, N., and Y.
Wang, "NVGRE: Network Virtualization using Generic Routing
Encapsulation", February 2013.
[RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
Encoding", RFC 3032, January 2001.
[RFC4023] Worster, T., Rekhter, Y., and E. Rosen, "Encapsulating
MPLS in IP or Generic Routing Encapsulation (GRE)",
RFC 4023, March 2005.
[RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP
Virtual Private Networks (VPNs)", RFC 4365, February 2006.
[RFC4379] Kompella, K. and G. Swallow, "Detecting Multi-Protocol
Label Switched (MPLS) Data Plane Failures", RFC 4379,
February 2006.
Jain, et al. Expires August 16, 2014 [Page 41]
Internet-Draft Detecting Overlay Segment Failure February 2014
14.2. Informative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC4330] Mills, D., "Simple Network Time Protocol (SNTP) Version 4
for IPv4, IPv6 and OSI", RFC 4330, January 2006.
Jain, et al. Expires August 16, 2014 [Page 42]
Internet-Draft Detecting Overlay Segment Failure February 2014
Authors' Addresses
Pradeep Jain
Nuage Networks
755 Ravendale Drive
Mountain View, CA 94043
USA
Email: pradeep@nuagenetworks.net
Kanwar Singh
Nuage Networks
755 Ravendale Drive
Mountain View, CA 94043
USA
Email: kanwar@nuagenetworks.net
Florin Balus
Nuage Networks
755 Ravendale Drive
Mountain View, CA 94043
USA
Email: florin@nuagenetworks.net
Wim Henderickx
Alcatel-Lucent
Copernicuslaan 50
Antwerp 2018
Belgium
Email: wim.henderickx@alcatel-lucent.be
Vinay Bannai
PayPal
2211 N. First St,
San Jose 95131
USA
Email: vbannai@paypal.com
Jain, et al. Expires August 16, 2014 [Page 43]
Internet-Draft Detecting Overlay Segment Failure February 2014
Ravi Shekhar
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
USA
Email: rshekhar@juniper.net
Anil Lohiya
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
USA
Email: alohiya@juniper.net
Jain, et al. Expires August 16, 2014 [Page 44]