Network Working Group Jim Boyle
Internet Draft Craig White
Expiration Date: January 2001 Joe Lawrence
Level 3
July 2000
SONET Synchronous Transport Signal over IP
draft-boyle-sts-ip-00.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Boyle, et al. [Page 1]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
Abstract
A proposal is made to carry Synchronous Transport Signal
(STS-N) services over packetized networks, in particular over IP
networks. This has the potential to dramatically lower the price,
ease the provisioning and grooming, and improve managability of
delivering these types of circuits. Two proposals are put forth for
discussion on how STS services could be transported over an IP
network. The first proposal, referred to as SPE1/IP, is to break OC-N
signals down into their STS-1 blocks, identify the SPE components and
packetize these for transmission across a packet network. The second
proposal, referred to as STSc/IP breaks down the STS signal into the
underlying concatenated services and transports configurable length
segments of the signal to the far end of the data network. With
either of these approaches, different STS-N services on a given port
can be connected to different ports across the network and the
network will groom them onto the most optimal path available.
Table of Contents
1 Specification of Requirements .......................... 3
2 Introduction ........................................... 3
3 SONET Background ....................................... 4
4 Encapsulation Alternatives ............................. 6
4.1 SPE-1 transport over IP (SPE1/IP) ...................... 6
4.2 STS-Nc transport over IP (STSc/IP) ..................... 8
4.3 Comparison of SPE1/IP and STSc/IP ...................... 8
5 Issues of Efficiency ................................... 9
5.1 Efficiency Losses ...................................... 9
5.2 Efficiency Gains ....................................... 9
6 Functionality Requirements ............................. 10
6.1 Playback buffers on IP/TDM conversion .................. 10
6.2 QoS in IP network ...................................... 10
6.3 Traffic engineered control plane ....................... 10
7 Security Considerations ................................ 11
8 Intellectual Property Disclaimer ....................... 11
9 Acknowledgements ....................................... 11
10 References ............................................. 11
11 Author Information ..................................... 11
Boyle, et al. [Page 2]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
1. Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
2. Introduction
Anyone familiar with operating a private line network understands
that, even though there maybe additional costs, technologies that
ease grooming and provisioning are worth consideration. This is one
of the areas where the next generation of transmission equipment is
improving by leveraging IP based control protocols and head of
service automated provisioning. However, there do exist some issues
of multi-vendor compatibility with this approach. This makes an
approach where private line services are converted into an open data
protocol, such as RTP/IP and/or MPLS attractive.
By converting the TDM service to IP, the provider is now also able to
use network elements such as ethernet switches which are geared to
much wider, mass markets and potentially ride a much more aggressive
price/performance curve than standard telecommunication equipment.
These benefits were the impetus of the authors investigation into
converting high capacity private line services, STS1 and STSNc
services in particular, into IP for transport over a data network.
Investigation yielded the result that this could likely be done with
rather plain technology, as is used in current transmission
equipment, and might yield significant network savings due to signal
compression potential.
This memo is intended as a strawman statement of approach and
essential functionality. The description, though written rather
close to the physical and network layer, is actually intended to be
more in line with a transport protocol, such as RTP. In fact, though
this memo is written in the form of a protocol specification, it may
well be that this functionality can, and should, be adapted to
existing protocols to ease implementation.
Boyle, et al. [Page 3]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
3. SONET Background
SONET, as its name implies, is a hierarchical protocol for
synchronously transporting TDM circuits across an optical network.
Over the length of a circuit, there is a hierarchical breakdown of
functionality which allows for multiplexing, service monitoring and
alarming and signal degradation monitoring at high data rates. From
end to end, at the highest layer, SONET provides monitoring service
via it's Path overhead. This is transported to the customer and
allows for continuity verification as well as error calculation over
the service payload. The physical optical fiber is often in a 1:1
relationship with the section, and often directly corresponds to the
line. The difference is that multiple lines usually terminate at a
Digital Cross Connect (DACS) or Add Drop Multiplexer (ADM) where
synchronization and alignment across lines is required. With the
advent of WDM, the section no longer directly corresponds to a fiber,
it refers to logically adjacent SONET equipment such as an ADM and a
3R. In fact, with some WDM products, it is possible for the WDM
equipment to operate in a non section processing, or pass-through,
mode in which case the WDM signal has 1:1 parity of Path, Line and
Section. Here is a diagram to illustrate SONET's signal hierarchy:
|---------------------- Path -----------------------|
|--- Line ---|--- Line ---|--- Line ---|--- Line ---|
|--- Sect ---| *** See Figure 2 *** |--- Line ---|
Router------ADM----------ADM----------ADM------Router
Figure 1 - SONET Signal Hierarchy
Below is a blowup of one of the ADM-ADM Lines to illustrate how SONET
section no longer corresponds directly to a fiber.
ADM-----WDM-----ILA-----3R-----ILA-----WDM-----ADM
|----------------------Line----------------------|
|-----------Sect--------|------------Sect--------|
Figure 2 - SONET Line and Section in WDM environment
In SONET, the digital signal is referred to as an STS (Synchronous
Transport Signal). This signal is broken down into a repetitive
series of STS frames, each frame for an STS being generated every 125
us (8000 fps). Each frame can be thought of as having 90 columns and
9 rows. The first 3 columns contain the section and line overhead.
The path overhead is contained within the Synchronous Payload
Envelope (SPE).
Boyle, et al. [Page 4]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
+-----------------------------------------------------+
| | | | ... |
| SOH | ... |
|_|_|_| ... |
| | | | ... PATH OH and Payload |
| | | | ... |
| LOH | ... |
| | | | ... |
| | | | ... |
| | | | ... |
+-----------------------------------------------------+
Figure 3 - STS frame with transport overhead
+-----------------------------------------------------+
| | | | |
| SOH | |
|_|_|_| _____________________________________|
| | | |_________| | |
| | | | |P| |
| LOH | |O| STS-1 SPE |
| | | | |H| |
| | | | | | |
|_|_|_|_ _ _ _ _|N|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|
| | | | | | |
| SOH | | | |
|_|_|_| |_|___________________________________|
| | | |_________| |
| | | | ... |
| LOH | ... |
| | | | ... |
| | | | ... |
| | | | ... |
+-----------------------------------------------------+
Figure 4 - STS frame including SPE
Notice that the payload floats inside the STS frame. It is
referenced in the LOH with a Path Payload Pointer, and can change.
This is to allow slight variations in line rates on DACS or ADMs
which a path traverses. The LTE can then either byte positive or
negative stuff a signal.
The basic building block of SONET is the STS-1. This is always
carried over a physical facility, in particular a fiber operates at a
bit rate which is a specified multiple of STS-1s. The STS-1s are
byte multiplexed together as show in Figure 5.
Boyle, et al. [Page 5]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
STS1-#01 +
\
STS1-#02 +-+- #03#02#01 +
/ \
STS1-#03 + \
\
STS1-#04 + \
\ \
STS1-#05 +-+- #06#05#04 + \
/ \___ \
STS1-#06 + \__\
__ + #12#09#06#03#11#08#05#02#10#07#04#01
STS1-#07 + ___/ /
\ / /
STS1-#08 +-+- #09#08#07 + /
/ /
STS1-#09 + /
/
STS1-#10 + /
\ /
STS1-#11 +-+- #12#11#10 +
/
STS1-#12 +
Figure 5 - STS-12 multiplexing of underlying STS-1s
The byte pattern for the STS-12 presented across an OC-12
#12#09#06#03#11#08#05#02#10#07#04#01->
is a byte interleaving of all the STS-1s. In 125 us, a byte from
each STS-1 is transmitted over the OC-12 signal.
At each SONET switching element, it is customary for the signal to be
demultiplexed into the STS-1s into a small buffer and then for the
appropriate egress port to pull the signal out of the buffers during
its multiplexing.
4. Encapsulation Alternatives
4.1. SPE-1 transport over IP (SPE1/IP)
In this mode, the STS-1s from an STS-N signal are broken down and are
written into a buffer. There is no need to forward any LOH or SOH,
however it is required to identify and forward the POH so that the
customer can monitor continuity and performance. A complete SPE is
written out in several SPE segments to a buffer. An SPE segment
refers to the portions of an SPE that reside in different STSs.
Although the POH for different services within an STS-N structure may
be located at different offsets from their respective LOH, the SPE
for concatenated STSs must be treated as all at the same offset as
the first STS in the concatenated service. The pointer to the start
of the SPE is known for all STSs within an existing service, and a
complete SPE has been written out to buffer. At this point, the SPE
Boyle, et al. [Page 6]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
segments are encapsulated and forwarded across the network. The
essential information for a header would include:
SPE1/IP Header (6 bytes)
o Version field (4 bits = 1)
o Operation Code, OP, (2 bits)
- 00 = standard user data format
- 01 = Unused, reserved for extended data format
- 10 = OAM
- 11 = unused
o Parity bit (2 bits)
- P0 - bit 1, even parity across header
- P1 - bit 2, even parity across entire PDU
o STS reference (16 bits)
o Sequence number (16 bits)
+-------------+-------------+-------------+-------------+
|V |OP|PP | Unused | STS Reference |
+-------------+-------------+-------------+-------------+
| Sequence Number |
+-------------+-------------+
The sequence number is synchronized across all of the SPE segments of
an SPE for a given concatenated service and increases by 1 every 125
us. At 8000 fps, it would wrap in just under 8 seconds. If a packet
is dropped, then an all 1s segment is inserted in its place for
multiplexing into the next line in the service. Bit errors within
the frame are noticed by standard SONET monitoring facilities, in
this case the B3 byte in the POH will alert the end-user with
potentially 1 or 3 BIP-8 violations depending on if the bit error
happened in the B3 byte of an SPE or not. For all 1s inserted
segments that are multiplexed into an SPE, at least 3 BIP-8
violations would occur at the path layer. Bit errors in the
encapsulation header are recognized in a non-recoverable fashion by
the parity bit P0 which is set or cleared to create even parity
across the rest of the header. The packet should be discarded if
such a bit error is detected in the encapsulation header. The P1 bit
can be used to check parity for bit error across the header and the
SPE. This can be done for service management to determine if the
data network has an unexpectedly high bit error rate. This is
similar to section and line monitoring in a standard SONET network.
The STS reference is similar to a UDP port and remains constant for a
given STS traversing a node. It is set by the node that converts the
STS from the TDM to the IP domain.
Boyle, et al. [Page 7]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
4.2. STS-Nc transport over IP (STSc/IP)
In this mode, the contents of the concatenated SPE are written out to
a buffer upon demultiplexing of the TDM line into the TDM/IP
conversion platform. The signal is then sampled at a configurable
block size, 1000 bytes for example, and this information is
encapsulated and sent across the IP network
STSc/IP Header (8 bytes)
o Version field (4 bits = 1)
o Operation Code, OP, (2 bits)
- 00 = standard user data format
- 01 = Unused, reserved for extended data format
- 10 = OAM
- 11 = unused
o Parity bit (2 bits)
o STS reference (16 bits)
o Sequence number (32 bits)
+-------------+-------------+-------------+-------------+
|V |OP|PP | Unused | STS Reference |
+-------------+-------------+-------------+-------------+
| Sequence Number |
+-------------+-------------+-------------+-------------+
The number of frames per second are a function of the number of the
sample size and the service bit-rate. Sequence number wrap can be
derived from that, but with a range of 0 to 2^32-1, a worst case
scenario of sampling an OC192 at 100 bytes yields wrap in over 5
minutes. Parity bits are as in section 3.2. The POH still must be
identified in the transported payload. One approach would be to use
integer math on the sequence number and have the concatenated STSc be
segmented into chunks which together exactly construct the payload of
one frames worth of SPE. Another might be to use an extended data
format that points into the payload to identify the start of SPE for
each frame.
4.3. Comparison of SPE1/IP and STSc/IP
SPE1/IP is not very flexible, but in such an approach a certain
simplicity is gained. In fact, most TDM gear today is built around
breaking down services into STS-1 for switching. STSc/IP is very
flexible, but that might add too much complexity to implementation
and interoperability. Either approach requires that the service
provider be intimate with the channel structure of the OC services
they provide, either through static provisioning or through
Boyle, et al. [Page 8]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
derivation from the LOH.
5. Issues of Efficiency
5.1. Efficiency Losses
As SONET is non-hierarchically built around the STS-1 payload, with
direct multiplexing, the overhead remains constant at around 4% (3
out of 90 columns, plus POH in first STS-1). It doesn't matter if
the services are all STS-1s, or a concatenated service occupying the
whole linespeed such as an STS-12c in an OC-12c, there exist LOH and
POH for each STS.
This creates a bit of an issue when the payload is encapsulated and
forwarded out into an IP network. As an example, if one of the core
links in a network was a OC-192c, it would not be able to carry 192
STS-1s of service, as the IP traffic must itself become SONET payload
on this link. In this case, one provisioned service is going to have
to take another route, which introduces the need for skew recovery
buffers. This could be problematic for networks whose service
bandwidth approaches their internal link speeds. Perhaps it is time
to try to take use of all those D and E bytes in the SONET header?
Even that wouldn't carry the additional 3-5% of overhead incurred by
IP encapsulation. One should note that an OC-12 can be routed across
a Gigabit Ethernet, and the assumption is that volume and target
market of Gigabit Ethernet make it likely cheaper than the
corresponding SONET interface.
5.2. Efficiency Gains
The losses outlined in the previous section appear problematic for
this application, however there exists a large potential for reducing
the amount of network that has to be built out to support a set of
private line services by using technology like this. It may be
trivial to remove bandwidth from STS services being used for data
transport by compressing predictable frame sequences. Suggested
sequences include HDLC idle (0x7Es), STS services that are either in
alarm (AIS) or are not in service (unequiped) (all ones), unequiped
asynchronous DS3 and idle ATM cells. One approach for compression
could be to send a packet that has a the base sequence written just
once, and it is to be repeated out to fill out the remainder of the
SPE. Another approach, especially for unequiped circuits would be to
use signalling to establish state on either end of the circuit
reflecting unequiped and transport nothing through the data network.
For a carrier who has sold an OC12 structured as 12 STS1, of which
only 8 are in service, a significant savings may be realizable.
Boyle, et al. [Page 9]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
Further efficiency gains can be realized by relaxing the timing
precision of the service if the STS payload is HDLC, PPP or FR
[Martini]. However, this draft outlines an approach to exactly mimic
a TDM based SONET service.
6. Functionality Requirements
6.1. Playback buffers on IP/TDM conversion
The extent of playback buffering required (and user experienced
latency) is somewhat traded off with QoS of IP network described in
the following section. The playback logic must allow for packets
received out of sequence to be ordered correctly and played out in
sequence and insertion of all ones payload for packets that didn't
arrive in time for enqueue. Also, if the contents of an STS-Nc
service take multiple routes through the network, additional skew
recovery buffers will be required.
6.2. QoS in IP network
It is expected that the network is not chronically congested. For a
multiservice network, the TDM/IP (and other delay sensitive traffic
such as VoIP) can be forwarded without too much delay. However, in
conjunction with the previous section, it might not be required that
this be strict "next packet served" type queuing.
6.3. Traffic engineered control plane
There must be a way to keep proper coordination of the traffic
assigned to network bandwidth resources. The application requires a
way to signal the bandwidth it requires from the network. One
potential approach is use of the Resource Reservation Protocol
[RSVP]. A more direct approach would be to have the host on which
the application runs signal traffic engineered MPLS tunnels [RSVP-
TE][CRLDP] between itself and other TDM/IP conversion devices with
which it has common services. These tunnels could be sized to
accommodate multiple STS services, or sized to carry just one, or a
portion of one, STS service. MPLS tunneling would allow the TDP/IP
device to find a route for their traffic across the network,
potentially find backup routes, and groom their traffic onto new,
more optimal paths in the network. The concern would be one of
scalability as all TDM/IP devices would have to belong to a common
TE-IGP domain. An alternate approach would be for the TDM/IP devices
to communicate the bandwidth information of the service via loosely
routed TE-LSPs, and to have switches along the way with more complete
Boyle, et al. [Page 10]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
information insert the route the service should take.
7. Security Considerations
8. Intellectual Property Disclaimer
This document is being submitted for use in IETF standards
discussions. Level 3 Communications, Inc. has filed one or more
patent applications relating to the technology outlined in this
document.
9. Acknowledgements
This draft has benefited from the discussions and assistance provided
by Tom Issenhuth, Jack Waters and Danny McPherson, among others.
10. References
[Martini] "Transport of Layer 2 Frames Over MPLS", draft-martini-
l2circuit-trans-mpls-01.txt (work in progress)
[RSVP] "Resource ReSerVation Protocol -- Version 1 Functional
Specification", RFC2205
[RSVP-TE] "Extensions to RSVP for LSP Tunnels", draft-ietf-mpls-lsp-
tunnel-06.txt (work in progress)
[CR-LDP] "Constraint-Based LSP Setup Using LDP", draft-ietf-mpls-cr-
ldp-04.txt (work in progress)
11. Author Information
Jim Boyle
Level 3 Communications, LLC.
1025 Eldorado Blvd.
Broomfield, CO 80021
e-mail: jboyle@level3.net
phone: 720.888.1192
Boyle, et al. [Page 11]
Internet Draft draft-boyle-sts-ip-00.txt July 2000
Craig White
Level 3 Communications, LLC.
1025 Eldorado Blvd.
Broomfield, CO 80021
e-mail: craig.white@level3.com
phone: 720.888.2375
Joe Lawrence
Level 3 Communications, LLC.
1025 Eldorado Blvd.
Broomfield, CO 80021
e-mail: joe.lawrence@level3.com
phone: 720.888.1000
Boyle, et al. [Page 12]