Network Working Group D. Malas
Internet Draft Level 3 Communications
Expires: November 2006 May 22, 2006
SIP Performance and Reliability Metrics
draft-malas-performance-metrics-00.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that
any applicable patent or other IPR claims of which he or she is
aware have been or will be disclosed, and any of which he or she
becomes aware will be disclosed, in accordance with Section 6 of
BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on November 22, 2006.
Abstract
This document defines the use of industry recommended reliability
metrics for use with the SIP.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [1].
Table of Contents
1. Introduction...................................................2
malas Expires November 22, 2006 [Page 1]
Internet-Draft SIP Reliability and Performance Metrics May 2006
2. Metrics........................................................3
2.1. Answer Seizure Ratio (ASR)................................3
2.2. Post Invite Delay (PID)...................................3
2.3. Call Hold Time (CHT)......................................4
2.4. Ineffective Attempts (IA).................................5
2.5. Cutoff Calls (CC).........................................5
2.6. Call Success Rate (CSR)...................................5
2.7. Defects Per Million (DPM).................................6
3. Security Considerations........................................6
4. IANA Considerations............................................6
5. Conclusions....................................................6
6. Acknowledgments................................................6
7. References.....................................................6
7.1. Normative References......................................6
7.2. Informative References....................................7
Author's Addresses................................................7
Intellectual Property Statement...................................7
Disclaimer of Validity............................................8
Copyright Statement...............................................8
Acknowledgment....................................................8
1. Introduction
SIP has become a standard among many service providers, vendors, and
end users. Although there are many different standards for measuring
the reliability and performance of signaling protocols, none of these
have been adapted for use with SIP. This document is intended for
providing a guideline for the above listed entities in providing a
standard approach for measuring and reporting SIP reliability and
performance metrics. This will allow a common approach and
understanding of expectations between service providers, vendors, and
the users of those services.
Not all metrics for reliability and performance map to all
applications of the SIP. This document provides an overview of many
different metrics, which may be used as an individual or set of
metrics necessary based on the use of SIP.
There are many metrics available for determining both reliability and
performance. Although this document contains a number of them, it is
not intended to be exhaustive. Instead, it is designed to provide a
common sub-set with a common agreed upon definition.
malas Expires November 22, 2006 [Page 2]
Internet-Draft SIP Reliability and Performance Metrics May 2006
2. Metrics
2.1. Answer Seizure Ratio (ASR)
ASR [4] is defined as the number of INVITE requests resulting in a
200 OK response, to the total number of attempted INVITE requests.
# of INVITE Requests w/ associated 200OK
ASR = ----------------------------------------
Total # of INVITE Requests
UA1 UA2
| |
|INVITE |
|---------------------> |
| /\ 100|
| <---------||----------|
| || |
| Seizure |
| || 180|
| <---------||----------|
| \/ 200|
| <---------------------|
| |
| |
2.2. Post Invite Delay (PID)
In a successful request attempt, PID [4] is defined as the time
interval from the moment the INVITE message containing the necessary
information is passed by the originating agent or user to the
intended mediation or destination agent until a response is received
indicating an audible or visual status of the INVITE request. When
SIP is utilized for traditional telephony-like dialing, this metric
is synonymous with the Post Dial Delay (PDD). In SIP, the message
indicating status would be a 180 and 183 message received in response
to an INVITE request. In some cases, a 180 or 183 message is not
received, but rather a 200 message is received as the first status
message instead. In these situations, the 200 message would be used
to calculate the interval.
In a failed request attempt, the interval is defined from the INVITE
request and a failure indication status response. A failure response
is described as a 4XX, 5XX, or possible 6XX message.
malas Expires November 22, 2006 [Page 3]
Internet-Draft SIP Reliability and Performance Metrics May 2006
PID = Time of Status Indicative Response Time of INVITE
SUM (Time of Status Indicative Response Time of INVITE)
APID = ---------------------------------------------------------
SUM # of INVITE Requests
UA1 UA2
| |
|INVITE |
|---------------------> |
| /\ 100|
| <---------||----------|
| PID |
| || |
| \/ 180|
| <---------------------|
| |
2.3. Call Hold Time (CHT)
CHT [4] is usually calculated as an average and is defined as the
duration of a call from receipt of a 200 OK and an associated BYE
message indicating call completion.
CHT = Time of BYE Time of 200 OK
SUM (Time of BYE Time of 200 OK)
ACHT = -----------------------------------------
SUM # of INVITE w/ 200OK & BYE
UA1 UA2
| |
|INVITE |
|---------------------> |
| 100|
| <---------------------|
| 180|
| <---------------------|
| 200|
| <---------------------|
| /\ |
| || |
| CHT |
| || |
|BYE \/ |
|---------------------> |
malas Expires November 22, 2006 [Page 4]
Internet-Draft SIP Reliability and Performance Metrics May 2006
2.4. Ineffective Attempts (IA)
IA [3] utilizing SIP is defined as a proxy or agent, which internally
releases a call with a failed or congested condition. The following
failure responses may provide a guideline for this criterion:
1. 408 Request Timeout
2. 500 Server Internal Error
3. 503 Service Unavailable
4. 504 Server Timeout
# of IA Attempts
IA % = -----------------------
# of INVITE Requests
UA1 UA2
| |
|INVITE |
|---------------------> |
| 503|
| <---------------------|
|ACK |
|---------------------> |
| |
2.5. Cutoff Calls (CC)
CC [3] utilizing SIP is defined as an active session, which is
terminated due to a condition other than a normal disconnected such
as a BYE or CANCEL.
# of CCs
CC % = -----------------------------------------
# of INVITE Requests w/ associated BYE
2.6. Call Success Rate (CSR)
CSR is intended to provide a metric, which summarizes the overall
service provided. A recommended method for calculating CSR is
through utilizing a combination of IA and CC.
CSR = 100% - (IA% - CC%)
malas Expires November 22, 2006 [Page 5]
Internet-Draft SIP Reliability and Performance Metrics May 2006
2.7. Defects Per Million (DPM)
DPM was originally developed by Motorola, and is now more commonly
referred to as Six Sigma quality management. It defines a method for
calculating the number of defects per million items produced. Many
service providers are using this metric. In SIP, this would be
calculated as the number of defective failure responses per million
final release responses received in response to an INVITE request.
The following failure responses may provide a guideline for this
defective criterion:
1. 500 Server Internal Error
2. 503 Service Unavailable
3. 504 Server Timeout
3. Security Considerations
Security should be considered in the aspect of securing the relative
data utilized in providing input to the above calculations. All
other aspects of security should be considered as described in [2].
4. IANA Considerations
There are no IANA considerations at this time.
5. Conclusions
The proposed guideline provides a description of common reliability
and performance metrics, and their defined use with SIP. The use of
these metrics will provide a common viewpoint across all vendors,
service providers, and customers.
6. Acknowledgments
TBD
7. References
7.1. Normative References
[1] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
malas Expires November 22, 2006 [Page 6]
Internet-Draft SIP Reliability and Performance Metrics May 2006
[2] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A.,
Peterson, J., Sparks, R., Handley, M. and E. Schooler, "SIP:
Session Initiation Protocol", RFC 3261, June 2002.
[3] Bellcore, LSSGR: Reliability, Section 12, GR-512-CORE Issue
2, January 1998.
[4] ITU-T, Terms and Defintions of Traffic Engineering,
Recommendation E.600, March 1993.
7.2. Informative References
Author's Addresses
Daryl Malas
Level 3 Communications LLC
1025 Eldorado Blvd.
Broomfield, CO 80021
USA
EMail: daryl.malas@level3.com
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
malas Expires November 22, 2006 [Page 7]
Internet-Draft SIP Reliability and Performance Metrics May 2006
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
malas Expires November 22, 2006 [Page 8]