CDNI Logging Formats for HTTP and HTTP Adaptive Streaming Deliveries
draft-lefaucheur-cdni-logging-delivery-00
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Authors | François Le Faucheur , Mahesh Viveganandhan , Kent Leung | ||
| Last updated | 2012-02-24 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-lefaucheur-cdni-logging-delivery-00
Network Working Group F. Le Faucheur
Internet-Draft M. Viveganandhan
Intended status: Informational K. Leung
Expires: August 26, 2012 Cisco
February 23, 2012
CDNI Logging Formats for HTTP and HTTP Adaptive Streaming Deliveries
draft-lefaucheur-cdni-logging-delivery-00
Abstract
The interconnection of Content Distribution Networks (CDNs) is
motivated by several use cases. CDN Interconnection can be achieved
through four CDNI interfaces, one of which is the CDNI Logging
interface. This document discusses CDNI logging formats for content
deliveries performed using HTTP or HTTP adaptive streaming.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 26, 2012.
Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
Le Faucheur, et al. Expires August 26, 2012 [Page 1]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4
2. CDNI Logging for regular HTTP Delivery . . . . . . . . . . . . 4
2.1. Regular HTTP Log Triggers . . . . . . . . . . . . . . . . 4
2.2. Regular HTTP Log Fields . . . . . . . . . . . . . . . . . 5
3. CDNI Logging for HTTP Adaptive Streaming . . . . . . . . . . . 6
3.1. HTTP Adaptive Logging Session . . . . . . . . . . . . . . 6
3.2. CDNI Log formats for HTTP Adaptive Streaming . . . . . . . 7
3.2.1. Segment-Based Logging . . . . . . . . . . . . . . . . 7
3.2.2. Event-Based Logging . . . . . . . . . . . . . . . . . 9
3.2.3. Summary-Based Logging . . . . . . . . . . . . . . . . 10
4. Performance Monitoring . . . . . . . . . . . . . . . . . . . . 10
5. CDNI Log Encoding and Transport . . . . . . . . . . . . . . . 11
5.1. CDNI Log Header Information . . . . . . . . . . . . . . . 11
5.2. CDNI per-Log-Record Information . . . . . . . . . . . . . 11
5.3. CDNI Log Footer Information . . . . . . . . . . . . . . . 12
6. Impact on CDNI Metadata Requirements . . . . . . . . . . . . . 12
7. Generation of CDNI Logs . . . . . . . . . . . . . . . . . . . 12
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
9. Security Considerations . . . . . . . . . . . . . . . . . . . 13
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 13
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13
11.1. Normative References . . . . . . . . . . . . . . . . . . . 13
11.2. Informative References . . . . . . . . . . . . . . . . . . 13
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14
Le Faucheur, et al. Expires August 26, 2012 [Page 2]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
1. Introduction
The interconnection of Content Distribution Networks (CDNs) is
motivated by several use cases, such as those described in
[I-D.ietf-cdni-use-cases]. The overall problem space for CDN
Interconnection (CDNI) is described in
[I-D.ietf-cdni-problem-statement]. The CDN Interconnection framework
is defined in [I-D.davie-cdni-framework] and the requirements for the
CDN Interconnection solution are specified in
[I-D.ietf-cdni-requirements].
One of the CDNI interfaces defined in these documents is the CDNI
Logging interface whose description is quoted here:
"o CDNI Logging interface: This interface allows the Logging system
in interconnected CDNs to communicate the relevant activity logs in
order to allow log consuming applications to operate in a multi-CDN
environments. For example, an upstream CDN may collect delivery logs
from a downstream CDN in order to perform consolidated charging of
the CSP or for settlement purposes across CDNs. Similarly, an
upstream CDN may collect delivery logs from a downstream CDN in order
to provide consolidated reporting and monitoring to the CSP."
The logging interface is also discussed in further details in
[I-D.bertrand-cdni-logging].
As discussed in the CDNI framework [I-D.davie-cdni-framework] logs
associated with content delivery using HTTP adaptive streaming have
specific requirements in terms of summarization:
"Most schemes to deliver HTTP-based adaptive bit-rate video use a
large number of relatively small HTTP requests (e.g., one request per
3-second chunk of video.) It may be desirable to aggregate Logging
information so that a single log entry is provided for the entire
video rather than for each chunk. Note however that such aggregation
requires a degree of application awareness in dCDN to recognize that
the many HTTP requests correspond to a single video."
The prime purpose of this document is to define CDNI Logging formats
corresponding to content delivery that uses HTTP or HTTP adaptive
streaming.
Section 2 discusses the aspect of CDNI Logging that are specific to
content delivery performed via regular HTTP. Section 3 discusses the
aspect of CDNI Logging that are specific to content delivery
performed via HTTP adaptive streaming. Section 5 discusses the
aspect of CDNI Logging that are generic to all the CDNI Logs for
delivery.
Le Faucheur, et al. Expires August 26, 2012 [Page 3]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
CDNI Logging for other events than content delivery (e.g. failures,
request routing, service monitoring,...) will be discussed in
subsequent versions.
1.1. Terminology
This document uses all terminology defined in
[I-D.ietf-cdni-problem-statement]. In addition, we introduce the
following terminology:
o HTTP Adaptive Streaming Segment (or Segment): piece of a content
item that is accessible as a standalone HTTP resource, and thus is
discreetly accessible via its own Unique Resource Identifier
(URI).
o HTTP Adaptive Streaming Logging Session (or Session): state
maintained by the delivering CDN allowing association, for logging
purposes, of the actions (e.g. delivery of Segments) corresponding
to a given content request by a given End User. Note that actions
associated with a given Session may all be performed by a single
Surrogate of the delivering CDN, or may be performed by different
Surrogates of the delivering CDN successively (e.g. in case of
surrogate fail-over) or performed by different Surrogates in
parallel (e.g. in case of load-balancing of Segments delivery).
o Adaptive Streaming Manifest File (or Manifest File): file that
indicates to the End-User the information (e.g. content
representations available, rules for mapping between a content
Segment and its URI) necessary to request the appropriate Segment
at the appropriate time based on the End-User adaptive logic (e.g.
increase bitrate, decrease bitrate) and behavior (e.g. seek,
pause, play).
2. CDNI Logging for regular HTTP Delivery
This section discusses the triggers on which a CDNI log is generated
for delivery of content using regular HTTP delivery, as well as the
fields contained in the corresponding log.
2.1. Regular HTTP Log Triggers
A CDNI Log is generated for a regular HTTP delivery on the following
triggers:
Le Faucheur, et al. Expires August 26, 2012 [Page 4]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
+-----------------+-------------------------------------------------+
| Event | Description |
+-----------------+-------------------------------------------------+
| content Request | Reception and processing of a request for a |
| | content |
+-----------------+-------------------------------------------------+
2.2. Regular HTTP Log Fields
A CDNI Log for regular HTTP delivery contains the following fields:
+---------------+------------+--------------------------------------+
| Field | Descriptio | Examples |
| | n | |
+---------------+------------+--------------------------------------+
| Current-Time | Time, in | [20/Feb/2012:00:29.510+0200] |
| | millisecon | |
| | ds, at | |
| | which the | |
| | request | |
| | was | |
| | received. | |
| Time-to-Serve | Time, in | 952195 |
| | microsecon | |
| | ds, taken | |
| | to | |
| | complete | |
| | the | |
| | request. | |
| Client-IP | IP address | 64.103.25.105 |
| | of the | |
| | requesting | |
| | client. | |
| Action | Squid | TCP_HIT, TCP_MISS, ... |
| | action | |
| | describing | |
| | how the | |
| | request | |
| | was | |
| | treated | |
| | locally | |
| | descriptio | |
| | n(e.g., | |
| | cache | |
| | hit/miss) | |
| | . | |
Le Faucheur, et al. Expires August 26, 2012 [Page 5]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
| Status-Return | HTTP | 200, 404, ... |
| ed | response | |
| | code. | |
| Bytes-Transfe | Bytes sent | 23567992 |
| rred | to the | |
| | client, | |
| | including | |
| | the | |
| | headers. | |
| Method | HTTP | GET |
| | request | |
| | method. | |
| URI | URI of | http://cache3.cdn1.com/movie/ice/ice |
| | requested | movie.mpg |
| | content. | |
| Content-Type | MIME-Type | video/mpeg |
| | from the | |
| | reply | |
| | header. | |
+---------------+------------+--------------------------------------+
3. CDNI Logging for HTTP Adaptive Streaming
This section discusses the CDNI logs for delivery of content using
HTTP adaptive streaming.
3.1. HTTP Adaptive Logging Session
There are multiple methods by which the delivering CDN can create/
maintain/delete an HTTP Adaptive Streaming Logging Session.
As a first example method, the delivering CDN may create a Session on
successful authorization of delivery of the Manifest File for a
particular End-User content request, create a per-Session coookie and
then validate the association of Segment requests with the Session
through HTTP cookie mechanisms. The delivering CDN may then maintain
the Session as long as Segment requests are being received for that
Session. The delivering CDN may ultimately delete the Session once
Segment requests for that Session are no longer received for some
period of time.
As another example, Session identification information included in
the fragment URIs may be used by the delivering CDN to associate
Segment requests with a Session and therefore for determining when a
Session is to be created/maintained/deleted.
Methods for tracking HTTP Adaptive Streaming Logging Sessions in a
Le Faucheur, et al. Expires August 26, 2012 [Page 6]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
delivering CDN are implementation specific and thus outside the scope
of the present document.
3.2. CDNI Log formats for HTTP Adaptive Streaming
Three CDNI Logging formats are defined for delivery using HTTP
adaptive streaming:
o Segment-Based Logging, as specified in Section 3.2.1.
o Event-Based Logging, as specified in Section 3.2.2.
o Summary-based Logging, as specified in Section 3.2.3.
An implementation of the CDNI Logging interface MUST support logging
for delivery of content using HTTP adaptive streaming in the Segment-
Based Logging format and the Event-Based Logging format, and MAY
support logging in the Summary-Based Logging format.
By default (i.e. unless it is explicitly requested that another
format be used as discussed in Section 6), CDNI logs for delivery of
content using HTTP adaptive streaming MUST be formatted as per the
Event-Based format.
3.2.1. Segment-Based Logging
3.2.1.1. Segment-Based Log Triggers
A Segment-Based Log is generated on the following triggers:
+---------------------+---------------------------------------------+
| Event | Description |
+---------------------+---------------------------------------------+
| Segment/Manifest | Reception of a request for a Segment (or |
| Request | Manifest File) |
+---------------------+---------------------------------------------+
3.2.1.2. Segment-Based Log fields
A Segment-based Log for HTTP adaptive streaming contains all the same
fields as a CDNI Log for a regular HTTP request as well as the
following additional fields:
Le Faucheur, et al. Expires August 26, 2012 [Page 7]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
+-----------------+-------------+-----------------------------------+
| Field | Description | Examples |
+-----------------+-------------+-----------------------------------+
| abr-protocol | Type of | hls,hds,hss |
| | HTTP | |
| | adaptive | |
| | streaming. | |
| Session-id | a string | 6141F5795BE774691D234A0465B9667A |
| | generated | |
| | by the | |
| | delivering | |
| | CDN and | |
| | unique (to | |
| | the | |
| | delivering | |
| | CDN) to | |
| | identify | |
| | the | |
| | Session. | |
| Representation | Characteris | 300000bps |
| | tics of the | |
| | content | |
| | representa | |
| | tion (e.g., | |
| | bitrate). | |
| manifest-uri | URI of the | http://cache3.cdn1.com/movie/ice/ |
| | Manifest | index300k.m3u8 |
| | File. | |
| content-id | Identifier | http://cache3.cdn1.com/movie/ice/ |
| | for the | index.m3u8 |
| | content | |
| | item | |
| | independent | |
| | ly of its | |
| | representa | |
| | tions. | |
| Session-trackin | The method | cookie, uri_query |
| g-method | used in the | |
| | delivering | |
| | CDN for | |
| | tracking of | |
| | the | |
| | Session. | |
+-----------------+-------------+-----------------------------------+
Le Faucheur, et al. Expires August 26, 2012 [Page 8]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
3.2.2. Event-Based Logging
3.2.2.1. Event-Based Log Triggers
An Event-Based Log is generated on the following triggers:
+----------------+--------------------------------------------------+
| Event | Description |
+----------------+--------------------------------------------------+
| Representation | Reception of a request for a Segment whose |
| Change/Bitrate | bitrate is higher than the bitrate of the |
| Upshift. | previously requested Segment. |
| Representation | Reception of a request for a Segment whose |
| Change/Bitrate | bitrate is lower than the bitrate of the |
| Downshift. | previously requested Segment. |
| Representation | TBD |
| Change/TBD. | |
| Session Seek. | Reception of a request for a Segment whose |
| | playtime is not sequentially following the |
| | previously requested Segment. |
| Session Stop. | Determination by delivering CDN that the |
| | End-User has temporarily stopped requesting |
| | Segments associated with the Session. This may |
| | be determined using heuristics such as |
| | expiration of a time-out (e.g. of 10 seconds) |
| | after reception of the last Segment request. |
| Session Close. | Determination by delivering CDN that the |
| | End-User has definitively stopped requesting |
| | Segments associated with the Session. This may |
| | be determined using heuristics such as |
| | expiration of a time-out after any End-USer |
| | activity associated with the Session. |
+----------------+--------------------------------------------------+
3.2.2.2. Event-Based Log Fields
An Event-Based Log for HTTP adaptive streaming contains all the same
fields as a Segment-Based Log as well as the following additional
fields:
+------------+----------------------+-------------------------------+
| Field | Description | Examples |
+------------+----------------------+-------------------------------+
| event-code | Event that triggered | Representation Change/Bitrate |
| | the generation of | Upshift, Session Seek, |
| | the log | Session Stop, ... |
+------------+----------------------+-------------------------------+
Le Faucheur, et al. Expires August 26, 2012 [Page 9]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
3.2.3. Summary-Based Logging
3.2.3.1. Summary-Based Log Triggers
A Summary-Based Log is generated on the following triggers:
+--------------+----------------------------------------------------+
| Event | Description |
+--------------+----------------------------------------------------+
| Session | Determination by delivering CDN that the End-User |
| Close | has definitively stopped requesting Segments |
| | associated with the Session. This may be |
| | determined using heuristics such as expiration of |
| | a time-out after any End-USer activity associated |
| | with the Session. |
| Intermediate | Expiration of a Logging Period Time-out since a |
| Logging | previous Summary-Based Log was generated for that |
| Period | Session (or since the start of the Session). |
+--------------+----------------------------------------------------+
3.2.3.2. Summary-Based Log Fields
An Event-Based Log for HTTP adaptive streaming contains all the same
fields as a Segment-Based Log, minus the Representation field, plus
the following additional fields:
[This will be specified in subsequent versions. The objective is to
define fields that summarize succinctly -while retaining enough
information- the fluctuations in content representation (e.g.
bitrate/quality variations) as well as stop/start/seek patterns over
the duration of the Session (or over the Intermediate Logging period)
for the audio and video components. ]
4. Performance Monitoring
The CDNI Log fields listed in the previous sections allow monitoring
of essential delivery performance indicators across the CDN Mesh.
For example, for a regular HTTP delivery, these fields allow tracking
of the time taken to serve the requested content, tracking of content
delivery failures, tracking of partial deliveries and tracking of
cache hit ratios. As another example, for HTTP adaptive streaming,
these fields allow tracking of the presentation (and its fluctuation
over time) served to the End-User as well as the End-User random
content access (e.g. Play/Stop/Seek).
Subsequent versions of this document will discuss potential
additional log fields for enhanced performance monitoring.
Le Faucheur, et al. Expires August 26, 2012 [Page 10]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
5. CDNI Log Encoding and Transport
Details for CDNI Log encoding and transport will be specified in
subsequent versions. However, Section 5.1 identifies the information
that is to be included in the header of CDNI Logs, Section 5.2
identifies information that is to be attached to every CDNI Log
record and Section 5.3 identifies the information that is to be
included in the footer of CDNI Logs.
5.1. CDNI Log Header Information
The header of CDNI Logs contains the following fields:
+----------------+-------------------+------------------------------+
| Field | Description | Examples |
+----------------+-------------------+------------------------------+
| Format-Version | Version of the | v1.0 |
| | CDNI Log format. | |
| Log-Type | The type of CDNI | Delivery_HTTP, |
| | Log. | Delivery_Adaptive_Segment, |
| | | Delivery_Adaptive_Event |
| Log-ID | Unique identifier | |
| | for the CDNI Log | |
| | (facilitates | |
| | detection of | |
| | duplicate Logs | |
| | and tracking in | |
| | case of | |
| | aggregation). | |
| Log-Timestamp | Time, in | [20/Feb/2012:00:29.510+0200] |
| | milliseconds, the | |
| | CDNI Log was | |
| | generated. | |
+----------------+-------------------+------------------------------+
5.2. CDNI per-Log-Record Information
In addition to the log fields discussed in previous sections, each
CDNI log record contains the following fields:
+----------+---------------------------------------------+----------+
| Field | Description | Examples |
+----------+---------------------------------------------+----------+
| Log | Digest of the Log records (facilitates | |
| Record | recovery of uncorrupted Log records inside | |
| Digest | a corrupted CDNI Log) | |
+----------+---------------------------------------------+----------+
Le Faucheur, et al. Expires August 26, 2012 [Page 11]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
5.3. CDNI Log Footer Information
The footer of CDNI Logs contains the following fields:
+---------+----------------------------------------------+----------+
| Field | Description | Examples |
+---------+----------------------------------------------+----------+
| Log | Digest of the complete Log (facilitates | |
| Digest | detection of Log corruption) | |
+---------+----------------------------------------------+----------+
6. Impact on CDNI Metadata Requirements
We request that the following requirement be added in section 6 of
[I-D.ietf-cdni-requirements]:
"
META-X [HIGH] The CDNI Metadata Distribution interface shall support
an optional mechanism allowing the Upstream CDN to indicate to the
Downstream CDN which CDNI Log format is to be provided for all, for
specific sets of, or for specific content items delivered using HTTP
adaptive streaming.
".
A CDNI implementation that does not support this optional CDNI
Metadata Distribution Interface mechanism MUST ignore this log format
indication and generate CDNI logging format for adaptive streaming
using the default CDNI Logging format for such delivery.
7. Generation of CDNI Logs
Like other CDNI interfaces, the CDNI Logging interface specifies
operations across CDNs and not inside a CDN. Therefore, the log
formats specified in this document apply to CDNI logging information
exchanged across CDNs and does not constrain the process for
generating such inter-CDN logs within a given CDN. The format of the
logs generated by a given CDN Surrogate is beyond the scope of the
present document. We observe that a given CDN could elect to have
its Surrogates natively generate logs in the same format as the one
to be used for exchange with another CDN, or that the CDN could elect
to have its Surrogates generate logs in any other format (as long as
they include the necessary information) and have these logs then
reformatted prior to exchange with another CDN.
Le Faucheur, et al. Expires August 26, 2012 [Page 12]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
8. IANA Considerations
[This will be specified in subsequent versions].
9. Security Considerations
CDNI Logs exchanged over the CDNI Logging interface can be consumed
by very sensitive applications including inter-CDN accounting and
billing. The associated security concerns are discussed in
[I-D.davie-cdni-framework]. At this stage, we observe that the CDNI
Logging interface can leverage the existing security mechanisms
supported by the underlying transport protocol that will be selected
for transport of CDNI Logs (e.g. to support authentication of the
entities exchanging CDNI Logs through the CDNI Logging interface, to
support privacy and protection against spoofing). This will be
further discussed in subsequent versions of this document.
10. Acknowledgements
The authors want to thank Gene Halbrooks for his input into this
document.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
11.2. Informative References
[I-D.bertrand-cdni-logging]
Gilles, B. and S. Emile, "CDNI Logging Interface",
draft-bertrand-cdni-logging-00 (work in progress),
February 2012.
[I-D.davie-cdni-framework]
Davie, B. and L. Peterson, "Framework for CDN
Interconnection", draft-davie-cdni-framework-01 (work in
progress), October 2011.
[I-D.ietf-cdni-problem-statement]
Niven-Jenkins, B., Faucheur, F., and N. Bitar, "Content
Distribution Network Interconnection (CDNI) Problem
Statement", draft-ietf-cdni-problem-statement-03 (work in
Le Faucheur, et al. Expires August 26, 2012 [Page 13]
Internet-Draft CDNI Logging for HTTP Delivery February 2012
progress), January 2012.
[I-D.ietf-cdni-requirements]
Leung, K. and Y. Lee, "Content Distribution Network
Interconnection (CDNI) Requirements",
draft-ietf-cdni-requirements-02 (work in progress),
December 2011.
[I-D.ietf-cdni-use-cases]
Gilles, B., Emile, S., Watson, G., Burbridge, T., Eardley,
P., and K. Ma, "Use Cases for Content Delivery Network
Interconnection", draft-ietf-cdni-use-cases-03 (work in
progress), January 2012.
Authors' Addresses
Francois Le Faucheur
Cisco Systems
Greenside, 400 Avenue de Roumanille
Sophia Antipolis 06410
France
Phone: +33 4 97 23 26 19
Email: flefauch@cisco.com
Mahesh Viveganandhan
Cisco Systems
375 East Tasman Drive
San Jose 95134
USA
Email: mvittal@cisco.com
Kent Leung
Cisco Systems
3625 Cisco Way
San Jose 95134
USA
Phone: +1 408 526 5030
Email: kleung@cisco.com
Le Faucheur, et al. Expires August 26, 2012 [Page 14]