PPSP N. Zong, Ed.
Internet-Draft Huawei Technologies
Intended status: Standards Track June 28, 2009
Expires: December 30, 2009
Chunk Discovery for P2P Streaming
draft-zong-ppsp-chunk-discovery-00
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 30, 2009.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Abstract
This document describes several mechanisms of Chunk Discovery in P2P
Streaming use case. The Chunk Discovery for P2P streaming provides
the functionality of streaming sources registrar and discovery in a
Zong Expires December 30, 2009 [Page 1]
Internet-Draft P2P Streaming Usage June 2009
fully-distributed system. It provides register, update and lookup
service for video chunks in the P2P Streaming applications. The
mechanisms include a P2P streaming usage for REsource LOcation And
Discovery (RELOAD), and a combined Trakcer and Peer Gossip method.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Implementation using Current RELOAD Mechanisms . . . . . . . . 4
3.1. Register Chunk Information . . . . . . . . . . . . . . . . 5
3.2. PEER-REGISTRATION Kind Definition . . . . . . . . . . . . 6
3.3. Lookup Chunk Information . . . . . . . . . . . . . . . . . 7
3.4. Extend PeerRegistrationData . . . . . . . . . . . . . . . 7
4. Combine Data on the Storage Peer . . . . . . . . . . . . . . . 9
4.1. Register Chunk Information . . . . . . . . . . . . . . . . 9
4.2. Combine Data . . . . . . . . . . . . . . . . . . . . . . . 10
4.3. New PEER-REGISTRATION Kind Definition . . . . . . . . . . 10
4.4. Lookup Chunk Information . . . . . . . . . . . . . . . . . 10
4.5. New Requirement to Storage Peer . . . . . . . . . . . . . 11
5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 11
6. A Combined Tracker and Peer Gossip Method . . . . . . . . . . 11
6.1. Tracker Communication . . . . . . . . . . . . . . . . . . 12
6.2. Peer Gossip . . . . . . . . . . . . . . . . . . . . . . . 13
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
8. Security Considerations . . . . . . . . . . . . . . . . . . . 14
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14
10.1. Normative References . . . . . . . . . . . . . . . . . . . 14
10.2. Informative References . . . . . . . . . . . . . . . . . . 14
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 15
Zong Expires December 30, 2009 [Page 2]
Internet-Draft P2P Streaming Usage June 2009
1. Introduction
As one of the architectures for deliverying streaming traffic on the
global Internet , the P2P streaming has the advantage of being
scalable to a large number of subscribers with low infrastructural
cost [PPSP1]. P2P streaming applications attract more and more users
to consume video content online [PPLive][PPStrm][UUSee].
P2P streaming applications adopt a decentralized streaming
architecture where the media content is separated into smaller chunks
to be shared among peers. A chunk is a basic unit of partitioned
streaming, which is used for the purpose of storage and advertising
to neighbors what parts of a movie a peer holds [SIGP2P]. In the
case of VoD, different users watch different parts of the video
content (i.e. have different chunks) so that a peer normally
maintains a buffer to share video chunks with other peers. In the
case of live content, because all peers are mostly interested in what
is actually happens "now", one possible role of a peer is to request
video chunks from a live source and then forward the chunks to more
peers. In all the cases, The procedure of chunk discovery for P2P
streaming is needed to allow peers to register the information of the
video chunks which have been received and ready for sharing with
other peers, as well as locate video chunks from multiple peers in a
fully-distributed manner.
This document describes several mechanisms of Chunk Discovery in P2P
Streaming use case. The Chunk Discovery for P2P streaming provides
the functionality of streaming sources registrar and discovery in a
fully-distributed system. It provides register, update and lookup
service for video chunks in the P2P Streaming applications. The
mechanisms include a P2P streaming usage for REsource LOcation And
Discovery (RELOAD) [I-D.ietf-p2psip-base], and a combined Trakcer and
Peer Gossip method [PPSP1][PPSP2].
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
We use the terminology and definitions from Concepts and Terminology
for Peer to Peer SIP [I-D.ietf-p2psip-concepts] and the RELOAD Base
Protocol [I-D.ietf-p2psip-base].
Zong Expires December 30, 2009 [Page 3]
Internet-Draft P2P Streaming Usage June 2009
3. Implementation using Current RELOAD Mechanisms
The P2P streaming usage for current RELOAD involves the following
functions.
1, Registration. A peer can use the RELOAD data storage
functionality to store a mapping from the Chunk-IDs of the stored
video chunks to its Node-ID.
2, Lookup. A peer can use the RELOAD data storage functionality
to retrieve the Node-IDs of the peers who have registered the
information of the requested video chunks.
For example, Peer A has received video chunks with Video-ID "Movie A"
and Chunk-IDs "Chunk 1", "Chunk 2" and ready for sharing these chunks
with other peers. Peer A could register its Node-ID "1234" under
these Video-ID and Chunk-IDs. Similarly, Peer B has received video
chunks with Video-ID "Movie A" and Chunk-IDs "Chunk 1", "Chunk 2" and
"Chunk 3" and register its Node-ID "5678" under these Video-ID and
Chunk-IDs. When Peer C wants to match "Movie A" from the chunk with
Chunk-ID "Chunk 2", it queries the overlay for "Movie A" and "Chunk
2" and gets back Node-IDs "1234" and "5678". Peer C then uses the
overlay to establish a direct connection with Peer A and Peer B and
can use that direct connection to perform media signaling process.
This example is depicted as follows:
Zong Expires December 30, 2009 [Page 4]
Internet-Draft P2P Streaming Usage June 2009
Peer A Overlay Peers Peer B Peer C
(1234) | (5678) |
| | | |
|StoreReq(MovieA, | | |
| Chunk1/2)-->| | |
|<------------StoreAns| | |
| |<--StoreReq(MovieA, | |
| | Chunk1/2/3)| |
| |StoreAns------------>| |
| | | |
| |<-----------------------FetchReq(MovieA, |
| | | Chunk 2)|
| | FetchAns(1234/5678)-------------------->|
| | | |
| | |<----Attach------->|
|<----------------------------Attach--------------------------->|
| | | |
| | |<----ICE Check---->|
|<----------------------------ICE Check------------------------>|
| | | |
| | |<--Media Signal--->|
|<------------------------Media Signaling---------------------->|
| | | |
| | |-----Media-------->|
|-----------------------------Media---------------------------->|
| | | |
3.1. Register Chunk Information
In conventional C/S streaming cases, the media information (e.g.
media file URL) can be obtained by media clients from the media
server. In P2P streaming usage of RELOAD, each peer registers the
information of the video chunks which have been received and ready
for sharing with other peers, as well as locate video chunks from
multiple peers through the RELOAD overlay. To register its chunk
information, a RELOAD peer stores a PeerRegistrationData structure.
This uses the PEER-REGISTRATION Kind-ID, which is formally defined in
Section 3.2.
As a simple example, if Peer A has obtained the video chunks with
Video-ID "Movie A" and Chunk-IDs "Chunk 1", "Chunk 2" and its Node-ID
is "1234", it may store the mapping "Movie A, Chunk 1/2 -> 1234" in
the RELOAD overlay. This would tell anyone who wanted to watch
"Movie A" from the Chunk with Chunk-ID "Chunk 1" or "Chunk 2" to
contact node "1234".
The contents of a PeerRegistrationData structure are as follows:
Zong Expires December 30, 2009 [Page 5]
Internet-Draft P2P Streaming Usage June 2009
struct {
uint16 destination_id_list_length;
Destination destination_id_list[destination_id_list_length];
} PeerRegistrationData;
The contents of the PeerRegistrationData PDU are:
destination_id_list_length: The number of the peers having
registered the chunk.
destination_id_list: The identifiers of peers who have registered
the chunk.
Note that RELOAD explicitly supports that multiple peers registration
the same combination of Video-ID and Chunk-ID. For example, both
Peer A and Peer B could register "Movie A" and "Chunk 1". Therefore,
the registrations are stored in a Dictionary with dictionary values
being PeerRegistrationData and dictionary keys being Node-IDs. So,
let's say Peer A (with Node-ID "1234") has "Chunk 1" and "Chunk 2" of
"Movie A" and Peer B (with Node-ID "5678") has "Chunk 1", "Chunk 2"
and "Chunk 3" of "Movie A". Peer A would generate
PeerResgistrationData of {1, [1234]}, and PeerB would create
PeerRegistrationData {1, [5678]}. Then they would do the following
stores (registers) into the overlay:
Peer A:
Movie A:Chunk 1 -> 1234, {1, [1234] }
Movie A:Chunk 2 -> 1234, {1, [1234] }
Peer B:
Movie A:Chunk 1 -> 5678, {1, [5678] }
Movie A:Chunk 2 -> 5678, {1, [5678] }
Movie A:Chunk 3 -> 5678, {1, [5678] }
3.2. PEER-REGISTRATION Kind Definition
PEER-REGISTRATION Kind-ID include:
1, Kind IDs. The Resource Name for the PEER-REGISTRATION is the
combination of Video-ID and Chunk-ID. The data stored is a
PeerRegistrationData, which contains the Node-IDs of peers who can
provide the video chunks.
Zong Expires December 30, 2009 [Page 6]
Internet-Draft P2P Streaming Usage June 2009
2, Data Model. The data model for the PEER-REGISTRATION Kind is
dictionary. The dictionary key is the Node-ID of the peer who can
provide the video chunk.
3, Access Control. If certificate-based access control is being
used, stored data of Kind PEER-REGISTRATION must be signed by a
certificate which contains a Node-ID matching the storing
dictionary key.
3.3. Lookup Chunk Information
In P2P streaming usage of RELOAD, when a peer wants to match a video
from certain chunk, it queries the overlay for the corresponding
Video-ID and Chunk-ID and gets back the Node-IDs of the peers who can
provide the video chunks.
As a simple example, if Peer C wants to match "Movie A" from the
chunk with Chunk-ID "Chunk 2", it performs a Fetch for kind PEER-
REGISTRATION at Video-ID "Movie A" and Chunk-ID "Chunk 2". If this
Fetch does not indicate any specific dictionary keys, it will result
in fetching all the stored values, which is PeerRegistrationData as
follows:
Dictionary Results:
1234, {1, [1234] }
5678, {1, [5678] }
Peer C then uses the overlay to establish direct connections with
Peer A and Peer B and can use these direct connections to perform
media signaling process and video transportation.
3.4. Extend PeerRegistrationData
Note that the performance of chunk discovery would be increased by
adding some fields to the PeerRegostrationData structure that lets a
peer list all the chunks it has. That is, if Peer C searches for
"Movie A" and "Chunk 2", it will find not only that both Peer A and
Peer B have "Movie A" and "Chunk 2", but aslo Peer A has "Chunk 1"
and Peer B has "Chunk 1", "Chunk 3". Peer C then would probably
directly request Peer B to serve "Chunk 3" after "Chunk 2", without
asking for the RELOAD to do a new lookup for "Chunk 3".
The contents of the extended PeerRegistrationData structure are as
follows:
Zong Expires December 30, 2009 [Page 7]
Internet-Draft P2P Streaming Usage June 2009
typedef opaque ChunkID[2];
struct {
Destination destination_id;
uint16 chunk_id_list_length;
ChunkID chunk_id_list[chunk_id_list_length];
} DestChunkData
struct {
uint16 destination_id_list_length;
DestChunkData destination_chunk_list[destination_id_list_length];
} PeerRegistrationData;
The contents of the DestChunkData PDU are:
ChunkID: A fixed-length 16-bit string.
chunk_id_list_length: The number of the chunks owned by the peer.
chunk_id_list: The identifiers of chunks owned by the peer.
The contents of the extened PeerRegistrationData PDU are:
destination_id_list_length: The number of the peers having
registered the chunk.
destination_chunk_list: The list of DestChunkData corresponding to
the peers that register the chunk.
Therefore, in the previous example, Peer A and Peer B would do the
following stores (registers) into the overlay:
Peer A:
Movie A:Chunk 1 -> 1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] }
Movie A:Chunk 2 -> 1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] }
Peer B:
Movie A:Chunk 1 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk
3]} ] }
Movie A:Chunk 2 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk
3]} ] }
Movie A:Chunk 3 -> 5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk
3]} ] }
Zong Expires December 30, 2009 [Page 8]
Internet-Draft P2P Streaming Usage June 2009
Similarly, if Peer C wants to match "Movie A" and "Chunk 2", it gets
back the following results:
Dictionary Results:
1234, {1, [ {1234, 2, [Chunk 1, Chunk 2]} ] }
5678, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ] }
4. Combine Data on the Storage Peer
However, if there are many chunks (say more than 1000) in a movie,
then a peer with the whole movie will have to perform many
registrations to list all the things that it has registered. This
would impact the RELOAD overlay as an inefficient way to do the chunk
registration.
4.1. Register Chunk Information
Now we define a new ChunkInfoData structure as follows:
typedef opaque ChunkID[2];
struct {
uint16 chunk_id_list_length;
ChunkID chunk_id_list[chunk_id_list_length];
} ChunkInfoData;
The contents of the ChunkInfoData PDU are:
ChunkID: A fixed-length 16-bit string.
chunk_id_list_length: The number of the chunks owned by the peer.
chunk_id_list: The identifiers of chunks owned by the peer.
Let's reuse the example in Section 3.1. Suppose that Peer A and Peer
B both do the following stores (registers) into the overlay:
Peer A:
Movie A -> 1234, {2, [Chunk1,Chunk2]}
Peer B:
Movie A -> 5678, {3, [Chunk1,Chunk2,Chunk3]}
Zong Expires December 30, 2009 [Page 9]
Internet-Draft P2P Streaming Usage June 2009
It can be observed that the number of registrations performed by
peers is largely reduced.
4.2. Combine Data
The storage peer needs to re-arrage the stored data to:
Movie A -> Chunk1, {2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3,
[Chunk 1, Chunk 2, Chunk 3]} ] }
Movie A -> Chunk2, {2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3,
[Chunk 1, Chunk 2, Chunk 3]} ] }
Movie A -> Chunk3, {1, [ {5678, 3, [Chunk 1, Chunk 2, Chunk 3]} ]
}
So, the registrations are stored in a Dictionary with dictionary
values being PeerRegistrationData and dictionary keys being Chunk-
IDs.
4.3. New PEER-REGISTRATION Kind Definition
The new PEER-REGISTRATION Kind-ID include:
1, Kind IDs. The Resource Name for the new PEER-REGISTRATION is
the Video-ID. The data stored is a PeerRegistrationData, which
contains the Node-IDs of the peers who can provide the video.
2, Data Model. The data model for the new PEER-REGISTRATION Kind
is dictionary. The dictionary key is the Chunk-ID of the chunk
registered by the peers who can provide the video chunk.
3, Access Control. If certificate-based access control is being
used, stored data of the new Kind PEER-REGISTRATION must be signed
by a certificate which contains a Chunk-ID matching the storing
dictionary key.
4.4. Lookup Chunk Information
If Peer C wants to match "Movie A" from the chunk with Chunk-ID
"Chunk 2", it performs a Fetch for kind PEER-REGISTRATION at Video-ID
"Movie A" with dictionary key "Chunk2". It will send back dictionary
results as follows:
Dictionary Results:
{2, [ {1234, 2, [Chunk 1, Chunk 2]}, {5678, 3, [Chunk 1, Chunk 2,
Chunk 3]} ] }
Zong Expires December 30, 2009 [Page 10]
Internet-Draft P2P Streaming Usage June 2009
Peer C then uses the overlay to establish direct connections with
Peer A and Peer B and can use these direct connections to perform
media signaling process and video transportation.
4.5. New Requirement to Storage Peer
It is now obviously required that the storage peer performes the
combining of the chunk information data (with the data structure of
ChunkInfoData) from different peers and convert them to the data
structure of PeerRegistrationData. This capability of combining and
converting data between different data structures would be a new
functionality of RELOAD.
5. Open Issues
Although the P2P streaming usage of RELOAD would be straightforward,
it is still an open question that if the RELOAD is suitable for P2P
streaming.
1, Searching Efficiency. P2P streaming applications normally
require retrieving the real-time/para real-time video chunks
quickly through the direct chunk information exchange among the
peers. On the other hand, iterative and recursive routing process
supported in RELAOD may not meet the real-time requirement of the
P2P streaming [PPSP2].
2, Working Load. P2P streaming applications need constant and
more frequent support of RELOAD. On one hand, during the P2P
streaming process, the requestor would constantly ask for RELOAD
to lookup a new set of peers to serve next chunks of video. This
is either because the current peers left or the requestor is not
sure if the current peers can serve the next chunks of video. On
the other hand, peers also need to constantly advise the chunks
they have to the RELOAD to keep the chunk information updated on
the RELOAD overlay. The need of constant support of RELOAD may
impose a large burden on RELOAD, supposing that there are a large
number (say hundreds of thousands) of simultaneously on-line P2P
streaming peers.
These issue of the performance of RELOAD on supporting P2P streaming
applications need to be evaluated in both PPSP and P2PSIP
communities.
6. A Combined Tracker and Peer Gossip Method
The combined tracker and peer gossip method generally contains
Zong Expires December 30, 2009 [Page 11]
Internet-Draft P2P Streaming Usage June 2009
following components:
1. tracker communications. Peer reports to some centralized
directory server (a.k.a. tracker) coarse information (i.e.
Video-ID). When the tracker receives the video request from a
peer, it returns to the requesting peer with a peer list (or, a
swarm) in which the peers are sharing the requested video.
2. peer gossip. The grain information (i.e. Chunk-ID) is
achieved by the requesting peer through directly communicating
with the peers in the obtained peer list to exchange the chunk
description.
+---------------+
------------>| Tracker |<------------------------
| +---------------+ | |
| | |
| Tracker Communications | |
| | |
V V |
+--------+ Peer Gossip (Chunk Description) +--------+ |
| |<------------------------------->| | |
| Peer | | Peer | |
| |<--------------------- | | |
+--------+ | +--------+ |
Peer Gossip | V
(Chunk Description) | +--------+
| | |
---------------------->| Peer |
| |
+--------+
More Peers
... ...
6.1. Tracker Communication
For example, Peer A is watching a video "Movie A" and ready for
sharing this video with other peers. Peer A could report to the
tracker by sending the Video-ID "Movie A" to the tracker. Similarly,
Peer B is watching "Movie A" and register itself under the Video-ID
"Movie A" to the tracker. Tracker then keep a record that both Peer
A and Peer B would share "Movie A" with other peers. When Peer C
wants to match "Movie A" , it firstly queries the tracker for the
list of peers who would provide the video. It gets back Peer A and
Peer B. Peer C then would communicate with both Peer A and Peer B to
check who would serve the requested chunks of "Movie A". This exaple
Zong Expires December 30, 2009 [Page 12]
Internet-Draft P2P Streaming Usage June 2009
is shown as below:
Peer A racker Peer B Peer C
| | | |
|Report(Movie A)--->| | |
|<---------ReportAns| | |
| | | |
| |<---Report(Movie A)| |
| |ReportAns--------->| |
| | | |
| |<-----------------------Request(Movie A)|
| |RequestAns(Peer A/Peer B)-------------->|
| | | |
6.2. Peer Gossip
For example, even Peer C gets back the peer list including Peer A and
Peer B, it still doesn't know who can serve the requested chunk (say
"Chunk 3"). Therefore, Peer C needs to discover which peer would
serve the request chunk of "Movie A". The chunk information (i.e.
whick chunks are stored on which peers) can be achieved by directly
communicating with both Peer A and Peer B to exchange the chunk
description. Suppose that Peer A stores video chunks with Chunk-ID
"Chunk 1", "Chunk 2" and Peer B stored "Chunk 1", "Chunk 2" and
"Chunk 3". This exaple is illustrated as follows:
Peer A Peer B Peer C
| | |
| <------------------------------------------ChunkDescReq |
| ChunkDescAns(Chunk 1/2)-------------------------------> |
| | |
| | <--------------ChunkDescReq |
| | ChunkDescAns(Chunk 1/2/3)-> |
| | |
| | <-----Media Signaling-----> |
| | |
| | ------Media---------------> |
| | |
Because only Peer B has the chunk with Chunk-ID of "Chunk 3", Peer C
then requests the data of "Chunk 3" from Peer B by establishing a
media connection with Peer B.
7. IANA Considerations
TODO define PEER-REGISTRATION Kind-ID in the case of using RELOAD for
P2P streaming chunk discovery,
Zong Expires December 30, 2009 [Page 13]
Internet-Draft P2P Streaming Usage June 2009
8. Security Considerations
TODO describe security considerations when performing chunk discovery
for P2P streaming.
9. Acknowledgements
The authors would like to thank the following for their contributions
to this document: David Bryan. TODO list more.
10. References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[I-D.ietf-p2psip-base]
Jennings, C., Lowekamp, B., Rescorla, E., Baset, S., and
H. Schulzrinne, "REsource LOcation And Discovery (RELOAD)
Base Protocol", draft-ietf-p2psip-base-02 (work in
progress), March 2009.
[I-D.ietf-p2psip-concepts]
Bryan, D., Matthews, P., Shim, E., Willis, D., and S.
Dawkins, "Concepts and Terminology for Peer to Peer SIP",
draft-ietf-p2psip-concepts-02 (work in progress),
July 2008.
10.2. Informative References
[PPLive] "www.pplive.com".
[PPStrm] "www.ppstream.com".
[UUSee] "www.uusee.com".
[SIGP2P] Huang, "Challenges, Design and Analysis of a Large-scale
P2P-VoD System".
[PPSP1] Zong, N. and X. Jiang, "Survey and Problem Statement of
P2P Streaming", IETF PPSP BoF, November 2008.
[PPSP2] Zhang, Y., Zong, N., Camarillo, G., Seng, J., and R. Yang,
"Problem Statement of P2P Streaming Protocol (PPSP)",
IETF PPSP BoF, May 2009.
Zong Expires December 30, 2009 [Page 14]
Internet-Draft P2P Streaming Usage June 2009
Author's Address
Ning Zong (editor)
Huawei Technologies
10F, Huihong Mansion, No.91 Baixia Road
Nanjing 210001
China
Phone: +86 25 84565866
Email: zongning@huawei.com
Zong Expires December 30, 2009 [Page 15]