Network Working Group                                          O. Boyaci
Internet-Draft                                            H. Schulzrinne
Intended status: Standards Track                             Columbia U.
Expires: April 30, 2009                                 October 27, 2008


         RTP Payload format for Application and Desktop Sharing
                    draft-boyaci-avt-app-sharing-00

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on April 30, 2009.


















Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 1]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


Abstract

   This document defines a protocol to support accessing general
   graphical user interface (GUI) desktops and applications remotely,
   either by a single remote user or embedded into a multiparty
   conference.  The protocol is designed to allow sharing of, and access
   to general windowing system applications that have not been expressly
   written to be accessed remotely.


Table of Contents

   1.  Definitions  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  5
   3.  Requirements Notation  . . . . . . . . . . . . . . . . . . . .  8
   4.  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .  9
     4.1.  Coordinate System  . . . . . . . . . . . . . . . . . . . .  9
     4.2.  Operation  . . . . . . . . . . . . . . . . . . . . . . . . 11
     4.3.  Participants using UDP . . . . . . . . . . . . . . . . . . 13
     4.4.  Participants using TCP . . . . . . . . . . . . . . . . . . 13
     4.5.  Protocol Overview  . . . . . . . . . . . . . . . . . . . . 13
       4.5.1.  Remoting Protocol Overview . . . . . . . . . . . . . . 14
       4.5.2.  Human Interface Protocol (HIP) Overview  . . . . . . . 15
   5.  Remoting Protocol  . . . . . . . . . . . . . . . . . . . . . . 16
     5.1.  Payload Format . . . . . . . . . . . . . . . . . . . . . . 16
       5.1.1.  RTP Header Usage . . . . . . . . . . . . . . . . . . . 16
       5.1.2.  Common remoting/HIP header . . . . . . . . . . . . . . 16
     5.2.  AH-to-participant messages . . . . . . . . . . . . . . . . 17
       5.2.1.  WindowManagerInfo  . . . . . . . . . . . . . . . . . . 17
       5.2.2.  RegionUpdate . . . . . . . . . . . . . . . . . . . . . 19
       5.2.3.  MoveRectangle  . . . . . . . . . . . . . . . . . . . . 21
       5.2.4.  MousePointerInfo . . . . . . . . . . . . . . . . . . . 22
     5.3.  Participant-to-AH Messages . . . . . . . . . . . . . . . . 22
       5.3.1.  Picture Loss Indication (PLI)  . . . . . . . . . . . . 22
       5.3.2.  NACK Request . . . . . . . . . . . . . . . . . . . . . 22
   6.  Human Interface Protocol (HIP) . . . . . . . . . . . . . . . . 23
     6.1.  Payload Format . . . . . . . . . . . . . . . . . . . . . . 23
       6.1.1.  RTP Header Usage . . . . . . . . . . . . . . . . . . . 23
       6.1.2.  Common remoting/HIP header . . . . . . . . . . . . . . 23
     6.2.  MousePressed . . . . . . . . . . . . . . . . . . . . . . . 24
     6.3.  MouseReleased  . . . . . . . . . . . . . . . . . . . . . . 24
     6.4.  MouseMoved . . . . . . . . . . . . . . . . . . . . . . . . 24
     6.5.  MouseWheelMoved  . . . . . . . . . . . . . . . . . . . . . 25
     6.6.  KeyPressed . . . . . . . . . . . . . . . . . . . . . . . . 25
     6.7.  KeyReleased  . . . . . . . . . . . . . . . . . . . . . . . 26
     6.8.  KeyTyped . . . . . . . . . . . . . . . . . . . . . . . . . 26
   7.  Implementation Notes . . . . . . . . . . . . . . . . . . . . . 27
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 28



Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 2]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   9.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 29
     9.1.  Remoting Message Types Subregistry . . . . . . . . . . . . 29
     9.2.  HIP Message Types Subregistry  . . . . . . . . . . . . . . 29
     9.3.  Media Type Registrations . . . . . . . . . . . . . . . . . 30
       9.3.1.  Registrations of Media Type application/remoting . . . 30
       9.3.2.  Registrations of Media Type application/hip  . . . . . 31
   10. Mapping to the Session Description Protocol (SDP)  . . . . . . 33
     10.1. Mapping remoting Media Type Parameters into SDP  . . . . . 33
     10.2. Mapping hip Media Type Parameters into SDP . . . . . . . . 33
     10.3. SDP Example  . . . . . . . . . . . . . . . . . . . . . . . 33
   Appendix A.  Using BFCP for Application and Desktop Sharing  . . . 35
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 36
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 36
     11.2. Informative References . . . . . . . . . . . . . . . . . . 37
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 38
   Intellectual Property and Copyright Statements . . . . . . . . . . 39



































Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 3]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


1.  Definitions

   Application Host (AH):  An application host (AH) is the computer
      which runs the shared application, distributes the screen updates
      to the participants, and regenerates human interface events
      received from participants.

   Participant:  Participant is the computer which receives screen
      updates from AH and sends human interface events back to the AH.
      Participants do not need to store or run the shared application.
      More than one participant may connect to a single AH.

   Remoting Protocol:  Remoting protocol messages allows the AH to
      distribute windowing information and screen updates to
      participants.

   Human Interface Protocol (HIP):  Human Interface Protocol (HIP)
      allows participants to send human interface device (HID) events to
      AH.  HIDs generates mouse or keyboard events such as a key press,
      key release, mouse move, and mouse click.































Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 4]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


2.  Introduction

   Application and desktop sharing allows sharing of any software
   application with one or more participants over the Internet.  The
   application host (AH) is the computer which runs the shared
   application.  The participants receive the screen-view of the shared
   application from the AH.  They do not need to run or store the shared
   application.  Their mouse and keyboard events are delivered and
   regenerated at the AH.  An Application and desktop sharing system
   adheres to a client-server architecture [Figure 1].

                                      +-------------+
                                      |             |
                                      | participant |
                                      |             |
                                      +-------------+
                                          ^   |
                                          |   |
                          Screen updates  |   | HIP
                                          |   |
                                          |   |
                                          |   v
      +-------------+  Screen updates  +---------+
      |             |<-----------------|         |
      | participant |                  |   AH    |
      |             |----------------->|         |
      +-------------+        HIP       +---------+


             Figure 1: Application sharing system architecture

   Application and desktop sharing enables collaborative work, software
   tutoring, and e-learning over the Internet.  While two-party and
   multi-party conferencing using standards-based protocols is now
   common and well-developed, protocols for sharing applications are
   largely proprietary or based on the aging T.120 [T120] suite of
   protocols.  In this document, we define an RTP payload format for
   application and desktop sharing.

   Application sharing differs from desktop sharing.  In desktop
   sharing, a computer distributes all screen updates.  In application
   sharing, the AH distributes screen updates if and only if they belong
   to the shared application's windows.  Applications often consist of a
   changing set of related windows which serve the same task and are
   usually associated with the same process on the AH.  Considering only
   the boundary of the shared windows is not enough.  Other non-shared
   windows may cover the shared window or shared application may open
   new child windows such as those for selecting options or fonts.  A



Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 5]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   true application sharing system must blank all the nonshared windows
   and must transfer all the child windows of the shared application.

   We note that remote access to an application ("remote desktop") and
   multiple users sharing an application within a collaboration setting
   such as a multimedia call or multiparty conference are very similar.
   The protocols defined in this document therefore support both.

   Remote access differs from video transmission of the sort for which
   most video encodings have been designed.  In particular, screen
   encoding may need to be lossless and typically operates on artificial
   rather than natural (photographic) video input.  The video input is
   characterized by large areas of the screen that remain unchanged for
   long periods of time, while others change rapidly.  (However,
   rendering the output of a modern computer-generated animation
   application such as video games blurs the distinction between
   traditional motion video output and screen sharing.)

   Application sharing can be integrated into the existing IETF session
   model, encompassing session descriptions using the Session
   Description Protocol (SDP) [RFC2327] or successors and the Session
   Initiation Protocol (SIP) [RFC3261].  Application sharing needs many
   of the same control functions as other multimedia sessions, such as
   address binding, session feature and media negotiation.  The session
   model is also beneficial for the remote desktop case, as it allows to
   re-use many of the well-developed session components and easily
   supports hybrid multimedia models, such as the delivery of desktop
   audio to the remote user.

   Remote access to graphical applications and desktops, as defined in
   this document, has two important characteristics.  First, the access
   protocol is unaware of any semantic characteristics of the
   applications being shared; it only transmits the visual
   characteristics of the windows.  This is different, therefore, from
   shared-drawing or shared-editing tools that allow distributed
   modification of documents.  Secondly, the protocol is designed to
   work with applications which were not written to be used remotely, by
   intercepting or simulating their connections to their native window
   systems.  That distinguishes it from systems such as the X Window
   System [X] which allow natively-written applications to be displayed
   on remote viewers.

   We distinguish between the AH user and participants.  The AH user
   interacts with the application using normal operating system
   mechanisms.  Participants interact via the delivery protocols
   described here.

   The application sharing problem can be divided into four components:



Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 6]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   (1) setting up a session to the AH, (2) transporting human interface
   events from the participants to the AH, (3) delivering screen output
   from the AH to the participants, (4) moderating access to shared
   human interface devices such as pointing devices (e.g., mouse,
   joystick, trackball) and text input (keyboard).  We refer to
   component (2) as the "human interface protocol (HIP)" and component
   (3) as the "remoting protocol".  Remote user input access is
   moderated by the Binary Floor Control Protocol (BFCP) [RFC4582].

   Session negotiation and description can be provided by existing
   session setup protocols.  Thus, they are beyond the scope of this
   document.

   The rest of this document is laid out as follows.  Section 3 defines
   the common terminology for normative references.  Section 4 gives an
   overview of the protocol's architecture and components.  The remoting
   protocol and HIP are described in Section 5 and 6, respectively.
   Section 8 gives implementation notes, and Section 9 discusses
   security considerations.
































Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 7]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


3.  Requirements Notation

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].














































Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 8]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


4.  Overview

4.1.  Coordinate System

   The origin (0,0) of the coordinate system is the upper left corner.
   All coordinates carried in the protocol messages are absolute and
   measured in pixels.

   0,0                  <- x axis ->
    +---------------------------------------------------+
    |                                                   |
    |    220,150                                        |
    |       +------  A -------+                         |
    |       |                 |        850,320          |
    |       |                 |           +--  C ---+   |
    |       |      450,400    |           |         |   |
    |       |         +---------  B ----+ |         |   |
    |       |         |                 | +---------+   |
    |       |         |                 |               |
    |       +---------|                 |               |
    |                 |                 |               |
    |                 +-----------------+               |
    |                                800,700            |
    |                                                   |
    |                                                   |
    +---------------------------------------------------+
                             AH                     1280,1024

                   Figure 2: An AH shares three windows






















Boyaci & Schulzrinne     Expires April 30, 2009                 [Page 9]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   0,0
    +------------------------------------------------+
    |                                                |
    |    220,150                                     |
    |       +------  A -------+                      |
    |       |                 |        850,320       |
    |       |                 |           +--  C ---+|
    |       |      450,400    |           |         ||
    |       |         +---------  B ----+ |         ||
    |       |         |                 | +---------+|
    |       |         |                 |            |
    |       +---------|                 |            |
    |                 |                 |            |
    |                 +-----------------+            |
    |                                800,700         |
    +------------------------------------------------+
                       Participant-1             1024,768

   Figure 3: Participant 1 displays the shared windows in their original
                                coordinates


   0,0
    +------  A -------+---------------------------------+
    |                 |        630,170                  |
    |                 |           +--  C ---+           |
    |      230,250    |           |         |           |
    |         +---------  B ----+ |         |           |
    |         |                 | +---------+           |
    |         |                 |                       |
    +---------|                 |                       |
    |         |                 |                       |
    |         +-----------------+                       |
    |                         580,550                   |
    |                                                   |
    |                                                   |
    |                                                   |
    |                                                   |
    +---------------------------------------------------+
                     Participant-2                 1280,1024

      Figure 4: Participant 2 displays the shared windows in shifted
                                coordinates








Boyaci & Schulzrinne     Expires April 30, 2009                [Page 10]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   0,0
    +------  A -------+----------+
    |                 |          |
    |  60,180      +--  C ---+   |
    |    +---------  B ----+ |   |
    |    |                 | |   |
    |    |                 |-+   |
    |    |                 |     |
    +----|                 |     |
    +----+-----------------+-----+
             Participant-3    640,480

     Figure 5: Participant 3 displays the shared windows in completely
                           different coordinates

   Figure 2 shows an example scenario where three windows are shared.
   All coordinates are absolute.  A participant can display the windows
   in their original coordinates or it can display them in different
   coordinates.  In Figure 3, participant 1 displays the windows in
   their original coordinates.  Participant 2 shifts all the windows 220
   pixels left and 150 pixels up Figure 4.  Participant 2 preserves the
   relations between windows, while participant 3 combines all the
   windows in order to fit them to its small screen Figure 5.  In this
   example scenario, all participants preserve the z-order of windows.
   The AH informs the participants about windows' positions and sizes,
   z-order, and their groupings.  The AH MAY assign same group
   identifier to the windows which belongs to the same process.
   Grouping information MAY be used by the participant while relocating
   the windows or enforcing the z-order.  A participant MAY allow
   changing the z-order (i.e., stacking order) of windows locally,
   without changing the z-order in the AH.  Several remoting/HIP message
   types carries left, top, width and height fields.  The units of these
   fields are in pixels and they are unsigned.

   The AH MUST only accept legitimate HIP events by checking whether the
   requested coordinates are inside the shared windows.

4.2.  Operation

   Detecting a change in the GUI of the shared application, the AH
   prepares an RTP [RFC3550] packet containing an encoded image of the
   updated region.  RTP allows the participants to re-order the packets,
   recognize missing packets and synchronize application sharing with
   other media types like audio and video.  The screen updates can be
   encoded with PNG [I-D.boyaci-avt-png], JPEG [RFC2435], JPEG 2000
   [RFC5371], Theora [theora] or other media types like H.264 [RFC3984],
   according to their characteristics.  PNG is an open image format
   which uses a lossless compression algorithm and more suitable for



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 11]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   computer generated images.  JPEG is lossy, but more suitable for
   photographic images.  JPEG 2000 supports both lossless and lossy
   compression, therefore suitable for both computer generated images
   and movies.  Theora is an open source video codec comparable to H.264
   and suitable for movie encoding.

   Although multiple users could receive the screen updates
   simultaneously, clearly only one of them can manipulate the
   application via keyboard and mouse events.  The Binary Floor Control
   Protocol (BFCP) [RFC4582] MAY be used to restrict the control of the
   application to a single user.  BFCP receives floor request and floor
   release messages from participants; and then it grants the floor to
   the appropriate participant for a period of time while keeping the
   requests from other participants in a FIFO queue.  The details of
   utilizing BFCP in the context of application and desktop sharing are
   given in Appendix A.

   The protocol supports two different mouse pointer models.  Mouse
   pointer images can be transmitted as RegionUpdate messages or they
   may be transmitted seperately as MousePointerInfo messages.  The AH
   decides which mouse model to use.  The participants MUST support both
   mouse models.

   HIP supports both UTF-8 encoded unicode characters and other keyboard
   keys which are not defined in unicode such as function and control
   keys.  For keyboard events publicly available Java virtual key codes
   [keycodes] are used.

   The application and desktop sharing models defined in this document
   can be integrated into the IETF conferencing model.  The Session
   Initiation Protocol (SIP) [9] can be used to intiate and control
   remote access.  This allows the use of existing SIP mechanisms for
   confidentiality, authentication and authorization, user location,
   conferencing.

   Additional, optional mechanisms can enhance application and desktop
   sharing.  Audio streams can be associated with a desktop or
   application; participant-side scaling can be used to optimize
   transmission of data to participants with a small screen; and it is
   often useful to allow copy-and-paste between applications running on
   a participant and those running on an AH.  This document does not
   define any such extensions.

   The AH can support both multicast and unicast transmissions.  For
   unicast connections, either UDP or TCP can be used.  The AH can share
   an application to TCP participants, UDP participants, and several
   multicast addresses in the same sharing session.




Boyaci & Schulzrinne     Expires April 30, 2009                [Page 12]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


4.3.  Participants using UDP

   The AH controls the transmission rate for participants using UDP,
   because UDP itself does not provide flow and congestion control.
   Several simultaneous multicast sessions with different transmission
   rates can be created at the AH.

   Participants can join a sharing session anytime, and they need the
   shared windows' information and full screen buffer before receiving
   partial updates.  Therefore, participants using UDP send an RCTP-
   based feedback message, Picture Loss Indication (PLI) [RFC4585],
   after joining the session.  The AH prepares and transmits the
   windows' state information and image of the whole shared region after
   receiving a PLI message.

4.4.  Participants using TCP

   Since TCP provides reliable communication and flow control, it is
   more suitable for unicast sessions.  TCP participants may have
   different bandwidths, so an algorithm which sends the updates at the
   link speed of each participant is needed.

   Neither TCP nor RTP declares the length of an RTP packet.  Therefore,
   RTP framing [RFC4571] is used to split RTP packets within the TCP
   byte stream.

   The AH prepares and transmits the windows' state information and
   image of the whole shared region to the new participant, right after
   the TCP connection establishment.

4.5.  Protocol Overview

   Application and desktop sharing protocol consists of two
   subprotocols: remoting and human interface protocol (HIP).  Remoting
   messages transmit screen updates from AH to participants.  HIP
   messages transmit mouse and keyboard events from the participant to
   the AH.

   Remoting and HIP messages are RTP messages.  They consist of an RTP
   header, common remoting/HIP header, message-type specific header, and
   message payload [Figure 6].  The HIP messages have a different
   payload type than the remoting messages.









Boyaci & Schulzrinne     Expires April 30, 2009                [Page 13]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                          RTP header                           .
   .                                                               .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Common remoting/HIP header                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                  Message-type specific header                 .
   .                                                               .
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                                                               .
   .                    Message Specific Payload                   .
   .                                               +-+-+-+-+-+-+-+-+
   .                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 6: Application sharing protocol message structure

4.5.1.  Remoting Protocol Overview

   The remoting protocol consists of four messages from the AH to the
   participant and two control messages from participant to AH.  The AH-
   to-participant messages are WindowStateInfo, RegionUpdate,
   MoveRectangle, and MousePointerInfo.  The RTCP messages from UDP-
   based participant to AH are "Picture loss indication (PLI)" and "NACK
   request".  Participants MUST implement all protocol messages
   described in this document.

   The WindowManagerInfo message informs the participants about the
   windowIDs of the windows, their positions and sizes, z-order, and
   their groupings.  All remoting messages carry the windowID to
   identify the target of message.  For TCP participants, the AH
   transmits WindowManagerInfo message right after establishing a
   connection.  UDP participants send a "Picture loss indication (PLI)"
   to the AH as soon as they join the session.  Receiving this PLI
   message, the AH transmits WindowManagerInfo message.  The AH
   transmits RegionUpdate messages for updated regions.  Whenever the
   shared window resizes or relocates, the AH sends a WindowManagerInfo
   message.  Similarly, if the z-order of windows changes, the AH send a
   WindowManagerInfo message.  MoveRectangle instructs the participant
   to move a region from one place to another, which is efficient for
   some drawing operations like scrolls.  The MousePointerInfo message
   transmits the position and icon of the mouse pointer.  Some AHs may
   transmit pointer images inside the RegionUpdate messages, so they may



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 14]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   not need MousePointerInfo message.

   "Picture loss indication (PLI)" and "NACK request" are control
   messages and they are transmitted as RTCP messages.  The "NACK
   request" is used only by UDP participants to request retransmission
   of missing packets from the AH.  AHs MAY support retransmissions.
   PLI can be used by both UDP and TCP participants to request a full
   screen refresh.

4.5.2.  Human Interface Protocol (HIP) Overview

   HIP consist of seven messages: namely, Mouse Pressed, Mouse Released,
   Mouse Moved, Mouse Wheel Moved, Key Pressed, Key Released and Key
   Typed.  These messages are all from AH to participant and carried as
   RTP messages.  However, these HIP messages have different payload
   type than the remoting messages.



































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 15]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


5.  Remoting Protocol

5.1.  Payload Format

5.1.1.  RTP Header Usage

   Marker bit:  The marker bit indicates the last packet of a multi-
      packet RegionUpdate (Section 5.2.2 message.  The marker bit allows
      the receiver to finish decoding the picture, without waiting for
      the next packet with a new timestamp.  Unless defined otherwise,
      all other message types MUST set this bit to zero

   Timestamp:  The RTP timestamp indicates the time instance the
      remoting message has been created at the AH.  The RTP timestamp is
      based on a 90-kHz clock.  If a RegionUpdate message occupies more
      than one packet, the timestamp SHALL be the same for all of those
      packets.  Furthermore, the initial value of the timestamp MUST be
      random (unpredictable) to make known-plaintext attacks more
      difficult [RFC3550].

   The remaining RTP header fields are used as specified in RFC 3550.

5.1.2.  Common remoting/HIP header

   All remoting protocol messages carry a common remoting/HIP header
   [Figure 7] which follows the RTP header.  Message type and parameter
   fields are 8 bit identifiers, whereas the windowID is a 16-bit
   identifier.  The windowID field is unsigned and has a range of
   0-65535.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Msg Type     |    Parameter  |          WindowID             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 7: Common remoting/HIP header

   Table 1 enumerates the message types defined in this document.
   Participants MUST implement all of them.  Section 9 describes how
   additional message types can be registered with IANA.  Participants
   MAY ignore such additional message types.









Boyaci & Schulzrinne     Expires April 30, 2009                [Page 16]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


                       +-------+-------------------+
                       | Value | Message Type      |
                       +-------+-------------------+
                       |   1   | WindowManagerInfo |
                       |       |                   |
                       |   2   | RegionUpdate      |
                       |       |                   |
                       |   3   | MoveRectangle     |
                       |       |                   |
                       |   4   | MousePointerInfo  |
                       +-------+-------------------+

                 Table 1: Remoting protocol message types

5.2.  AH-to-participant messages

5.2.1.  WindowManagerInfo

   The WindowManagerInfo message informs the participants about windows,
   their positions and sizes, z-order, and their groupings.  This
   message transfers the complete window manager state to the
   participants.  Each shared window resize and relocation in any
   coordinate triggers a WindowmangerInfo message.  Parameter and
   WindowID fields of common remoting/HIP header MUST be ignored.  This
   message carries a message specific payload.  One or more window
   records [Figure 8] follow the common remoting/HIP header.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |            WindowID           |   GroupID     |   Reserved    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Width                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Height                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                         Figure 8: A Window Record

   Each window record is 20-bytes.  The z-order information is given
   implicitly to the participants.  The first record describes the
   window at the bottom of the stacking order, the last record the one
   on top.  The "left" and "Top" fields carries the upper-left
   coordinate of the window.  The "Width" and "Height" fields carries



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 17]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   the width and the height of the window, respectively.  Each window is
   assigned a WindowID.  The participant MUST create a window for each
   new WindowID and MUST close this window after receiving a
   WindowManagerInfo message which does not contain this WindowID.
   GroupID fields informs the participant about grouping of windows.
   The AH MAY assign same GroupID to the windows which belongs to the
   same process.  Grouping information MAY be used by the participant
   while relocating the windows or enforcing the z-order.  The value of
   "0" for GroupID field is reserved and represents no grouping for
   given window.

   Figure 9 is an example WindowManagerInfo message for the three shared
   windows in Figure 2.  The participant MUST keep the existing window
   image after a resize and relocation.





































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 18]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Msg Type = 1  | Parameter = 0 |          WindowID = 0         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          WindowID = 1         |  GroupID = 1  |  Reserved = 0 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Left = 220                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Top = 150                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Width = 350                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Height = 450                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         WindowID = 2          |  GroupID = 2  |  Reserved = 0 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Left = 850                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Top = 320                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Width = 160                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Height = 150                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         WindowID = 3          |  GroupID = 1  |  Reserved = 0 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Left = 450                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Top = 400                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Width = 350                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Height = 300                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Figure 9: An example WindowManagerInfo message with three Window
                                  Records

5.2.2.  RegionUpdate

   The RegionUpdate message instructs the participant to update the
   specified region of a window with new content.  This message carries
   a message-type specific header and payload.  This protocol supports
   all media types which have an RTP payload specification.  It is
   possible that AH or participant may support only some media types.
   Therefore, they should negotiate supported media types during the
   session establishment.  The 8 bit "parameter" field of the common



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 19]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   remoting/HIP header will carry both the FirstPacket bit and the
   actual payload type of the content.  The 7 bit PT field carries the
   actual payload type of the content which can be PNG, JPEG, Theora, or
   any other media type which has an RTP payload specification.  All AH
   and participant software implementations MUST support PNG images
   [I-D.boyaci-avt-png].  The message-type specific header follows
   common remoting/HIP header.  Message-type specific header consists of
   two 32 bit parameters, left and top.  These two parameters informs
   the participants about the left-top coordinate of the RegionUpdate.
   The width and height of the RegionUpdate is not transmitted
   explicitly by this protocol.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | RegionUpdate  |F|      PT     |          WindowID             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 10: Common remoting/HIP header for RegionUpdate messages

   If the content of the update does not fit into a single RTP message,
   it will be carried in several RTP payloads.  All the payloads will
   carry the 32 bit common remoting/HIP header, while left and top
   fields are carried only in the first RTP payload.  The marker bit and
   FirstPacket bit informs the particapants about the fragmentation.

         +------------+-----------------+-----------------------+
         | Marker bit | FirstPacket bit | Fragment Type         |
         +------------+-----------------+-----------------------+
         |      1     |        1        | Not Fragmented        |
         |            |                 |                       |
         |      0     |        1        | Start Fragment        |
         |            |                 |                       |
         |      0     |        0        | Continuation Fragment |
         |            |                 |                       |
         |      1     |        0        | End Fragment          |
         +------------+-----------------+-----------------------+

       Table 2: Marker and FirstPacket bits carry fragmentation info

   Figure 11 displays an example RegionUpdate message.  The RTP header
   is omitted in Figure 11.  The RegionUpdate is non fragmented,
   therefore both the marker bit in the RTP header and FirstPacket bit
   in the payload header is set to 1.







Boyaci & Schulzrinne     Expires April 30, 2009                [Page 20]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Msg Type = 2 |1|      PT     |          WindowID = 1         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                                                               .
   .                            Payload                            .
   .                                               +-+-+-+-+-+-+-+-+
   .                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 11: An example non fragmented RegionUpdate message

5.2.3.  MoveRectangle

   The MoveRectangle message instructs the participant to move the
   specified region of a window to a new position.  This message carries
   a message-type specific header.  The AH informs the participants
   about the source rectangle via source left, source top, width and
   height parameters.  Participants learns the destination coordinates
   from destination left and destination top parameters.  Source and
   destination rectangles may overlap.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Source Left                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Source Top                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Width                                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Heigth                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Destination Left                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Destination Top                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

          Figure 12: Message specific payload for move rectangle






Boyaci & Schulzrinne     Expires April 30, 2009                [Page 21]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


5.2.4.  MousePointerInfo

   Some AHs MAY include the mouse pointer image inside the RegionUpdate
   messages.  However, some AHs MAY choose to inform the participant
   about the mouse position and icon explicitly.  If the RegionUpdate
   messages contain the mouse pointer icon, then the MousePointerInfo
   message is unnecessary.  When receiving this message, the participant
   should draw the mouse pointer to the given position.  This message
   carries a message specific payload.  The format of this message is
   same as RegionUpdate message Section 5.2.2 except they have different
   message types.  The payload of MousePointerInfo message can be only
   the left and top coordinates.  In this case, the participant MUST
   move the existing pointer image to the given coordinates.  Payload
   MAY carry both the left and top coordinates and the new image of the
   mouse pointer.  The participant MUST store and use this image until a
   new image arrives from the AH.  If the AH uses MousePointerInfo
   messages, it MUST inform the late joiners about the current position
   and image of mouse pointer.

5.3.  Participant-to-AH Messages

   Participants using UDP can send two RTCP messages to the AH.  Late-
   joiners MAY inform the AH using the "Picture loss indication (PLI)"
   message in order to receive a full screen update.  For the missing
   packets, UDP participants MAY send a "NACK Request".

5.3.1.  Picture Loss Indication (PLI)

   "Picture Loss Indication (PLI)" message instructs the AH to generate
   a full screen update of the shared region.  Before the full screen
   update, the AH will send a WindowManagerInfo message to inform the
   new participant about windows.  Both TCP and UDP participants MAY
   transmit this message.  The message format conforms to the "Picture
   Loss Indication (PLI)" section 6.3.1 of [RFC4585].

5.3.2.  NACK Request

   "NACK Request" message informs the AH about missing RTP packets.  The
   message format conforms to the "Generic NACK" section 6.2.1 of
   [RFC4585].  Multicast participants and AHs MAY take necessary
   precautions to prevent NACK storms such as waiting random amount of
   time before sending a "NACK Request" message.









Boyaci & Schulzrinne     Expires April 30, 2009                [Page 22]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


6.  Human Interface Protocol (HIP)

   Participants MAY send human interface events to the AH in order to
   interact with the shared application.

6.1.  Payload Format

6.1.1.  RTP Header Usage

   Marker Bit:  The marker bit MUST be set to zero by the participant
      and ignored by the AH.

   Timestamp:  The RTP timestamp indicates when the keyboard or mouse
      event occurred at the participant.  The RTP timestamp of HIP
      messages is based on a 90-kHz clock.  The initial value of the
      timestamp MUST be random (unpredictable) to make known-plaintext
      attacks on encryption more difficult; see RTP [RFC3550].

   The remaining RTP header fields are used as specified in RFC 3550.

6.1.2.  Common remoting/HIP header

   All HIP messages carry the same common remoting/HIP header shown in
   Figure 7 and discussed in Section 5.1.2.  The WindowID parameter
   indicates the window where the keyboard or mouse event took place,
   i.e., the window that had keyboard or mouse focus.

   The following HIP message types are defined:

                        +-------+-----------------+
                        | Value | Message Type    |
                        +-------+-----------------+
                        |  121  | MousePressed    |
                        |       |                 |
                        |  122  | MouseReleased   |
                        |       |                 |
                        |  123  | MouseMoved      |
                        |       |                 |
                        |  124  | MouseWheelMoved |
                        |       |                 |
                        |  125  | KeyPressed      |
                        |       |                 |
                        |  126  | KeyReleased     |
                        |       |                 |
                        |  127  | KeyTyped        |
                        +-------+-----------------+

                        Table 3: HIP Message Types



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 23]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


6.2.  MousePressed

   The MousePressed message instructs the AH to generate a mouse pressed
   event at the given coordinates of the screen.  This message carries a
   message-type specific header.  The "parameter" field of the common
   remoting/HIP header carries the mouse button information.  The values
   of 1, 2 and 3 are defined for left, right, and middle button,
   respectively.  The AH and participant MAY negotiate additional values
   for other mouse buttons.  The AH MAY ignore unrecognized values.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 13: Message specific payload for mouse pressed

6.3.  MouseReleased

   The MouseReleased message instructs the AH to generate a mouse
   released event at the given coordinates of the screen.  This message
   carries a message-type specific header.  The "parameter" field of the
   common remoting/HIP header carries the mouse button information.  The
   values of 1, 2 and 3 are defined for left, right, and middle button,
   respectively.  Other values MAY be defined for other mouse buttons.
   The AH MAY ignore unrecognized values.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

          Figure 14: Message specific payload for mouse released

6.4.  MouseMoved

   The MouseMoved message instructs the AH to move the mouse pointer to
   the coordinates provided.  This message carries a message-type
   specific header.






Boyaci & Schulzrinne     Expires April 30, 2009                [Page 24]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 15: Message specific payload for mouse moved

6.5.  MouseWheelMoved

   The MouseWheelMoved message instructs the AH to generate a mouse
   wheel moved event at given coordinates of the screen.  This message
   carries a message-type specific header.  The "distance" field carries
   the wheel rotation amount as "120 * (number of notches)".  A mouse
   wheel has discrete, evenly spaced notches.  When user rotates the
   wheel, a wheel message is sent to OS as each notch is encountered.
   The "distance" field does not carry number of notches in order to
   support a smooth-scrolling wheel.  Instead, "distance" field carries
   each notch as 120.  Some mice MAY only send multiples of 120, while a
   smooth-scrolling mouse MAY send any values.  A positive value
   indicates that the wheel was rotated forward, away from the user; a
   negative value indicates that the wheel was rotated backward, toward
   the user.  The negative values are transmitted using 2's complement
   method.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Left                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Top                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Distance                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 16: Message specific payload for mouse wheel moved

6.6.  KeyPressed

   The KeyPressed message instructs the AH to generate a "key pressed"
   event.  This message carries a message-type specific header which
   consists of a 32 bit KeyCode.  Java virtual keycodes are used and
   they are publicly available on the openJDK website [keycodes].  The
   actual values are inside the KeyEvent.java file.  For example, F1 key
   is defined as "int VK_F1 = 0x70;" in KeyEvent.java.




Boyaci & Schulzrinne     Expires April 30, 2009                [Page 25]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          KeyCode                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 17: Message specific payload for key pressed

6.7.  KeyReleased

   The KeyReleased message instructs the AH to generate a "key released"
   event.  This message carries a message-type specific header which
   consists of a 32 bit KeyCode.  Java keycodes are used and they are
   publicly available at openJDK website [keycodes].  The actual values
   are inside the KeyEvent.java file.  For example, F1 key is defined as
   "int VK_F1 = 0x70;" in KeyEvent.java.  A KeyReleased event for a key
   without a prior KeyPressed event for this key is acceptable.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          KeyCode                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 18: Message specific payload for key released

6.8.  KeyTyped

   KeyTyped message instructs the AH to inject some number of UTF-8
   encoded characters into the operating systems input queue.  This
   message carries a message specific payload.  There is no padding for
   the UTF-8 string.  The participant MUST send more than one KeyTyped
   message if the string does not fit into a single KeyTyped packet.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                                                               .
   .                         UTF-8 String                          .
   .                                               +-+-+-+-+-+-+-+-+
   .                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 19: Message specific payload for key released






Boyaci & Schulzrinne     Expires April 30, 2009                [Page 26]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


7.  Implementation Notes

   Application hosts shouldn't blindly send every screen update they
   observed to the participants.  Instead, they should monitor the state
   of their TCP transmission buffers (through mechanisms such as the
   select() command) and only send the most recent screen data when
   there is no backlog.  This will prevent screen latency for rapidly-
   changing images, when a viewer usually only needs to see the final
   state of the image.










































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 27]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


8.  Security Considerations

   RTP packets using the payload format defined in this specification
   are subject to the security considerations discussed in the RTP
   specification [RFC3550].

   Application sharing inherently exposes the shared applications to
   risks by malicious participants.  They may, for example, access
   resources beyond the application itself, e.g., by installing or
   running scripts.  It may be difficult to constrain access to specific
   user data, e.g., a specific set of slides, unless the user
   application can be sandboxed or run in some kind of "jail", with the
   sandbox control outside the view of the remoting protocol.






































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 28]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


9.  IANA Considerations

   The IANA has created a new registry for Application and Desktop
   Sharing Parameters called "Application and Desktop Sharing
   parameters".  This new registry has a number of subregistries which
   are described in the following sections.

9.1.  Remoting Message Types Subregistry

   This section establishes the Remoting Message Types subregistry under
   the Application and Desktop Sharing Parameters registry.  As per the
   terminology in RFC 2434 [RFC2434], the registration policy for
   Remoting Message Types shall be "Specification Required".  For the
   purposes of this subregistry, the Remoting Message Types for which
   IANA registration is requested MUST be defined by a standards-track
   RFC.  Such an RFC MUST specify the Remoting Message Type's value,
   name, format, and semantics.

   For each Remoting Message Type, the IANA registers its value, its
   name, and the reference to the RFC where the Remoting Message Type is
   defined.  The following table contains the initial values of this
   subregistry.

                +-------+-------------------+------------+
                | Value | Message Type      | Reference  |
                +-------+-------------------+------------+
                |   1   | WindowManagerInfo | [RFC nnnn] |
                |       |                   |            |
                |   2   | RegionUpdate      | [RFC nnnn] |
                |       |                   |            |
                |   3   | MoveRectangle     | [RFC nnnn] |
                |       |                   |            |
                |   4   | MousePointerInfo  | [RFC nnnn] |
                +-------+-------------------+------------+

     Table 4: Initial values of the Remoting Message Type subregistry

9.2.  HIP Message Types Subregistry

   This section establishes the HIP Message Types subregistry under the
   Application and Desktop Sharing Parameters registry.  As per the
   terminology in RFC 2434 [RFC2434], the registration policy for HIP
   Message Types shall be "Specification Required".  For the purposes of
   this subregistry, the HIP Message Types for which IANA registration
   is requested MUST be defined by a standards-track RFC.  Such an RFC
   MUST specify the HIP Message Type's value, name, format, and
   semantics.




Boyaci & Schulzrinne     Expires April 30, 2009                [Page 29]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   For each HIP Message Type, the IANA registers its value, its name,
   and the reference to the RFC where the HIP Message Type is defined.
   The following table contains the initial values of this subregistry.

                 +-------+-----------------+------------+
                 | Value | Message Type    | Reference  |
                 +-------+-----------------+------------+
                 |  121  | MousePressed    | [RFC nnnn] |
                 |       |                 |            |
                 |  122  | MouseReleased   | [RFC nnnn] |
                 |       |                 |            |
                 |  123  | MouseMoved      | [RFC nnnn] |
                 |       |                 |            |
                 |  124  | MouseWheelMoved | [RFC nnnn] |
                 |       |                 |            |
                 |  125  | KeyPressed      | [RFC nnnn] |
                 |       |                 |            |
                 |  126  | KeyReleased     | [RFC nnnn] |
                 |       |                 |            |
                 |  127  | KeyTyped        | [RFC nnnn] |
                 +-------+-----------------+------------+

        Table 5: Initial values of the HIP Message Type subregistry

9.3.  Media Type Registrations

   Following the guidelines in RFC 4855 [RFC4855] and RFC 4288
   [RFC4288], this section registers new 'application' media subtypes
   for remoting and hip.

9.3.1.  Registrations of Media Type application/remoting

   Type name:  application

   Subtype name:  remoting

   Required parameters:

      rate:  RTP timestamp clock rate, which is equal to the sampling
         rate.  The typical rate is 90000; other rates may be specified.

      retransmissions:  Informs the participants whether the AH supports
         UDP retransmissions.  The possible values are yes and no.








Boyaci & Schulzrinne     Expires April 30, 2009                [Page 30]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   Optional parameters:  none

   Encoding considerations:  This media type is framed binary data (see
      RFC 4288, Section 4.8).

   Security considerations:  See Section Section 8 of RFC nnnn

   Interoperability considerations:  none

   Published specification:  RFC nnnn

   Additional information:  none

   Person & email address to contact for further information:  Omer
      Boyaci <boyaci@cs.columbia.edu> and Henning Schulzrinne
      <hgs@cs.columbia.edu>

   Intended usage:  COMMON

   Restrictions on usage:  This media type depends on RTP framing, and
      hence is only defined for transfer via RTP (RFC 3550).  Transfer
      within other framing protocols is not defined at this time.

   Applications that use this media type:  Application and Desktop
      sharing tools.  Remote tutoring tools.

   Author:  Omer Boyaci and Henning Schulzrinne

   Change controller:  IETF Audio/Video Transport working group
      delegated from the IESG.

9.3.2.  Registrations of Media Type application/hip

   Type name:  application

   Subtype name:  hip

   Required parameters:

      rate:  RTP timestamp clock rate, which is equal to the sampling
         rate.  The typical rate is 90000; other rates may be specified.

   Optional parameters:  none

   Encoding considerations:  This media type is framed binary data (see
      RFC 4288, Section 4.8).





Boyaci & Schulzrinne     Expires April 30, 2009                [Page 31]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   Security considerations:  See Section Section 8 of RFC nnnn

   Interoperability considerations:  none

   Published specification:  RFC nnnn

   Additional information:  none

   Person & email address to contact for further information:  Omer
      Boyaci <boyaci@cs.columbia.edu> and Henning Schulzrinne
      <hgs@cs.columbia.edu>

   Intended usage:  COMMON

   Restrictions on usage:  This media type depends on RTP framing, and
      hence is only defined for transfer via RTP (RFC 3550).  Transfer
      within other framing protocols is not defined at this time.

   Applications that use this media type:  Application and Desktop
      sharing tools.

   Author:  Omer Boyaci and Henning Schulzrinne

   Change controller:  IETF Audio/Video Transport working group
      delegated from the IESG.


























Boyaci & Schulzrinne     Expires April 30, 2009                [Page 32]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


10.  Mapping to the Session Description Protocol (SDP)

   The information carried in this payload format has a specific mapping
   to fields in the Session Description Protocol (SDP) [RFC4566], which
   is commonly used to describe RTP sessions.  When SDP is used to
   specify sessions, the mappings are as follows:

10.1.  Mapping remoting Media Type Parameters into SDP

      The media type ("application") is carried in the SDP m= attribuate
      as the media name.

      The media subtype ("remoting") is carried in the SDP a=rtpmap as
      the encoding name.

      The parameter "rate" is carried in the SDP a=rtpmap attribuate as
      the clock rate.

      The mandated parameter "retransmissions" MUST be included in the
      SDP a=fmtp attribute.

10.2.  Mapping hip Media Type Parameters into SDP

      The media type ("application") is carried in the SDP m= attribuate
      as the media name.

      The media subtype ("hip") is carried in the SDP a=rtpmap
      attribuate as the encoding name.

      The parameter "rate" is carried in the SDP a=rtpmap attribuate as
      the clock rate.

10.3.  SDP Example

   The following example shows an example SDP usage.  This SDP message
   is from AH to participant.  The HIP stream and BFCP session are
   associated with each other via "label" and "m-stream" attributes
   according to SDP Format for BFCP Streams [RFC4583].  This SDP message
   informs the participant that AH supports both TCP and UDP for
   remoting.  The AH supports UDP retransmissions, so participants MAY
   send NACK requests for missing packets.  The port numbers MUST be
   same if AH is remoting the same content over both TCP and UDP.  In
   this example, AH is sending the same content from port 6000.  It is
   possible that AH may have more than one remoting session, in this
   case each session MUST use different port numbers.






Boyaci & Schulzrinne     Expires April 30, 2009                [Page 33]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


          m=application 50000 TCP/BFCP *
          a=floorid:0 m-stream:10
          m=application 6000 RTP/AVP 99
          a=rtpmap:99 remoting/90000
          a=fmtp: retransmissions=yes
          m=application 6000 TCP/RTP/AVP 99
          a=rtpmap:99 remoting/90000
          m=application 6006 TCP/RTP/AVP 100
          a=rtpmap:99 hip/90000
          a=label:10









































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 34]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


Appendix A.  Using BFCP for Application and Desktop Sharing

   Application and desktop sharing tools MAY utilize Binary Floor
   Control Protocol (BFCP) [RFC4582] for managing the ownership of AH's
   human interface devices (HID).

   BFCP defines several messages, but only five of them is a MUST for
   Application and Desktop Sharing, namely "Floor Request", "Floor
   Release", "Floor Granted", "Floor Released" and "Floor Request
   Queued".

   In Application and Desktop Sharing context, the floor is the AH's
   HIDs.  In this context, it is possible that the AH MAY temporarily
   block HID events without revoking the floor control.  For example,
   the AH MAY temporarily block HID events if the shared application
   loses the focus or is covered by a non-shared application.  The AH
   informs the current floor holder about the status of HIDs via STATUS-
   INFO attribute of "Floor Granted" messages.  The participant MAY
   receive several "Floor Granted" messages with different "HID Status"
   values.  Participant applications MAY inform the user about current
   "HID Status".  HID Status values are 16-bit unisgned values and are
   defined as:

                       +-------+------------------------+
                       | Value | Status                 |
                       +-------+------------------------+
                       |   0   | STATE_NOT_ALLOWED      |
                       |   1   | STATE_KEYBOARD_ALLOWED |
                       |   2   | STATE_MOUSE_ALLOWED    |
                       |   3   | STATE_ALL_ALLOWED      |
                       +-------+------------------------+

                       Figure 20: HID Status Values


















Boyaci & Schulzrinne     Expires April 30, 2009                [Page 35]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


11.  References

11.1.  Normative References

   [I-D.boyaci-avt-png]
              Boyaci, O. and H. Schulzrinne, "RTP Payload Format for
              Portable Network Graphics (PNG) image",
              draft-boyaci-avt-png-00 (work in progress),
              September 2008.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC2327]  Handley, M. and V. Jacobson, "SDP: Session Description
              Protocol", RFC 2327, April 1998.

   [RFC2435]  Berc, L., Fenner, W., Frederick, R., McCanne, S., and P.
              Stewart, "RTP Payload Format for JPEG-compressed Video",
              RFC 2435, October 1998.

   [RFC3261]  Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
              A., Peterson, J., Sparks, R., Handley, M., and E.
              Schooler, "SIP: Session Initiation Protocol", RFC 3261,
              June 2002.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", STD 64, RFC 3550, July 2003.

   [RFC3984]  Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund,
              M., and D. Singer, "RTP Payload Format for H.264 Video",
              RFC 3984, February 2005.

   [RFC4571]  Lazzaro, J., "Framing Real-time Transport Protocol (RTP)
              and RTP Control Protocol (RTCP) Packets over Connection-
              Oriented Transport", RFC 4571, July 2006.

   [RFC4582]  Camarillo, G., Ott, J., and K. Drage, "The Binary Floor
              Control Protocol (BFCP)", RFC 4582, November 2006.

   [RFC4583]  Camarillo, G., "Session Description Protocol (SDP) Format
              for Binary Floor Control Protocol (BFCP) Streams",
              RFC 4583, November 2006.

   [RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
              "Extended RTP Profile for Real-time Transport Control
              Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
              July 2006.



Boyaci & Schulzrinne     Expires April 30, 2009                [Page 36]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


   [RFC5371]  Futemma, S., Itakura, E., and A. Leung, "RTP Payload
              Format for JPEG 2000 Video Streams", RFC 5371,
              October 2008.

11.2.  Informative References

   [RFC2434]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", BCP 26, RFC 2434,
              October 1998.

   [RFC4288]  Freed, N. and J. Klensin, "Media Type Specifications and
              Registration Procedures", BCP 13, RFC 4288, December 2005.

   [RFC4855]  Casner, S., "Media Type Registration of RTP Payload
              Formats", RFC 4855, February 2007.

   [T120]     International Telecommunication Union, "Data Protocols for
              Multimedia Conferencing", X ITU-T Recommendation T.120.

   [X]        Scheifler, R., "X Window System Protocol", X Consortium
              Standard X Version 11, November 2004.

   [keycodes]
              OpenJDK Community, "Java key-codes", <http://
              hg.openjdk.java.net/jdk7/awt-gate/jdk/archive/tip.zip>.

   [theora]   Xiph.org Foundation, "Theora".
























Boyaci & Schulzrinne     Expires April 30, 2009                [Page 37]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


Authors' Addresses

   Omer Boyaci
   Columbia University
   Dept. of Computer Science
   1214 Amsterdam Avenue
   New York, NY  10027
   US

   Email: boyaci@cs.columbia.edu


   Henning Schulzrinne
   Columbia University
   Dept. of Computer Science
   1214 Amsterdam Avenue
   New York, NY  10027
   US

   Email: schulzrinne@cs.columbia.edu































Boyaci & Schulzrinne     Expires April 30, 2009                [Page 38]


Internet-Draft     RTP Payload Format for App Sharing       October 2008


Full Copyright Statement

   Copyright (C) The IETF Trust (2008).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.











Boyaci & Schulzrinne     Expires April 30, 2009                [Page 39]