Network Working Group I. Johansson
Internet-Draft Ericsson AB
Intended status: Standards Track June 26, 2008
Expires: December 28, 2008
Negotiation of Generic Image Attributes in SDP
draft-johansson-mmusic-image-attributes-01
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 28, 2008.
Abstract
This draft proposes a new generic session setup attribute to make it
possible to negotiate different image attributes such as image size.
A possible use case is to make it possible for a e.g a low-end hand-
held terminal to display video without the need to rescale the image,
something that may consume large amounts of memory and processing
power. The draft also helps to maintain an optimal bitrate for video
as only the image size that is desired by the receiver is
transmitted.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
Johansson Expires December 28, 2008 [Page 1]
Internet-Draft Image Attributes in SDP June 2008
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 3
3. Defintion of Attribute . . . . . . . . . . . . . . . . . . . . 3
3.1. SDPCapNeg . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2. Asymmetric Sessions . . . . . . . . . . . . . . . . . . . 6
4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . . . 7
4.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . . . 7
4.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . . . 7
4.5. Example 5 (SDPCapNeg) . . . . . . . . . . . . . . . . . . 8
4.6. Example 6 (Asymmetry) . . . . . . . . . . . . . . . . . . 8
5. Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
7. Security Considerations . . . . . . . . . . . . . . . . . . . 11
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
9.1. Informative References . . . . . . . . . . . . . . . . . . 11
9.2. Normative References . . . . . . . . . . . . . . . . . . . 12
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 12
Intellectual Property and Copyright Statements . . . . . . . . . . 13
Johansson Expires December 28, 2008 [Page 2]
Internet-Draft Image Attributes in SDP June 2008
1. Introduction
This draft proposes a new attribute to make it possible to negotiate
different image attributes such as image size. The term image size
is defined here as it may differ from the physical screen size of e.g
a hand-held terminal. For instance it may be beneficial to display a
video image on a part of the physical screen and leave space on the
screen for e.g menus and other info.
There are a number of benefits with a possibility to negotiate the
image size:
o Less image distortion: Rescaling of images introduces additional
distortion, something that can be avoided (at least on the
receiver side) if the image size can be negotiated.
o Reduced complexity: Image rescaling can be quite computation
intensive. For low end devices this can be a problem.
o Optimal quality for the given bitrate: The sender does not need to
encode an entire CIF (352x288) image if only an image size of
288x256 is displayed on the receiver screen. This gives a saving
in bitrate.
o Memory requirement: The receiver device will know the size of the
image and can then allocate memory accordingly.
Several of the existing standards (see [RFC4587], [RFC4629] and
[RFC3984]) has support for enabling the receiver to ask for different
resolutions at different framerates. The purpose of this memo is to
provide for a generic mechanism and is targeted mainly at the
negotiation of the image size but to make it more general the
attribute is named "imageattr". A problem statement and discussion
that gives a motivation to this draft can be found in [S4-080144].
2. Conventions, Definitions and Acronyms
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
3. Defintion of Attribute
A new "imageattr" attribute is defined. The imageattr attribute
contains a set of different image-resolution options that the offerer
can provide. The receiver can then select the desired image
Johansson Expires December 28, 2008 [Page 3]
Internet-Draft Image Attributes in SDP June 2008
attribute (e.g image size) and has then the ability to avoid costly
transformations (e.g rescaling) of the images. In this approach only
the image resolution and optionally the framerate, sample aspect
ratio and preference is covered but the framework makes it possible
to extend with other image related attributes that makes sense.
The syntax for the imageattr attribute is:
Incomplete formulation:
a=imageattr:<PT> 1*WSP *
May be given in an offer to indicate that the answer
MUST contain a complete formulation given below.
Complete formulation:
a=imageattr:<PT> 1*WSP <sar=range 1*WSP> <list>
list = set *(1*WSP set) ; defined in RFC 4566
<PT> is the payload type number
sar is the a set of supported sample aspect ratios (optional)
Each set for the complete formulation is defined by
set=[x=range,y=range<,par=range><,fr=range><,q=value>]
x is the horizontal image size range
y is the vertical image size range
fr is the framerate range for the given set (optional)
par is a range of valid picture aspect ratios (optional)
q is the preference for the given set in the
range [0.0..1.0] (optional)
range is (in matlab or octave syntax) given by
range=value
or
range=[value:value]
or
range=[value:step:value]
or
range=[value,value...value]
value is a positive integer or real value
step is a positive integer or real value
If step is left out in the syntax a stepsize of 1 is implied.
Note the use of brackets [..] if more that one value specified.
Selection of a desired image attribute entry follows the syntax:
Johansson Expires December 28, 2008 [Page 4]
Internet-Draft Image Attributes in SDP June 2008
a=imageattr:<PT> 1*WSP <sar=value 1*WSP> \
<[<fr=range,>x=value,y=value]>
or
a=imageattr:<PT> 1*WSP <sar_w=value 1*WSP sar_h=value 1*WSP> \
<[<fr=range,>x=value,y=value]>
"\" denotes a line break that MUST NOT be present in the actual SDP
The imageattr attribute is bound to a specific media by means of the
payload type number. Several imageattr can be defined e.g for
different video codec alternatives conditioned that the payload type
number differs.
The incomplete formulation (e.g "a=imageattr:97 *" ) in an offer
indicates that the answer MUST contain a complete formulation or that
the imageattr is removed from the answer. This is used for offer/
answer in a session in the likely case that the two participants have
different display capabilities.
The preference for each set is 0.5 by default, setting the optional q
parameter to another value makes it possible to set different
preferences for the sets.
The sar parameter is a list of the possible SAR (Sample Aspect Ratio)
conversions that is supported by the encoder, this applies to all the
defined sets for the given imageattr line. This list is according to
table E1 in [H.264]. A value of 255 indicates that the encoder
supports any conversion. In this case the answer should specify the
desired sar_w and sar_h values (if desired). The answer SDP
indicates what sample aspect ratio (on the receiver display) that the
sender shall compensate for.
The par (width/height = x/y ratio) parameter indicates a range of
picture aspect ratios. This is used to limit the number of x and y
image size combinations, par is given as
par=[par_min,par_max]
Where par_min and par_max are the min and max allowed picture aspect
ratios.
The offerer MUST be able to support the image attributes that it
offers. As an example, an offerer that cannot provide with
compensation for sample aspect ratios MUST NOT include sar in the
offer. The receiver MAY accept the imageattr but is not required to
do so. The receiver MUST only include a valid entry taken from the
offer in the SDP answer.
Johansson Expires December 28, 2008 [Page 5]
Internet-Draft Image Attributes in SDP June 2008
The answerer MUST only pick a valid entry from the multidimensional
solution space spanned by the offer. If "fr" is given in the offer
it MUST be present in the answer, the answer MAY limit the fr range
but MUST NOT increase the range.
3.1. SDPCapNeg
For use with the SDP capability negotiation framework (SDPCapNeg) an
extra attribute "imcap" is defined with the syntax:
a=imcap:<tag> 1*WSP <sar=range 1*WSP> <list>
<tag> is a number from 1 to 1^32-1 both included.
list = set *(1*WSP set) ; defined in RFC 4566
The set and sar is defined as earlier in this text.
The imcap attributes are referenced with the parameter "im" in the
pcfg and acfg lines. Several imcap may be referenced in the
potential configurations.
The ability to support the imcap attribute MUST be specified with the
"im-v0" identifier on the a=creq line or using the "+" option on the
pcfg line and optionally also on the a=csup line.
3.2. Asymmetric Sessions
As the display capabilities in the devices involved in a session may
vary greatly it comes natural that this will likely lead to an
asymmetry in terms of display size.
As an example Alice (offerer) may prefer an image size of 320x240
while Bob prefers an image size of 800x600. This problem is made
even worse if the two participants have different sample aspect
ratios on the displays.
To solve this issue it is necessary to use the sendonly and recvonly
attributes as described in [RFC3264]. An example how this may look
like is found in Section 4.6.
4. Examples
A few examples to highlight the syntax
4.1. Example 1
a=imageattr:97 sar=[1:16] [x=800,y=640,q=0.6] [x=480,y=320]
Johansson Expires December 28, 2008 [Page 6]
Internet-Draft Image Attributes in SDP June 2008
Two image resolution alternatives are offered with 800x600 having the
highest preference, furthermore the encoder supports conversion for
the sar indices 1..16 given in table E1 in [H.264].
The answer may look like
a=imageattr:97 sar=2 [x=800,y=640]
Indicating that the receiver wish the encoder to compensate for a
sample aspect ratio of 12:11 and desires an image size on its screen
of 800x600.
4.2. Example 2
a=imageattr:97 sar=255 [fr=[5,15],x=800,y=640] \
[fr=[15,30],x=480,y=320]
Two image resolution alternatives with different framerates are
offered, furthermore the encoder supports conversion for any sample
aspect ratio.
The answer may look like
a=imageattr:97 sar_w=13 sar_h=11 [x=480,y=320]
Indicating a desired image size of 480x320 and requesting the encoder
to compensate for a sample aspect ratio of 13:11. No specific
framerate is requested.
4.3. Example 3
a=imageattr:97 \
[x=[480:16:800],y=[320:16:640],par=[1.2,1.3],q=0.6] \
[x=[176:8:208],y=[144:8:176],par=[1.2,1.3]]
Two image resolution sets are offered with the first having a higher
preference (q=0.6). The x-axis resolution can take the values 480 to
800 in 16 pixels steps and 176 to 208 in 8 pixels steps. The par
parameter limits the set of possible x and y screen resolution
combinations such that 800x640 (par=1.25) is a valid combination
while 800x672 (par=1.19) or 800x608 (par=1.31) are invalid
combinations.
4.4. Example 4
Given the offer
a=imageattr:98 [x=[480:16:800],y=[320:16:640]] \
Johansson Expires December 28, 2008 [Page 7]
Internet-Draft Image Attributes in SDP June 2008
[x=[176:8:208],y=[144:8:176]]
An answer may look like
a=imageattr:98 [x=192,y=160]
Indicating that the answerer wish to render the video through a
rectangular 192x160 block (image).
4.5. Example 5 (SDPCapNeg)
An offer using SDPCapNeg may look like
m=video ....
a=creq:med-v0,im-v0
m=mcap:1 H_264
a=imcap:1 [x=[480:16:800],y=[320:16:640]] \
[x=[176:8:208],y=[144:8:176]]
a=pcfg t=.. a=.. m=.. im=1 pt=1:97
The SDP answer may then look like
m=video ....
a=csup:med-v0,im-v0
a=imageattr:97 [x=192,y=160]
a=acfg t=.. a=.. m=.. im=1 pt=1:97
Note the inclusion of the "clear text" imageattr in the answer that
gives the desired image size.
4.6. Example 6 (Asymmetry)
The offer/answer procedure between Alice (offer) and Bob with
different display capabilities may look like below
Offer:
m=video 12340 RTP/AVP 97
a=rtpmap:97 H264/90000
a=imageattr:97 [x=[480:16:800],y=[320:16:640]] \
[x=[176:8:208],y=[144:8:176]]
a=sendonly
m=video 12342 RTP/AVP 98
a=rtpmap:98 H264/90000
a=imageattr:98 *
a=recvonly
The image attribute for the 2nd m-line is incomplete, which means
that Bob must fill in the imageattr capabilities that it can support,
Johansson Expires December 28, 2008 [Page 8]
Internet-Draft Image Attributes in SDP June 2008
optionally Bob may remove this attribute for the 2nd m-line, thus
indicating that it does not offer any special image attributes for
the media in the direction from Bob to Alice other than what may be
supported by the payload format.
The answer may look like below
Answer:
m=video 12340 RTP/AVP 97
a=rtpmap:97 H264/90000
a=imageattr:97 [x=480,y=320]
a=recvonly
m=video 12342 RTP/AVP 98
a=rtpmap:98 H264/90000
a=imageattr:98 sar=[1:16] [x=[480:16:800],y=[320:16:640]] \
[x=[176:8:208],y=[144:8:176]]
a=sendonly
The answer indicates that Bob wish to render the received video
stream with an image size of 480x320. The imageattr for the 2nd
m-line is filled with the image attribute capabilities that Bob can
support when sending media to Alice.
Alice receives the answer and performs another offrer/answer, the 2nd
offer answer is necessary to make it possible for Alice to select e.g
a desired image size.
Offer (2):
m=video 12340 RTP/AVP 97
a=rtpmap:97 H264/90000
a=imageattr:97 [x=480,y=320]
a=sendonly
m=video 12342 RTP/AVP 98
a=rtpmap:98 H264/90000
a=imageattr:98 sar=2 [x=800,y=640]
a=recvonly
Answer (2):
m=video 12340 RTP/AVP 97
a=rtpmap:97 H264/90000
a=imageattr:97 [x=480,y=320]
a=recvonly
m=video 12342 RTP/AVP 98
a=rtpmap:98 H264/90000
a=imageattr:98 sar=2 [x=800,y=640]
a=sendonly
The second offer indicates that Alice prefers to render the video on
the screen with a image size of 800x600, furthermore it indicates
Johansson Expires December 28, 2008 [Page 9]
Internet-Draft Image Attributes in SDP June 2008
that it wants the sender (Bob) to compensate for a sample aspect
ratio of 12:11.
5. Issues
Here is listed a few possible issues and observations connected to
the use of the imageattr SDP attribute :
o Conflict with sprop-parameter-set: H.264 defines the sprop-
parameter-set which may define features that are defined in this
draft. To avoid conflicts, sessions that use the imageattr
attribute MUST NOT specify similar features in sprop-parameter-
set.
o Image scaling on encoder side: e.g the H.264 codec is not mandated
to rescale the video to fit a preferred receiver image size. If
this is the case then a possible solution is that only a part of
the input, corresponding to the preferred size, is encoded
(cropping). Preferably the cropped image is also displayed as a
miniature image on the sender display as well to avoid that the
important parts drop out of the image. Cropping of the image may
also occur if the picture aspect ratio differs between the video
camera and the preferred image size on the receiver end.
o Sample aspect ratios (SAR): In this draft version table E1 in
[H.264] is referenced. Question if it is better to reference to
another table.
o Image sharpening: In normal cases image sharpening is to be
performed after rescaling on the sender side. There may however
occur cases where it is preferable to do the sharpening on the
receiver side instead.
o A high end device may not see any need for imageattr for it as it
most likely has the processing capacity to rescale incoming video
and may therefore not include the attribute in the offer. The
first answer MAY however include an imageattr with incomplete
formulation, this will of course lead to extra offer/answer
rounds.
o As the likelihood for asymmetric properties (different image size)
for each participant is quite big, the outcome is likely that it
is necessary to specify media in each direction as given in
Section 4.6. This in practice means more overhead in terms of
both extra offer/answer and increased RTCP traffic.
Johansson Expires December 28, 2008 [Page 10]
Internet-Draft Image Attributes in SDP June 2008
o What other methods are available to handle asymmetry?
o The syntax in this draft may not follow the rules in RFC4566
6. IANA Considerations
T.B.D
7. Security Considerations
This draft does not add any additional security issues other than
those already existing with currently specified offer/answer
procedures.
8. Acknowledgements
The author would like to thank the people who has contributed with
objections and suggestions to this draft.
9. References
9.1. Informative References
[H.264] ITU-T, "ITU-T Recommendation H.264,
http://www.itu.int/rec/T-REC-H.264-200711-I/en".
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
June 2002.
[RFC3984] Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund,
M., and D. Singer, "RTP Payload Format for H.264 Video",
RFC 3984, February 2005.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC4587] Even, R., "RTP Payload Format for H.261 Video Streams",
RFC 4587, August 2006.
[RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R.
Even, "RTP Payload Format for ITU-T Rec", RFC 4629,
January 2007.
Johansson Expires December 28, 2008 [Page 11]
Internet-Draft Image Attributes in SDP June 2008
[S4-080144]
3GPP, "Signaling of Image Size: Combining Flexibility and
Low Cost, http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/
TSGS4_48/Docs/S4-080144.zip".
9.2. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Author's Address
Ingemar Johansson
Ericsson AB
Laboratoriegrand 11
SE-971 28 Lulea
SWEDEN
Phone: +46 73 0783289
Email: ingemar.s.johansson@ericsson.com
Johansson Expires December 28, 2008 [Page 12]
Internet-Draft Image Attributes in SDP June 2008
Full Copyright Statement
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Johansson Expires December 28, 2008 [Page 13]