[Search] [txt|pdfized|bibtex] [Tracker] [Email] [Nits]

Versions: 00                                                            
       Internet Engineering Task Force                  SIPPING WG
       Internet Draft                                   van Wijk/Rosenberg/
       Document: <draft-vanwijk-sipping-deaf-req-       Gearhart/Sinnreich/
       00.txt>                                          Schulzrinne
       July 2001                                        Ericsson/Dynamicsoft
       Expires: December 2001                           /WCOM/Columbia U.
       Category: Informational
                 SIP Support for Hearing and Speech Impaired Users
       Status of this Memo
          This document is an Internet-Draft and is in full conformance with
          all provisions of Section 10 of RFC2026 [10].
          Internet-Drafts are working documents of the Internet Engineering
          Task Force (IETF), its areas, and its working groups. Note that
          other groups may also distribute working documents as Internet-
          Drafts. Internet-Drafts are draft documents valid for a maximum of
          six months and may be updated, replaced, or obsoleted by other
          documents at any time. It is inappropriate to use Internet-Drafts
          as reference material or to cite them other than as "work in
          The list of current Internet-Drafts can be accessed at
          The list of Internet-Draft Shadow Directories can be accessed at
       1. Abstract
          SIP is attracting more and more attention as a valuable tool to
          enable and support voice and multimedia communications over the
          Internet. However, some users of SIP-based services will be unable
          or severely restricted to use plain voice communication. In
          particular, people who are hearing impaired often cannot send
          and/or receive voice. This document is a merger between 2
          previously written drafts [6] and [7] regarding SIP and The
          Hearing Impaired and its goal is to lay a foundation for design
          and implementation of SIP based services.
          These services are generally enabled by baseline SIP [1], or
          through the use of the caller preferences specification [3]. No
          additional extensions are proposed here in order to support
          universal access.
       2. Terminology and Conventions Used in This Document
          In this document, the key words "MUST", "MUST NOT",
          "RECOMMENDED","MAY", and "OPTIONAL" are to be interpreted as
          described in RFC2119 [8] and indicate requirement levels for
          compliant SIP implementations. For the purposes of this document,
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 1 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          the term terminal will be used to represent the hearing impaired
          individual and his/her SIP-enabled end-user equipment; this could
          be a TTY[9], a personal computer or PDA, a specially equipped
          mobile phone, video phone, etc. The term user, in this document,
          shall be understood to mean a hearing or speech impaired
       3. Introduction
          The Session Initiation Protocol (SIP)[1] is used to initiate,
          modify, and terminate interactive sessions between sets of users.
          Typically these are voice sessions, described by the Session
          Description Protocol (SDP)[2]. However not all users or potential
          users have access to SIP services. Specifically, people who are
          Culturally Deaf, Audiologically deaf, hard of hearing, speech
          impaired or disabled, etc., on either a temporary or permanent
          basis, are unable to participate in plain voice-based
          communications. For the purposes of this document, this group of
          people will be referred to collectively as Hearing Impaired
          individuals.  Also, the term Sign Language (SL), is used in this
          document to refer to the natural signed language used by the Deaf.
          In the United States and parts of Canada and Mexico, American Sign
          Language (ASL) is used. For Deaf individuals in other countries,
          for example Britain, British Sign Language (BSL) is used. Also,
          since Deaf people can and do use various types of manual
          communications systems in addition to SL such as Signed English
          for this document the term SL will be assumed to cover all forms
          of manual communications.
          Within the Public Switched Telephone System (PSTN), services have
          been defined that allow for access to circuit switched based relay
          voice services by these users. We believe it is important to offer
          similar or better services in an IP context. The flexibility of
          SIP affords us the ability to both offer and improve on these
          services, and to offer more extensive forms of universal service
          access to this group of customers.  This is a formative time for
          the future of IP-based communications and as such it is an
          appropriate time to ensure that such requirements as are necessary
          to ensure full accessibility by all customers are included in
          planning the new networks using SIP, such as the 3G wireless
       4. Purpose and Scope
          This document will first describe a few possible services using
          call flows that enable universal access of voice and multimedia
          sessions, initiated by SIP to users who are hearing impaired. And
          second, offers a proposed set of requirements based on the
          services and attempts to identify issues that need resolution to
          allow full accessibility for all customers to SIP-based Internet
          These services are generally enabled by baseline SIP[1], or
          through the use of the caller preferences specification [3]. No
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 2 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          additional extensions are proposed here in order to support
          universal access.
       5. Background
          In the current telephony world, most hearing impaired individuals
          have access to either standalone or PC-based TTY devices.  These
          text-based devices or software packages are adequate for
          communication with other hearing impaired individuals or with
          hearing individuals that have similar devices or software.  In
          addition, most if not all states in the United States have state-
          sponsored relay service, in which a human operator with both
          standard telephone equipment and TTY devices acts as a go-between
          ("relay operator" or interpreter), allowing a hearing impaired
          person with a TTY to place or receive calls from hearing
          individuals with standard telephones. Individuals with speech
          difficulties that render it impossible for the individual to use
          ordinary voice-enabled devices can use the same TTY equipment and
          relay services. Introduction of the relay service has been a great
          benefit to both the hearing impaired and hearing people because it
          has enabled communication between them, where, prior to this time,
          it was difficult or impossible.  It has provided independence to
          hearing impaired people, allowing them to communicate on their own
          terms rather than being forced to rely on hearing friends to act
          as telephone interpreters.
          In recent years, other technology such as two-way e-mail pagers
          have become available that provide portable communications for
          hearing impaired people in a manner similar to cellular telephones
          for hearing people. These have provided a great benefit to hearing
          impaired people, allowing them the ease of near-instant
          communications that hearing people now take for granted. Some e-
          mail pager services have also supported interface with other
          devices such as TTYs and FAX.  In addition, most European GSM
          phones and other mobile services elsewhere offer SMS (short
          message service) for exchanging short text messages to other
          subscribers. This will also allow hearing impaired people to
          communicate near-instantly.
          Most recently, the advent of video relay services has provided
          ways for hearing impaired individuals to converse with hearing
          individuals using their most natural means of communication,
          visual, through the use of an oral or sign language interpreter.
          The human interpreter is typically physically located at the relay
          center and provides interpreting services via video connection to
          the hearing impaired person and voice connection to the hearing
          It is not easy for a hearing person to understand the strong need
          for the hearing impaired people to have a reliable and easy to use
          relay service. But imagine yourself to live one day without a
          telephone. From doing business to simply ordering a pizza, the
          telephone is often unavoidable today. And that is no small feat.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 3 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          Imagine then how this is if this was not just one day, but for the
          rest of your life? Now you may get an idea why the relay service
          is so important for the hearing impaired people. Even a small
          improvement will be received as a mayor improvement.
          The reader of this draft is supposed to have at least a
          superficial knowledge and understanding of the deaf and hard of
          hearing world and the communication problems that have been
          endemic in that population since the creation of voice-only, long-
          distance communications. This may not be realistic, so the authors
          recommend the interested readers the following books as
          supplemental readings [11] [12] [13] [14].
        6. Example Services and Call Flows
          We provide the following examples services and accompanying call
          1 - Redirect to IM:
          The caller has a phone and an IM client. The called party has a
          phone and an IM client. The phone call is redirected to IM and
          both parties use IM to communicate.
          2 - One-way speech to text translation service:
          The caller has only a phone. The called party has a text terminal
          to receive and a phone to send.  A relay service translates in one
          direction only from speech to text.
          3 - One-way speech to sign language translation service:
          The caller has just a phone. The called party has a video terminal
          to receive and a phone to send. A relay service translates in one
          direction only from speech to video, with the video being a sign
          language representation of the speech.
          4 - Two-way speech to text and text to speech with translation
          The caller has a phone only. The called party uses text both ways.
          A relay service translates in one direction from text to speech
          and from speech to text in the other direction. A computer can do
          the text to speech translation.
          5 - Hearing impaired calling party calling through relay:
          The caller has text only. The called party only has a phone. A
          relay service translates in one direction from text to speech and
          from speech to text in the other direction. A computer can do the
          text to speech translation.
       6.1 Redirect to IM
          One advantage of providing SIP based voice services through the
          Internet is the seamless access to other IP services that can be
          used in conjunction with voice. Instant Messaging (IM) is
          particularly useful for the hearing impaired. IM allows for
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 4 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          instantaneous text messaging between IP connected users. Using SIP
          for IM service is recently described [15].
          One way to use IM to support the hearing impaired is to redirect a
          voice call to an IM ôcallö (provided the caller supports IM). The
          service works as follows. A voice call is initiated by a terminal
          that can support IM as well. Indication of support for IM is done
          through the caller preferences specification [3], which allows the
          caller to indicate characteristics of the URLs they are willing to
          be redirected to. In this case, they would indicate support of the
          MESSAGE method, used for instant messaging within SIP. Support for
          other instant messaging protocols, so long as they are described
          by standardized URL schemes, can also be indicated.
          When the call arrives at the user agent of the hearing impaired
          user, the UA checks for support of instant messaging. If such
          support is indicated, the UAS sends a 302 (Use IM - Hearing
          Impaired) redirect, containing a URL to be used for IM. This
          redirect is forwarded back to the calling party, whose IM tool
          pops up with an IM filled in with the address of the called party.
          The two can then communicate in a pure IM session.
          The service can also be provided by an application server serving
          the hearing impaired user. The application server, upon receiving
          the INVITE, would initiate its own INVITE towards the hearing
          impaired user (without indicating any kind of media session). This
          has the effect of alerting (through a flashing light or some other
          means) that an incoming call is taking place. If accepted, the
          application server can then redirect the initial caller to send an
          IM to a pre-configured IM address.
          Figure 1 contains a call flow for the service assuming it is being
          provided by the called UA.
                       |                             |
                       |    F1: INVITE               |
                       | --------------------------> |
                       |                             |
                       |                             |
                       |   F2: 302 IM                |
                       | <-------------------------- |
                       |                             |
                       |                             |
                       |   F3: ACK                   |
                       | --------------------------> |
                       |                             |
                       |                             |
                       |                             |
                       |                             |
                       |    F4: MESSAGE              |
                       | --------------------------> |
                       |    F5: 200 OK               |
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 5 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
                       | <-------------------------- |
                       |                             |
                       |    F6: MESSAGE              |
                       | <-------------------------- |
                       |    F7: 200 OK               |
                       | --------------------------> |
                       |                             |
                    Caller                        Hearing
          Figure 1: Redirecting to an IM
          Message F1 is:
             INVITE sip:hiu@example.com SIP/2.0
             Via: SIP/2.0/UDP a.example.com
             From: sip:caller@example.com
             To: sip:hiu@example.com
             Call-ID: 9asdg9a7@
             CSeq: 1 INVITE
             Contact: sip:caller@a.example.com
             Accept-Contact: *;methods=''MESSAGE,SUBSCRIBE''
             Content-Type: application/sdp
             Content-Length: XX
             Message F2 is:
             SIP/2.0 302 Use IM - Hearing Impaired
             Via: SIP/2.0/UDP a.example.com
             From: sip:caller@example.com
             To: sip:hiu@example.com;tag=9ajsd9aumlaa
             Call-ID: 9asdg9a7@
             CSeq: 1 INVITE
             Contact: sip:hiu@example.com;method=MESSAGE
       6.2 One-way Speech-to-text Translation Service
          The above IM service is very interesting and quite useful for
          short messages. But when a hearing impaired user needs to place or
          receive a telephone call, a relay service is necessary. A relay is
          a person who can listen to the calling party, type up the text,
          and send it to the hearing impaired user either through instant
          messages or through text over RTP [5].
          In one variant of this service, a call is made to a hearing
          impaired person. If the hearing impaired user wishes to accept the
          call, they send a 183 (Using a Relay for Hearing Impaired)
          response to the call.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 6 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          The provisional response to the caller is used by the client to
          alert the caller to the fact that the called party is hearing
          disabled and that a relay service will be part of the call. This
          is useful to help the caller to tune the speaking style, so as to
          adjust for such a type of communication.
          Then, after sending the 183, using the third party call control
          mechanisms [4], the called party launches a call to a relay
          server, with the INVITE containing SDP that indicates support for
          only the RTP payload format for text messages. The response from
          the relay speech to text (STT) server (presumably accepting the
          call), contains SDP where the relay server expects to receive
          audio to be translated to text. When this 200 OK arrives at the
          hearing impaired user, that SDP is placed into the 200 OK of the
          call. The result is that the caller will be sending media to the
          relay server, and the hearing impaired user will receive a textual
          version of it over RTP. However, the hearing impaired user sends
          audio directly to the caller voice carry over (VCO), assuming that
          the hearing impaired user is able to communicate verbally). When
          this is the case, it has the advantage of sending the speech
          directly between the participants in the direction that is
          possible, resulting in less latency and more privacy for the
          callers since the relay will now hear only half of the
          conversation. With the current PSTN, the relay interpreter would
          hear the whole conversation when VCO is enabled (relay acts then
          as a conference bridge in VCO mode).
          The call flow for this service is depicted in Figure 2.
             Message F1 is:
             INVITE sip:hiu@example.com SIP/2.0
             Via: SIP/2.0/UDP a.example.com
             From: sip:caller@example.com
             To: sip:hiu@example.com
             Call-ID: 9asdg9a7@
             CSeq: 1 INVITE
             Contact: sip:caller@a.example.com
             Accept-Contact: *;methods=''MESSAGE,SUBSCRIBE''
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 1>
             message F2 is:
             SIP/2.0 183 Using Relay for Hearing Impaired... Please Wait
             Via: SIP/2.0/UDP a.example.com
             From: sip:caller@example.com
             To: sip:hiu@example.com;tag=9ajsd9aumlaa
             Call-ID: 9asdg9a7@
             CSeq: 1 INVITE
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 7 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
             message F3 is:
             INVITE sip:speech2txt@example.com SIP/2.0
             Via: SIP/2.0/UDP b.example.com
             From: sip:hiu@example.com
             To: sip:speech2txt@example.com
             Call-ID: 88725392k@
             CSeq: 7 INVITE
             Contact: sip:hiu@b.example.com
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 2 with text RTP payload format as only codec>
             message F4 is:
             SIP/2.0 200 OK - translating
             Via: SIP/2.0/UDP b.example.com
             From: sip:hiu@example.com
             To: sip:speech2txt@example.com;tag=1238827819
             Call-ID: 88725392k@
             CSeq: 7 INVITE
             Contact: sip:speech2txt@c.example.com
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 3>
             message F5 is:
             SIP/2.0 200 OK
             Via: SIP/2.0/UDP a.example.com
             From: sip:caller@example.com
             To: sip:hiu@example.com;tag=9ajsd9aumlaa
             Call-ID: 9asdg9a7@
             CSeq: 1 INVITE
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 3>
                  |                        |                         |
                  |   F1: INVITE           |                         |
                  | ---------------------> |                         |
                  |                        |                         |
                  |   F2: 183              |                         |
                  | <--------------------- |                         |
                  |                        |   F3: INVITE            |
                  |                        | ----------------------> |
                  |                        |                         |
                  |                        |   F4: 200 OK            |
                  |                        | <---------------------- |
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 8 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
                  |   F5: 200 OK           |                         |
                  | <--------------------- |                         |
                  |                        |                         |
                  |                        |                         |
                  |   F6: ACK              |                         |
                  | ---------------------> |                         |
                  |                        |   F7: ACK               |
                  |                        | ----------------------> |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                  |    RTP (audio)         |                         |
                  | ===============================================> |
                  | <===================== |                         |
                  |                        |                         |
                  |                        |                         |
                  |                        |    RTP (text)           |
                  |                        | <====================== |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                  |                        |                         |
                Caller                   Hearing                   Relay
                                         Impaired                   STT
             Figure 2: One Way Translation Service
             message F6 and F7 are standard ACK messages, not shown.
       6.3 One-way Speech-to-Sign-Language Translation Service
          The caller from a normal phone makes a call to a hearing impaired
          user who needs to communicate with Sign language. The hearing
          impaired user establishes a connection with a Relay service that
          will listen to speech and "convert" it to sign language. The sign
          language is sent to the hearing impaired used through a video
          This service is accomplished identically to the one-way speech to
          text translation service. The call flow is the same as listed in
          Figure 2. The only difference is that the SDP that indicates text,
          will instead indicate video. The RTP stream marked as containing
          text, will instead contain video. And the Relay STT should be read
          as Relay Speech to Sign language (STS).
       6.4 Two-way speech to text and text to speech with Relay service
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne       [Page 9 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          The service in the previous section can be extended to include one
          relay for speech to text and another that does text to speech
          (where the text is typed by the user who is unable to communicate
          verbally). The text to speech translation can be done by a
          computer this is cheaper then using a human operator). If people
          are used to translate in both directions, these translators may be
          the same person, but they need not be. As mentioned in 6.2 this
          has an interesting effect of introducing some form of privacy.
          With two different translators, neither receives the complete
          conversation, and in all likelihood, would not be able to
          ascertain what is actually being talked about. But since Relay
          operators adhere to privacy and security rules as mentioned in 7,
          it is expected to be the same (human) operator. A call flow for
          this variant on the service is shown in Figure 3.
          Messages F1, F2, F3 and F4 are the same as above. F5 is a standard
             F6 is:
             INVITE sip:text2speech@example.com SIP/2.0
             Via: SIP/2.0/UDP b.example.com
             From: sip:hiu@example.com
             To: sip:text2speech@example.com
             Call-ID: 87765448902@
             CSeq: 88 INVITE
             Contact: sip:hiu@b.example.com
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 1>
             and F7 looks like:
             SIP/2.0 200 OK
             Via: SIP/2.0/UDP b.example.com
             From: sip:hiu@example.com
             To: sip:text2speech@example.com;tag=9asdgnzli98a0
             Call-ID: 87765448902@
             CSeq: 88 INVITE
             Contact: sip:text2speech@d.example.com
             Content-Type: application/sdp
             Content-Length: XX
             <SDP 4 w/ RTP payload type for text>
          F8 is a standard ACK. F9 looks like F5 from the asymmetric version
          of the service.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 10 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
                  |                        |                       |       |
                  |   F1: INVITE           |                       |       |
                  | ---------------------> |                       |       |
                  |                        |                       |       |
                  |   F2: 183              |                       |       |
                  | <--------------------- |                       |       |
                  |                        |   F3: INVITE          |       |
                  |                        | --------------------> |       |
                  |                        |                       |       |
                  |                        |   F4: 200 OK          |       |
                  |                        | <-------------------- |       |
                  |                        |                       |       |
                  |                        |   F5: ACK             |       |
                  |                        | --------------------> |       |
                  |                        |                       |       |
                  |                        |   F6: INVITE          |       |
                  |                        | ----------------------------> |
                  |                        |                       |       |
                  |                        |   F7: 200 OK          |       |
                  |                        | <---------------------------- |
                  |                        |                       |       |
                  |                        |   F8: ACK             |       |
                  |                        | ---------------------------- >|
                  |                        |                       |       |
                  |   F9: 200 OK           |                       |       |
                  | <--------------------- |                       |       |
                  |                        |                       |       |
                  |                        |                       |       |
                  |   F10 ACK              |                       |       |
                  | ---------------------> |                       |       |
                  |                        |  RTP (speech)         |       |
                  |===============================================>|       |
                  |                        |<======================|       |
                  |                        |     RTP (text)        |       |
                  |                        |                       |       |
                  |                        |                       |       |
                  |                        |     RTP (text)        |       |
                  |                        |==============================>|
                  |         RTP (Speech)   |                       |       |
                  |                        |                       |       |
                  |                        |                       |       |
                Caller                   Hearing                 STT     TTS
           Figure 3: Two-way speech to text and text to speech
          This approach has also the advantage that any application service
          provider can be used for these translation services. Different
          providers can be used for each direction, and these providers do
          not need to be affiliated in any way with the ISP providing IP
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 11 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          services for the hearing impaired user. This provides the
          potential for competition, and thus improved service.
          This approach also has the advantage of allowing one direction
          (speech to text), the other direction (text to speech), or both,
          to be performed by automated systems. For example, text to speech
          technology is fairly robust, and could be used in one direction,
          whereas a human operator could be used in the reverse (speech to
          text) direction, since speech recognition is not (yet) that
          robust. The call flow is completely identical, independently of
          whether the translation is done by human or machine. A machine
          would simply answer all calls to a specific address
          (sip:translator@asp.com), and echo the media (text or speech) back
          to the caller after conversion (conversion direction would be
          determined by the media capabilities indicated in the INVITE). In
          fact, there are other applications for such conversion systems.
          Providers could not only enable services for the hearing impaired,
          but other applications as well. Examples include voice browsing of
          the web, e-mail to speech readout over phones, and instant message
          to voicemail services. In fact, the opposite direction is quite
          likely - providers that perform these services can reuse their
          systems, without any work, to also provide services to the hearing
          impaired, as long as a relay service is reachable across the net.
       6.5 Hearing Impaired Calling Party through Relay
          In this section, we consider a relay where the calling party is
          hearing impaired.
          This service works much like the one described above, relying on
          third party call control mechanisms. The hearing impaired caller
          sends an INVITE with SDP containing no codecs, targeted for the
          called party. If the called party accepts, the caller launches an
          INVITE to one or two Relay services (depending on whether the
          hearing impaired caller is able to communicate verbally or not).
          The INVITE to speech to text translation service contains SDP
          where the caller would like to receive the text; the response
          contains SDP that the caller places in the ACK to the called
          party. This connects the called party with the speech to text
          translator, with the resultant text being sent to the caller. If
          text to speech service is also needed, the caller places the SDP
          it received in the 200 OK from the called party into an INVITE to
          the translator. The response contains SDP with an address where
          the caller can send text.
          Figure 4 shows a call flow using only speech to text translation
                   |                       |                         |
                   |   F1: INVITE no SDP   |                         |
                   | --------------------> |                         |
                   |                       |                         |
                   |   F2: 200 OK  SDP1    |                         |
                   | <-------------------- |                         |
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 12 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
                   |                       |                         |
                   |                       |                         |
                   |     F3: INVITE        |                         |
                   | ----------------------------------------------> |
                   |                       |                         |
                   |                       | F4: 200 OK SDP2         |
                   | <---------------------------------------------- |
                   |                       |                         |
                   |     F5: ACK           |                         |
                   | ----------------------------------------------> |
                   |                       |                         |
                   |                       |                         |
                   |  F6: ACK SDP2         |                         |
                   | --------------------> |                         |
                   |                       |                         |
                   |                       |                         |
                   |   RTP (speech)        |                         |
                   |======================>|   RTP (speech)          |
                   |                       |========================>|
                   |   RTP (text)          |                         |
                   |                       |                         |
                   |                       |                         |
                   |                       |                         |
                   |                       |                         |
                   |                       |                         |
                   |                       |                         |
                Hearing                 Called                  Speech to
                Impaired                Party                 Text(STT)Relay
             Figure 4: Hearing Impaired Caller Call Flow
       7. General Requirements
          It is desired among the hearing impaired user community that a
          relay service to automatically be enabled depending on the pre-set
          user preferences. This enables avoiding the inconvenience of
          current relay services that requires a caller to call the relay
          service first and then having the relay call the called party.
          The above described call flows are only to illustrate the
          possibilities and are not considered final call flow designs. It
          may be a good idea work out the call flows for Relays calls, so
          that any IP relay service such as 3G wireless can expect the pre-
          defined scenarios.  To aid the design of those scenarios, we could
          identify a number of general requirements that relay service
          providers and terminal manufacturers SHOULD adhere to. Keep also
          in mind that in some countries, relay services are NOT for free as
          in other. Another problem in some countries is that the
          availability of relay centers is severely limited. This will be
          reflected in some of the requirements.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 13 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          The relay enabled terminal (from now on just named terminal) MUST
          be able to place calls transparently, the hearing impaired user
          does not have to call the relay center first and then tell the
          phone number to the operator who to call. It SHOULD be done
          automatically. The terminal will connect to the relay center and
          call the called party automatically. Very useful for ordering
          Pizza via the telephone for example. Placing a call should be as
          easy and comfortable for any user, including hearing and speech
          impaired users.
          The Relay Service providers and terminals MUST be able to
          interwork with each other regardless the Relay Service provider or
          the manufacturer of the (Relay enabled) terminal. This also means
          that when a hearing impaired user from the Netherlands calls a
          user located in the United States, that a Dutch Relay Provider or
          an American Relay Provider CAN be used without any problem during
          the call. This also means that modern terminals SHOULD be able to
          support the legacy relay protocols during the first years until
          all users have switched over to IP SIP based relay services.
          The terminal MUST be able to receive relay calls any time and from
          any location.  This capability is already included in SIP. The
          user preferences in the REGISTER will indicate what relay
          requirements are desired (at minimum text support MUST be
          supported). Upon logging in, the terminal SHOULD be able to
          automatically download all user settings.
          The terminal MUST be able to activate the Relay service ANYTIME
          during a call without interrupting the call. It MUST also be
          possible to disable the relay service during the call.
          The terminal MUST be able to set up user preferences easily to
          specify language, mode of relay (such as: SL/video to/from speech,
          text to/from video or speech, also as an extended service English
          to i.e. Spanish text, relay can cross language barriers if
          supported). This is pre-set, but it can be overruled by one button
          or via a short list with alternative options. The terminal and
          relay service provider SHOULD be able to enable/deliver services
          like a "real-time closed captioning" where the terminal receives
          the video/audio of a caller/called, but the relay center will
          translate the audio and display the text as subtitles. This will
          also offer possibilities like commercial translation service (via
          voice-over or text).
          The terminal MUST be able to receive the correct Caller-ID of the
          caller that calls the hearing impaired user via the relay, and not
          the relay's Caller-ID (note: this is the caller ID and not the SIP
          The terminal MUST be able to specify which pre-defined numbers
          require the use of relay service and which are direct calls (TTY
          to TTY for text, video to video for sign language) and do not
          require relay.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 14 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          The terminal MUST be able to download and upload the user settings
          and the address book, which should be stored as a database file at
          a centrally located server, which can be the relay service
          provider or the home provider of the user.
          It SHOULD be possible for a user to store user preferences and
          settings on a web service, similar to the "My Yahoo" type service,
          to allow the user access to his/her personal profile and services
          from any web-enabled device. In case of a mobile terminal user;
          the above described requirement SHOULD also be available via a
          wireless web service, where available.
          The terminal SHOULD be able to poll for the nearest Relay Proxy to
          reduce data traffic and reduce the cost of network usage. Also the
          terminal MUST be able to change relay service providers at any
          time (preferably via a menu with the relay service providers
          The terminal SHOULD be able to use the relay service provider list
          to sequence the relay service providers in a preferred position:
          which to use first for outgoing calls and automatically move on to
          the next in the list if it is busy or does not support the
          required services.
          Relay service providers SHOULD be able to advertise the services
          they have and update for new services, perhaps via a central
          (voluntary) registration or via dialing into an info number.  The
          terminal should be able to dial automatically to such info numbers
          or via the SIP INFO method. This would stimulate competition
          between relay service providers, which will lead to lower service
          prices and/or more different kind of services.
          Relay service providers SHOULD be able to act as an answering
          machine and provide (unified) message services. The terminal
          SHOULD have the capability to retrieve messages (answering machine
          mode). The terminal SHOULD have a pre-configurable setting that
          automatically connects a calling party to the relay service
          provider for answering machine service when the user does not want
          to receive incoming calls. This happens transparently without
          extra handling of the call (assumed that the called user is
          subscribed to such a service).
          Voice messages left for the hearing impaired user MUST be
          interpreted at the relay service provider into the format
          designated by the user (email, IM, video clip, etc.) and
          transmitted to the hearing impaired userÆs terminal on demand or
          when the terminal registers.  Note:  This "answering service"
          would also be a possible commercial service for hearing users. If
          a third party unified messaging service is used by the hearing
          impaired user; a specific relay service provider SHOULD be able to
          be authorized to access the unified message box and translate all
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 15 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          audio messages into a pre-selected form. For example voice to text
          (for fax or e-mail or IM) or voice to video for sign language.
          The terminal MUST be able to distinguish non-relay calls from
          relay calls and direct TTY calls, if this number is NOT listed in
          the address book (which stores the number and options for direct
          terminal to terminal calls such as TTY to TTY, which do not
          require a relay). A message will be sent to the originator of the
          call (183 Hearing impaired user RELAY call only).  Depending on
          technological advances and the user's preferences, instead of the
          terminal returning a 183 message, the terminal MAY connect to the
          relay service provider and accept the call as if there is NO relay
          service provider in between.
          The Relay Service provider SHOULD be able to offer the STEALTH
          mode; invisible relay services for the hearing impaired caller, in
          this way the user can conceal his/her hearing impairment).
          This may be seen as a separate service and possibly charged extra
          due the speed of accepting the call (minimal delay on picking-up)
          and extra effort to be invisible to the caller. Special hardware
          and software may be required for computer run speech to text
          conversion (this may require specialized relay service providers).
          Note: this service can be used selectively for certain callers,
          and other callers are just notified by the 183 message.
          The terminal and the Relay Service Provider MUST be able to allow
          calls to be placed anonymously, so that the called user cannot see
          who is calling. The Relay Service provider MUST in this case keep
          the identity of the calling user confidential as described in 7.
          When starting a relay session; the Relay Service Provider MUST
          show the charges of the relaying service per minute. The user MUST
          be able to refuse the service if the charges are too high or opt
          for a cheaper service (use text instead of the video).
          An independent payment service provider SHOULD be used, so that
          any relay service provider is assured of payment of the charges.
          This requires that the hearing impaired users use some form of
          subscription for relay services that require extra features. This
          leaves room for subsidizing the hearing impaired users, for
          example a monthly pre-paid amount will be deposited to the relay
          account. If a user uses more then the pre-paid amount, the user
          will be billed for the additional charges.
          Wireless telephony systems (GSM, UMTS, CDMA2000, cellular systems
          etc) MUST enable the wireless phones to connect to relay service
          providers. This can be done directly via wireless IP connections
          OR via a special relay node. This node MUST be able to translate
          circuit switched (GSM, etc) relay into IP relay.
          The Relay Service provider MAY also offer remote interpreting, the
          terminal MAY be modified to enabled this service. This is
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 16 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          desirable in countries experiencing a shortage of Interpreters.
          And there are many situations where it just requires a short time
          of interpreting.
       7. Security Considerations
          Because an interpreter is generally required when a hearing
          impaired individual has a conversation with a hearing individual,
          whether in person or using a medium such as the telephone,
          interpreters are privy to a great deal of private information.
          For this reason, both the Deaf and interpreter communities are
          vitally interested in the ethics and professionalism of the
          interpreter.  For example, interpreters in the United States that
          are certified by the organizations recognized by the American Deaf
          Community, such as the National Association of the Deaf (NAD[16])
          and the Registry of Interpreters for the Deaf (RID[17]), are
          required as part of their certification to support the Codes of
          Ethics of their organizations. Other countries may also have
          pertinent legislation.
          Hence these requirements:
          All relay operators and other interpreters or organizations
          involved in relaying calls SHALL be required to subscribe to a
          generally accepted Code of Ethics for interpreters. As an example,
          the Code of Ethics required for membership in RID is as follows:
          The Registry of Interpreters for the Deaf, Inc. has set forth the
          following principles of ethical behavior to protect and guide
          interpreters and transliterators and hearing and deaf consumers.
          Underlying these principles is the desire to insure for all the
          right to communicate.
          This Code of Ethics applies to all members of the Registry of
          Interpreters for the Deaf, Inc. and to all certified non-members.
          Interpreters/transliterators shall keep all assignment-related
          information strictly confidential.
          Interpreters/transliterators shall render the message faithfully,
          always conveying the content and spirit of the speaker using
          language most readily understood by the person(s) whom they serve.
          Interpreters/transliterators shall not counsel, advise or
          interject personal opinions.
          Interpreters/transliterators shall accept assignments using
          discretion with regard to skill, setting, and the consumers
          Interpreters/transliterators shall request compensation for
          services in a professional and judicious manner.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 17 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          Interpreters/transliterators shall function in a manner
          appropriate to the situation.
          Interpreters/transliterators shall strive to further knowledge and
          skills through participation in workshops, professional meetings,
          interaction with professional colleagues, and reading of current
          literature in the field.
          Interpreters/transliterators, by virtue of membership or
          certification by the RID, Inc., shall strive to maintain high
          professional standards in compliance with the Code of Ethics.
          This Code of Ethics is widely accepted and supported as a standard
          within the American Deaf community and the American community of
          interpreters.  More information on RID can be found from the
          organizationÆs web site, see reference [16].
          In addition to the standards required for individual relay
          operators, interpreters, and companies that provide relay
          services, the actual transmissions MUST be secured.
          An extension to the requirement for "STEALTH": relay
          operators/interpreters MAY act on behalf of the user, at the
          request of the user.  For example, the user can ask the operator
          to call a company to file a complaint.  This requires
          confidentially and an extension to the usual role of an
       9. References and Footnotes
          [1] M. Handley, H. Schulzrinne, E. Schooler, and J. Rosenberg,
          "SIP: session initiation protocol," Request for Comments 2543,
          Internet Engineering Task Force, Mar. 1999.
          [2] M. Handley and V. Jacobson. "SDP: Session Description
          Protocol." Request for Comments 2327, Internet Engineering Task
          Force,April 1998.
          [3] H. Schulzrinne and J. Rosenberg, "SIP caller preferences and
          callee capabilities," Internet Draft, Internet Engineering Task
          Force, June 2001. Work in progress.
          [4] J. Rosenberg, H. Schulzrinne, J. Peterson and G. Camarillo,
          "Third party call control in SIP," Internet Draft, Internet
          Engineering Task Force, Mar. 2001. Work in progress.
          [5] G. Hellstrom, "RTP payload for text conversation," Request for
          Comments 2793, Internet Engineering Task Force, May 2000.
          [6] J. Rosenberg, H. Schulzrinne and H. Sinnreich, "SIP Enabled
          Services to Support the Hearing Impaired" Internet Draft, Internet
          Engineering Task Force, July 2000.  Work in progress.
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 18 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          [7] C. Gearhart, A. Van Wijk and H. Sinnreich, "A Proposed Set of
          requirements for SIP Support for Deaf or Speech Impaired
          Customers" Internet Draft, Internet Engineering Task Force,
          November 2000.  Work in progress.
          [8] S. Bradner, "Key words for use in RFCs to indicate requirement
          levels".  Request for Comments 2119, Internet Engineering Task
          Force. March 1997.
          [9] TTY: an acronym for a text telephone device used by Deaf
          individuals to communicate via telephone systems;  commonly
          referred to as a TDD by the hearing community.
          [10] Bradner, S., "The Internet Standards Process -- Revision 3",
          BCP9, RFC 2026, October 1996.
          [11] Baker, Charlotte, and Robbin Battison.  "Sign Language and
          the Deaf Community:  Essays in Honor of William Stokoe".  National
          Association of the Deaf, June 1980.
          [12] Moore, Matthew, et al.  "For Hearing People Only:  Answers to
          Some of the Most Commonly Asked Questions About the Deaf
          Community, Its Culture, and the Deaf Reality".  MSM Productions
          Ltd., 2nd Edition, September 1993.
          [13] Padden, Carol, and Tom Humphries.  "Deaf in America: Voices
          from a Culture". Harvard University Press, Reprint September 1990.
          [14] Stokoe, William.  "Sign and Culture:  A Reader for Students
          of American Sign Language".  Linstok Press, June 1980.
          [15] J. Rosenberg, R. Sparks, D. Willis, B. Campbell, H.
          Schulzrinne, J. Lennox, C. Huitema, B. Aboba, D. Gurle and D.
          Oran, "SIP extensions for instant messaging," Internet Draft,
          Internet Engineering Task Force, April 2001.  Work in progress.
          [16] National Association of the Deaf.  A national organization
          of, for, and operated by Americans who are Deaf or deaf.
          Organized in 1880, it is "the oldest and largest organization
          representing people with disabilities in the United States.  The
          NAD safeguards the accessibility and civil rights of 28 million
          deaf and hard of hearing Americans in a variety of areas including
          education, employment, health care and social services, and
          telecommunications. A private, non-profit 501(c)(3) organization,
          the NAD is a dynamic federation of 51 state association
          affiliates, sponsoring and organizational affiliates, and direct
          members."  See "www.nad.org" for more information on this
          [17] Registry of Interpreters for the Deaf (RID).  A national,
          professional organization of interpreters and transliterators for
          the Deaf in America.  "The philosophy of RID is that excellence in
          the delivery of interpretation and transliteration services among
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 19 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          people who are Deaf, or Hard of Hearing, and people who are
          hearing, will ensure effective communication. As the professional
          association for interpreters and transliterators, the RID serves
          as an essential arena for its members in their pursuit of
          excellence.  It is the mission of the Registry of Interpreters for
          the Deaf, Inc., to provide international, national, regional,
          state, and local forums and an organizational structure for the
          continued growth and development of the professions of
          interpretation and transliteration of American Sign Language and
          English."   See "www.rid.org" for detailed information on this
       10  Acknowledgments
          The authors would like to thank the following deaf individuals,
          professional interpreters, and others who have contributed to the
          development of this document:
          Vint Cerf/WCOM.
          Teresa Hastings/WCOM.
          Charles Estes/WCOM.
          Helene Cohen-Gilbert, Coordinator, Collin County Community
          College, Texas, USA, Interpreter Preparation Program û Deaf.
          Nathan Charlton, Royal National Institute for Deaf People (RNID),
          London, United Kindom.
          Grant Laird.
          Brenden Gilbert.
       11. Author's Addresses
          Arnoud van Wijk
          Ericsson EuroLab Netherlands BV
          P.O. Box 8
          5120 AA Rijen
          The Netherlands
          Fax: +31-161-247569
          email: Arnoud.van.Wijk@eln.ericsson.se
          Jonathan Rosenberg
          72 Eagle Rock Avenue
          First Floor
          East Hanover, NJ 07936
          email: jdrosen@dynamicsoft.com
          Cathy Gearhart
          Ericsson, Inc.
          P.O. Box 833675, M/S L-04
          Richardson, TX 75083-3875
          email: cathy.gearhart@ericsson.com
          Henry Sinnreich
          400 International Parkway
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 20 of 21]
       SIP for Hearing and Speech Impaired Customers                July 2001
          Richardson, Texas 75081
          email: henry.sinnreich@wcom.com
          Henning Schulzrinne
          Columbia University
          M/S 0401
          1214 Amsterdam Ave.
          New York, NY 10027-7003
          email: schulzrinne@cs.columbia.edu
       Full Copyright Statement
          "Copyright (C) The Internet Society 2001. All Rights Reserved.
          This document and translations of it may be copied and furnished
          to others, and derivative works that comment on or otherwise
          explain it or assist in its implmentation may be prepared, copied,
          published and distributed, in whole or in part, without
          restriction of any kind, provided that the above copyright notice
          and this paragraph are included on all such copies and derivative
          works. However, this document itself may not be modified in any
          way, such as by removing the copyright notice or references to the
          Internet Society or other Internet organizations, except as needed
          for the purpose of developing Internet standards in which case the
          procedures for copyrights defined in the Internet Standards
          process must be followed, or as required to translate it into
          languages other than English. The limited permissions granted
          above are perpetual and will not be revoked by the Internet
          Society or its successors or assigns.
          This document and the information contained herein is provided on
       van Wijk/Rosenberg/Gearhart/Sinnreich/Schulzrinne      [Page 21 of 21]