[Search] [txt|pdfized|bibtex] [Tracker] [Email] [Diff1] [Diff2] [Nits]
Versions: 00 01                                                         
Network Working Group                                           J. Maloy
Internet-Draft                                                  Ericsson
Expires: April 27, 2005                                         S. Blake
                                                              Modularnet
                                                               M. Koning
                                                               WindRiver
                                                           J. Hadi Salim
                                                                    Znyx
                                                             H. Khosravi
                                                                   Intel
                                                        October 27, 2004


   TIPC: Transparent Inter Process Communication Protocol,  a Layer 2
                      TML for the ForCES protocol
                        draft-maloy-tipc-01.txt

Status of this Memo

   This document is an Internet-Draft and is subject to all provisions
   of section 3 of RFC 3667.  By submitting this Internet-Draft, each
   author represents that any applicable patent or other IPR claims of
   which he or she is aware have been or will be disclosed, and any of
   which he or she become aware will be disclosed, in accordance with
   RFC 3668.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on April 27, 2005.

Copyright Notice

   Copyright (C) The Internet Society (2004).

   Copyright (C) The Internet Society (2004).  This document is subject



Maloy, et al.            Expires April 27, 2005                 [Page 1]


Internet-Draft                    TIPC                      October 2004


   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights."

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

Abstract

   This document describes TIPC, a protocol intended to be used as a TML
   (Transport Mapping Layer) for the ForCES protocol[ForCES] when that
   protocol is transported over L2 carriers such as Ethernet, RapidIO or
   PCI-Express.  TIPC is specially designed for efficient communication
   within clusters of loosely coupled nodes, and may as such even be
   used outside the context of being a ForCES protocol carrier.  It
   would even be an excellent candidate as a ForCES pre-association
   phase setup protocol.

   TIPC is a reliable transport protocol typically operating on top of
   L2 packet networks, but it should also work well on higher-level
   protocols such as DCCP, TCP, or SCTP.

   TIPC offers the following services to its users:
   o  A functional addressing scheme providing full addressing
      transparency over the whole cluster.
   o  A topology information and subscription service, providing
      up-to-date information about functional and physical topology.
   o  Lightweight, highly reactive connections reporting errors or
      destination unreachability within a fraction of a second.
   o  A reliable multicast service, based on functional addressing, but
      using the underlying network multicast service when possible.
   o  Acknowledged, loss-free, error-free, non-duplicated transfer of
      user data, both in connectionless and connection-oriented mode.
   o  Configurable congestion control both at bearer, link, and
      connection level.
   o  Data fragmentation conforming to discovered carrier MTU size.
   o  Bundling of multiple user messages into a single TIPC packet in
      situations where messages cannot be sent immediately.



Maloy, et al.            Expires April 27, 2005                 [Page 2]


Internet-Draft                    TIPC                      October 2004


   o  Transparent, link-level load sharing and redundancy, through
      support of heterogeneous multi-homing.
   o  A slim, non-layered protocol header allowing efficient protocol
      implementations.

   Apart from common process-to-process communication, the design of
   TIPC even includes the possibibily to commmunicate process-to-kernel
   and kernel-to-kernel, still with full addressing and interface
   transparency.

Table of Contents

   1.   Introduction . . . . . . . . . . . . . . . . . . . . . . . .   6
     1.1  Motivation . . . . . . . . . . . . . . . . . . . . . . . .   6
       1.1.1  Existing Protocols . . . . . . . . . . . . . . . . . .   6
       1.1.2  Assumptions  . . . . . . . . . . . . . . . . . . . . .   7
     1.2  Architectural View . . . . . . . . . . . . . . . . . . . .   8
     1.3  Functional View  . . . . . . . . . . . . . . . . . . . . .   9
       1.3.1  API Adapters . . . . . . . . . . . . . . . . . . . . .  10
       1.3.2  Address Subscription . . . . . . . . . . . . . . . . .  10
       1.3.3  Address Distribution . . . . . . . . . . . . . . . . .  10
       1.3.4  Address Translation  . . . . . . . . . . . . . . . . .  10
       1.3.5  Multicast  . . . . . . . . . . . . . . . . . . . . . .  10
       1.3.6  Connection Supervision . . . . . . . . . . . . . . . .  11
       1.3.7  Routing and Link Selection . . . . . . . . . . . . . .  11
       1.3.8  Neighbour Detection  . . . . . . . . . . . . . . . . .  11
       1.3.9  Link Establishment/Supervision . . . . . . . . . . . .  11
       1.3.10   Link Failover  . . . . . . . . . . . . . . . . . . .  11
       1.3.11   Fragmentation/Defragmentation  . . . . . . . . . . .  11
       1.3.12   Bundling . . . . . . . . . . . . . . . . . . . . . .  11
       1.3.13   Congestion Control . . . . . . . . . . . . . . . . .  12
       1.3.14   Sequence and Retransmission Control  . . . . . . . .  12
       1.3.15   Bearer Layer . . . . . . . . . . . . . . . . . . . .  12
     1.4  Terminology  . . . . . . . . . . . . . . . . . . . . . . .  12
       1.4.1  ForCES Terminolgy  . . . . . . . . . . . . . . . . . .  12
       1.4.2  TIPC Specific Terminolgy . . . . . . . . . . . . . . .  13
     1.5  Abbreviations  . . . . . . . . . . . . . . . . . . . . . .  14
   2.   Mapping ForCES/PL to TIPC/TML  . . . . . . . . . . . . . . .  16
     2.1  Fulfilment of TML Requirements . . . . . . . . . . . . . .  16
     2.2  Address Mapping  . . . . . . . . . . . . . . . . . . . . .  17
   3.   TIPC Features  . . . . . . . . . . . . . . . . . . . . . . .  22
     3.1  Network Topology . . . . . . . . . . . . . . . . . . . . .  22
       3.1.1  Network  . . . . . . . . . . . . . . . . . . . . . . .  22
       3.1.2  Zone . . . . . . . . . . . . . . . . . . . . . . . . .  23
       3.1.3  Cluster  . . . . . . . . . . . . . . . . . . . . . . .  23
       3.1.4  Node . . . . . . . . . . . . . . . . . . . . . . . . .  23
       3.1.5  Secondary Node . . . . . . . . . . . . . . . . . . . .  23
     3.2  Addressing . . . . . . . . . . . . . . . . . . . . . . . .  24



Maloy, et al.            Expires April 27, 2005                 [Page 3]


Internet-Draft                    TIPC                      October 2004


       3.2.1  Location Transparency  . . . . . . . . . . . . . . . .  24
       3.2.2  Network Address  . . . . . . . . . . . . . . . . . . .  24
       3.2.3  Port Identity  . . . . . . . . . . . . . . . . . . . .  24
       3.2.4  Port Name  . . . . . . . . . . . . . . . . . . . . . .  24
       3.2.5  Port Name Sequence . . . . . . . . . . . . . . . . . .  25
       3.2.6  Multicast Addressing . . . . . . . . . . . . . . . . .  26
       3.2.7  Publishing Scope . . . . . . . . . . . . . . . . . . .  27
       3.2.8  Lookup Policies  . . . . . . . . . . . . . . . . . . .  27
       3.2.9  Name Translation . . . . . . . . . . . . . . . . . . .  28
       3.2.10   Distributed Naming Table . . . . . . . . . . . . . .  28
     3.3  Topology Services  . . . . . . . . . . . . . . . . . . . .  29
       3.3.1  Inquiry  . . . . . . . . . . . . . . . . . . . . . . .  29
       3.3.2  Subscriptions  . . . . . . . . . . . . . . . . . . . .  30
       3.3.3  Functional Topology  . . . . . . . . . . . . . . . . .  31
       3.3.4  Physical Topology  . . . . . . . . . . . . . . . . . .  31
     3.4  Ports  . . . . . . . . . . . . . . . . . . . . . . . . . .  32
       3.4.1  Port State Machine . . . . . . . . . . . . . . . . . .  32
     3.5  Connections  . . . . . . . . . . . . . . . . . . . . . . .  36
       3.5.1  Connection Setup . . . . . . . . . . . . . . . . . . .  36
       3.5.2  Connection Shutdown  . . . . . . . . . . . . . . . . .  37
       3.5.3  Connection Abortion  . . . . . . . . . . . . . . . . .  39
       3.5.4  Connection Supervision . . . . . . . . . . . . . . . .  40
       3.5.5  Flow Control . . . . . . . . . . . . . . . . . . . . .  42
       3.5.6  Sequentiality Check  . . . . . . . . . . . . . . . . .  42
     3.6  Neighbour Detection  . . . . . . . . . . . . . . . . . . .  43
       3.6.1  Link Requests  . . . . . . . . . . . . . . . . . . . .  43
       3.6.2  Inter-Cluster Link Setup . . . . . . . . . . . . . . .  43
       3.6.3  Multicast Link Setup . . . . . . . . . . . . . . . . .  47
     3.7  Links  . . . . . . . . . . . . . . . . . . . . . . . . . .  48
       3.7.1  Link Activation  . . . . . . . . . . . . . . . . . . .  49
       3.7.2  Link Continuity Check  . . . . . . . . . . . . . . . .  51
       3.7.3  Sequence Control and Retransmission  . . . . . . . . .  51
       3.7.4  Message Bundling . . . . . . . . . . . . . . . . . . .  52
       3.7.5  Message Fragmentation  . . . . . . . . . . . . . . . .  52
       3.7.6  Link Congestion Control  . . . . . . . . . . . . . . .  53
       3.7.7  Bearer Congestion Control  . . . . . . . . . . . . . .  53
       3.7.8  Link Load Sharing vs Active/Standby  . . . . . . . . .  54
       3.7.9  Link Changeover  . . . . . . . . . . . . . . . . . . .  54
     3.8  Routing  . . . . . . . . . . . . . . . . . . . . . . . . .  56
       3.8.1  Routing Algorithm  . . . . . . . . . . . . . . . . . .  56
       3.8.2  Routing Table  . . . . . . . . . . . . . . . . . . . .  57
       3.8.3  Routing Table Updates  . . . . . . . . . . . . . . . .  57
     3.9  Multicast Transport  . . . . . . . . . . . . . . . . . . .  58
       3.9.1  Conditional Cluster Broadcast  . . . . . . . . . . . .  58
       3.9.2  Conditional Tunneled Retransmission  . . . . . . . . .  59
       3.9.3  Piggybacked Acknowledge  . . . . . . . . . . . . . . .  60
       3.9.4  Coordinated Acknowledge Interval . . . . . . . . . . .  61
       3.9.5  Replicated Delivery  . . . . . . . . . . . . . . . . .  61



Maloy, et al.            Expires April 27, 2005                 [Page 4]


Internet-Draft                    TIPC                      October 2004


       3.9.6  Congestion Control . . . . . . . . . . . . . . . . . .  61
     3.10   Fault Handling . . . . . . . . . . . . . . . . . . . . .  61
       3.10.1   Fault Avoidance  . . . . . . . . . . . . . . . . . .  62
       3.10.2   Fault Detection  . . . . . . . . . . . . . . . . . .  63
       3.10.3   Fault Recovery . . . . . . . . . . . . . . . . . . .  63
       3.10.4   Overload Protection  . . . . . . . . . . . . . . . .  63
   4.   TIPC Packet Format . . . . . . . . . . . . . . . . . . . . .  65
     4.1  TIPC Payload Message Header  . . . . . . . . . . . . . . .  65
       4.1.1  Payload Message Header Format  . . . . . . . . . . . .  65
       4.1.2  Payload Message Header Field Descriptions  . . . . . .  66
       4.1.3  Payload Message Header Size  . . . . . . . . . . . . .  70
     4.2  TIPC Internal Header . . . . . . . . . . . . . . . . . . .  71
       4.2.1  Internal Message Header Format . . . . . . . . . . . .  71
       4.2.2  Internal Message Header Fields Description . . . . . .  72
     4.3  Message Users  . . . . . . . . . . . . . . . . . . . . . .  76
       4.3.1  Broadcast Protocol . . . . . . . . . . . . . . . . . .  76
       4.3.2  Message Bundler Protocol . . . . . . . . . . . . . . .  77
       4.3.3  Link State Maintenance Protocol  . . . . . . . . . . .  77
       4.3.4  Connection Manager . . . . . . . . . . . . . . . . . .  78
       4.3.5  Routing Table Update Protocol  . . . . . . . . . . . .  79
       4.3.6  Link Changeover Protocol . . . . . . . . . . . . . . .  80
       4.3.7  Name Table Update Protocol . . . . . . . . . . . . . .  80
       4.3.8  Message Fragmentation Protocol . . . . . . . . . . . .  82
       4.3.9  Neighbour Detection Protocol . . . . . . . . . . . . .  82
     4.4  Media Adapter Protocols  . . . . . . . . . . . . . . . . .  88
       4.4.1  Ethernet Adaptation  . . . . . . . . . . . . . . . . .  88
   5.   Management . . . . . . . . . . . . . . . . . . . . . . . . .  89
     5.1  Command Types  . . . . . . . . . . . . . . . . . . . . . .  89
     5.2  Command Message Formats  . . . . . . . . . . . . . . . . .  89
       5.2.1  Command Messages . . . . . . . . . . . . . . . . . . .  89
       5.2.2  Command Response Messages  . . . . . . . . . . . . . .  90
     5.3  Commands . . . . . . . . . . . . . . . . . . . . . . . . .  91
       5.3.1  Group 1: Query Commands  . . . . . . . . . . . . . . .  91
       5.3.2  Group 2: Manipulating Commands . . . . . . . . . . . . 103
       5.3.3  Group 3: Subscriptions . . . . . . . . . . . . . . . . 107
   6.   Security . . . . . . . . . . . . . . . . . . . . . . . . . . 110
   7.   References . . . . . . . . . . . . . . . . . . . . . . . . . 110
        Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 111
        Intellectual Property and Copyright Statements . . . . . . . 113












Maloy, et al.            Expires April 27, 2005                 [Page 5]


Internet-Draft                    TIPC                      October 2004


1.  Introduction

   This section explains the rationale behind the development of the
   Telecom Inter Process Communication (TIPC) protocol.  It also gives a
   brief introduction to the services provided by this protocol, as well
   as the basic concepts needed to understand the further description of
   the protocol in this document.

1.1  Motivation

   There are no standard protocols available today that fully satisfy
   the special needs of application programs working within highly
   available, dynamic cluster environments.  Clusters may grow or shrink
   by orders of magnitude, having member nodes crashing and restarting,
   having routers failing and replaced, having functionality moved
   around due to load balancing considerations, etc.  All this must be
   possible to handle without significant disturbances of the service
   offered by the cluster.  In order to minimize the effort by the
   application programmers to deal with such situations, and to maximize
   the chance that they are handled in a correct and optimal way, the
   cluster internal communication service should provide special support
   helping the applications to adapt to changes in the cluster.  It
   should also, when possible, leverage the special conditions present
   within cluster environments to present a more efficient and more
   fault-tolerant communication service than more general protocols are
   capable of.  This is the purpose of TIPC.

   Version 1 of TIPC has been widely deployed in customer networks.
   This document describes version 2 of TIPC.  An open source
   implementation of version 2 is available at [TIPC]

1.1.1  Existing Protocols

   TCP [RFC793] has the advantage of being ubiquitous, stable, and
   wellknown by most programmers.  Its most significant shortcomings in
   a real-time cluster environment are the following:
   o  It lacks any notion of functional addressing and addressing
      transparency.  Mechanisms exist (DNS, CORBA Naming Service) for
      transparent and dynamic lookup of the correct IP-adress of a
      destination, but those are in general too static and expensive to
      use.
   o  TCP has non-optimal performance, especially for intra-node
      communication and for short messages in general.  For intra-node
      communication there are other and more efficient mechanisms
      available, at least on Unix, but then the location of the
      destination process has to be assumed, and can not be changed.  It
      is desirable to have a protocol working efficiently for both
      intra-node and inter-node messaging, without forcing the user to



Maloy, et al.            Expires April 27, 2005                 [Page 6]


Internet-Draft                    TIPC                      October 2004


      distinguish between these cases in his code.
   o  The rather heavy connection setup/shutdown scheme of TCP is a
      disadvantage in a dynamic environment.  The minimum number of
      packets exchanged for even the shortest TCP transaction is nine
      (SYN, SYNACK etc.), while with TIPC this can be reduced to two, or
      even to one if connectionless mode is used.
   o  The connection-oriented nature of TCP makes it impossible to
      support true multicast.

   SCTP [RFC2960] is message oriented, it provides some level of user
   connection supervision, message bundling, loss-free changeover, and a
   few more features that may make it more suitable than TCP as an
   intra-cluster protocol.  Otherwise, it has all the drawbacks of TCP
   already listed above.

   Apart from these weaknesses, neither TCP nor SCTP provide any
   topology information/subscription service, something that has proven
   very useful both for applications and for management functionality
   operating within cluster environments.

   Both TCP and SCTP are general purpose protocols, in the sense that
   they can be used safely over the Internet as well as within a closed
   cluster.  This virtual advantage is also their major weakness: they
   require funtionality and header space to deal with situations that
   will never happen, or only infrequently, within clusters.

1.1.2  Assumptions

   TIPC [TIPC] has been designed based on the following assumptions,
   empirically known to be valid within most clusters.
   o  Most messages cross only one direct hop.
   o  Transfer time for most messages is short.
   o  Most messages are passed over intra-cluster connections.
   o  Packet loss rate is normally low; retransmission is infrequent.
   o  Available bandwidth and memory volume is normally high.
   o  For all relevant bearers packets are check-summed by hardware.
   o  The number of inter-communicating nodes is relatively static and
      limited at any moment in time.
   o  Security is a less crucial issue in closed clusters than on the
      Internet.

   Because of the above one can use a simple, traffic-driven, fixed-size
   sliding window protocol located at the signalling link level, rather
   than a timer-driven transport level protocol.  This in turn gives a
   lot of other advantages, such as earlier release of transmission
   buffers, earlier packet loss detection and retransmission, earlier
   detection of node unavailability, only to mention some.  Of course,
   situations with long transfer delays, high loss rates, long messages,



Maloy, et al.            Expires April 27, 2005                 [Page 7]


Internet-Draft                    TIPC                      October 2004


   security issues, etc.  must be dealt with as well, but rather from
   the viewpoint of being exceptions than as the general rule.

1.2  Architectural View

   TIPC should be seen as a layer between the TIPC user, the ForCES
   protocol, and a packet transport  service such as Ethernet, ATM,
   DCCP, TCP, or SCTP.  The latter are denoted by  the generic term
   "bearer service" or just "bearer" throughout this document.

   TIPC provides reliable transfer of user messages between TIPC users,
   or more specifically between two TIPC ports, which are the endpoints
   of all TIPC communication.  A TIPC user normally means a user
   process, but may also be a kernel-level function or a driver, for
   which a specific interface has been defined.

   Described by standard terminology TIPC spans the level of transport,
   network, and signalling link layers, although this does not inhibit
   it from using another transport level protocol as bearer, so that
   e.g.  an SCTP association may serve as bearer for a TIPC signalling
   link.



     -------------                                      -------------
    |  ForCES/PL  |                                    |  ForCES/PL  |
    |  Layer      |                                    |  Layer      |
    |-------------|                                    |-------------|
    |             |                                    |             |
    |  TIPC/TML   |TIPC address            TIPC address|  TIPC/TML   |
    |             |                                    |             |
    |-------------|                                    |-------------|
    |   L2 Bearer |Bearer address  \/    Bearer address|   L2 Bearer |
    |   Service   |                /\                  |   Service   |
     -------------|                                    |-------------
           Node A |<--------- Bearer Transport ------->|  Node B


                  Figure 1: Architectural view of TIPC












Maloy, et al.            Expires April 27, 2005                 [Page 8]


Internet-Draft                    TIPC                      October 2004


1.3  Functional View

   Functionally TIPC can be described as consisting of several layers
   performing different tasks, as shown in Figure 2.  It must be
   emphasized that this layering reflects a functional model, not the
   way TIPC should be (or actually is) implemented.



                         TIPC User

    ----------------------------------------------------------
           -------------    -----------     -------------
          |   Socket    |  |  Port     |   |  Other API
          |   Adapter   |  |  Adapter  |   |  Adapters..
           -------------    -----------     -------------
    =========================================================
     ----------------------------
    | Address      |  Address    |
    | Subscription |  Resolution |
    |--------------+----------------------------------------
    | Address Table|        Connection Supervision          |
    | Distribution |        Routing/Link Selection          |
     -----------------------------------------------------+-
    |                   |  Neighbour Detection        |   | Node
    |     Multicast     |  Link Establish/Supervision |    ---------->
    |                   |  Link Failover              |     Internal
     -----------------------------------------------+-
    |      Fragmentation/Defragmentation      |     |
    |                                         |     |
     -----------------------------------------      |
    |               Bundling                  |     |
    |          Congestion Control             |     |
     -----------------------------------+-----      |
    |   Sequence/Retransmission  |      |           |
    |         Control            |      |           |
     -------+--------------+-----       |           |
    ========|==============|============|===========|========
            |              |            |           |
       -----V-----    -----V----    ----V-----    --V-------
     -|  Ethernet |  |   DCCP   |  |   SCTP   |  | Mirrored |
    | |           |  |          |  |          |  | Memory   |
    |  ---------+-    ----------    ----------    ----------
     -----------+


                   Figure 2: Functional view of TIPC




Maloy, et al.            Expires April 27, 2005                 [Page 9]


Internet-Draft                    TIPC                      October 2004


1.3.1  API Adapters

   TIPC makes no assumptions about which APIs should be used, except
   that they must allow access to the TIPC services.  It is possible to
   provide all functionality via a standard socket interface, an
   asynchronous port API, and any other form of dedicated interface that
   can be motivated.  In these layers there is also support for
   transport-level congestion and overload protection control.

1.3.2  Address Subscription

   The service "Topology Information and Subscription" provides the
   ability to interrogate and if necessary subscribe for the
   availability of a functional or physical address.  This helps the
   application to synchronize its startup, and may even serve as a
   simple, distributed event channel if used with care.

1.3.3  Address Distribution

   Functional addresses must be equally available within the whole
   cluster node.  In order for a message to reach its destination they
   must also at some stage be translated into a physical address.  For
   performance and fault tolerance reasons it is not acceptable to keep
   the necessary translation tables in one node, but rather TIPC must
   ensure that they are distributed to all nodes in the cluster, and
   that they are kept consistent at any time.  This is the task of the
   Address Distribution Service, also called Name Distribution Service.

1.3.4  Address Translation

   The translation from a functional to a physical address is performed
   on-the-fly during message sending by this functional layer.  It goes
   without saying that this step must use an efficient algorithm, but in
   many cases it can even be omitted altogether.  When it makes sense,
   the sender may choose to use a physical address instead, e.g.  a
   server responding to a connection setup request, or when
   communication is connection-oriented.

1.3.5  Multicast

   This layer, supported by the underlying three layers, provides a
   reliable intra-cluster broadcast service, typically defined as a
   semi-static multicast group over the underlying bearer.  It also
   provides the same features as an ordinary unicast link, such as
   message fragmentation, message bundling, and congestion control.






Maloy, et al.            Expires April 27, 2005                [Page 10]


Internet-Draft                    TIPC                      October 2004


1.3.6  Connection Supervision

   There are several mechanisms to ensure immediate detection and report
   of connection failure.

1.3.7  Routing and Link Selection

   This is the step of finding the correct destination node, and, if
   applicable, the correct next-hop router node, plus selecting the
   right link to use for reaching that node.  If the destination node
   turns out to be the own node, the rest of the stack is omitted, and
   the message is sent directly to the receiving port.

1.3.8  Neighbour Detection

   When a node is started it must make the rest of the cluster aware of
   its existence, and itself learn the topology of the cluster.  By
   default this is done by use of broadcast, but there are other methods
   available.

1.3.9  Link Establishment/Supervision

   Once a neighbouring node has been detected on a bearer, a signalling
   link is established towards it.  The functional state of that link
   has to be supervised continuously, and proper action taken if it
   fails.

1.3.10  Link Failover

   TIPC on a node will establish one link per-destination node and
   functional bearer instance, typically one per-configured ethernet
   interface.  Normally these will run in parallel and share load
   equally, but special care has to be taken during the transition
   period when a link comes up or goes down, to ensure the guaranteed
   cardinality and sequentiality of the message delivery.  This is done
   by this layer.

1.3.11  Fragmentation/Defragmentation

   When necessary TIPC fragments and reassembles messages that can not
   be contained within one MTU-size packet.

1.3.12  Bundling

   Whenever there is some kind of congestion situation, i.e.  when a
   bearer or a link can not immediately send a packet as requested, TIPC
   starts to bundle messages into packets already waiting to be sent.
   When the congestion abates the waiting packets are sent immediately,



Maloy, et al.            Expires April 27, 2005                [Page 11]


Internet-Draft                    TIPC                      October 2004


   and unbundled at the receiving node.

1.3.13  Congestion Control

   When a bearer instance becomes congested, e.g.  it is unable to
   accept more outgoing packets, all links on that bearer are marked as
   congested, and no more messages are attempted to be sent over those
   links until the bearer opens up again for traffic.  During this
   transition time messages are queued or bundled on the links, and then
   sent whenever the congestion has abated.  A similar mechanism is used
   when the send window of a link becomes full, but affects only that
   particular link.

1.3.14  Sequence and Retransmission Control

   This layer ensures the cardinality and sequentiality of packets over
   a link.

1.3.15  Bearer Layer

   This layer adapts to some connectionless or connection-oriented
   transport service, providing the necessary information and services
   to enable the upper layers to perform their tasks.

1.4  Terminology

1.4.1  ForCES Terminolgy

   o  ForCES Protocol: While there may be multiple protocols used within
      the overall ForCES architecture, the term "ForCES protocol" refers
      only to the protocol used at the Fp reference point in the ForCES
      Framework in RFC3746 [RFC3746].  This protocol does not apply to
      CE-to-CE communication, FE-to-FE communication, or to
      communication between FE and CE managers.  Basically, the ForCES
      protocol works in a master-slave mode in which FEs are slaves and
      CEs are masters.
   o  ForCES Protocol Layer (ForCES PL): A layer in ForCES protocol
      architecture that defines the ForCES protocol messages, the
      protocol state transfer scheme, as well as the ForCES protocol
      architecture itself (including requirements of ForCES TML (see
      below)).  Specifications of ForCES PL are defined by this
      document.
   o  ForCES Protocol Transport Mapping Layer (ForCES TML): A layer in
      ForCES protocol architecture that specifically addresses the
      protocol message transportation issues, such as how the protocol
      messages are mapped to different transport media (like TCP, IP,
      ATM, Ethernet, etc), and how to achieve and implement reliability,
      multicast, ordering, etc.  This document defines an L2/Ethernet



Maloy, et al.            Expires April 27, 2005                [Page 12]


Internet-Draft                    TIPC                      October 2004


      based ForCES TML.

1.4.2  TIPC Specific Terminolgy

   o  Port:  The endpoint of all user communication.  On Unix it
      typically takes the form of a socket.
   o  Zone:  A "super-cluster" of clusters interconnected via TIPC.
   o  Cluster: A part of a zone where all nodes are directly
      interconnected (fully meshed).
   o  Node: A physical computer within a cluster, identified by a TIPC
      address.
   o  System Node: A node having direct links to all other system nodes
      in the cluster, and a TIPC address defined within a certain range.
      When using the term 'node' in the remainder of this document we
      normally mean 'system node',unless the context makes a different
      interpretation obvious.
   o  Secondary Node: A node identified by a TIPC address within a
      certain range, and potentially having limited physical
      connectivity to the rest of the cluster.  Secondary nodes can
      communicate with all system nodes in the cluster, and vice versa,
      but the messages may have to pass via a system node acting as
      router.  Secondary nodes can not communicate with each other.
   o  Link: A signalling link connecting two nodes, performing tasks
      such as message transfer, sequence ordering, retransmission etc.
      A node pair may be interconnected by 1 or 2 parallel links, in
      load sharing or active/standby configuration.
   o  Bearer: A generic term for an instance of a physical or logical
      transport media, such as Ethernet, ATM/AAL or DCCP.
   o  Network Address: A TIPC internal node identifier.  It is in
      reality a 32 bit integer, subdivided into three fields (8/12/12),
      representing zone, cluster and node number respectively.  Normally
      depicted as <Z.C.N>.
   o  Network Identity: A TIPC internal identifier, used to keep
      different TIPC networks separated from each other, e.g.  on a LAN
      in a lab environment.
   o  Location transparency, sometimes called addressing tranparency, is
      the ability to let processes communicate within a cluster without
      either of them knowing the physical location of their peer.
   o  Port Name: (or just Name) A persistent functional address
      identifying a port within a zone.  A port may move between nodes
      while retaining its name.  For load sharing and redundancy
      purposes several ports may bind to the same name.
   o  Port Identity:  A volatile address identifying a unique physical
      port within a zone.  Once a physical port is deleted its identity
      will not be reused for a very long time.
   o  Connection:  A logical channel for passing messages between two
      ports.  Once a connection is established no address need be
      indicated when sending a message from any of the endpoints.  A



Maloy, et al.            Expires April 27, 2005                [Page 13]


Internet-Draft                    TIPC                      October 2004


      connection also implies automatic supervision of the endpoints'
      existence and state.
   o  Message Bundling: The act of bundling several messages into one
      bearer level packet, typically an Ethernet frame.  TIPC bundles
      messages e.g.  during media congestion.
   o  Message Fragmentation: Dividing a long message into several
      bearer-level packets, and reassembling the fragments at the
      receiving end.
   o  Message Forwarding: Ability to pass a received message on to a new
      destination port while pretending that the original sender port is
      the original sender.
   o  Link Failover: Moving all traffic from a failing link to the
      remaining link, while retaining original sequence order and
      cardinality.
   o  Naming Table: A TIPC internal table which keeps track of the
      mapping between port names and corresponding port identities.  It
      performs an on-the-fly translation from the one to the other
      during the message transfer phase.
   o  Message: The unit of data delivered from one user to another, i.e.
      between ports.
   o  Packet: The unit of data sent over a bearer.  It may contain one
      or more complete TIPC messages, as well as fragments of a message.
   o  Broadcast: The notion of sending a copy of the same message to all
      other nodes in the cluster.  Note that what is considered a
      broadcast from the TIPC viewpoint typically is mapped onto a
      multicast at the bearer (Ethernet or DCCP) level.
   o  Multicast: Sending a copy of the same message to muliple receivers
      by one user call.  In TIPC multicasts may be transferred both by
      broadcast and unicast between nodes, dependent of the number of
      identified receivers and the capabilities of the bearer layer.
   o  Unicast: Sending a message to one particular destination, i.e.
      over a TIPC link.
   o  Domain: A TIPC network addess designating a part of a TIPC
      network.  E.g., <Z.C.N> means the specific node with that address,
      <Z.C.0> any node within the specified cluster, and <Z.0.0> any
      node within the specified zone.  <0.0.0> means any node, anywhere
      within the network, except when it is used as Lookup Domain.
   o  Scope: A domain around a given node, as seen from that node.  E.g.
      <own_zone.own_cluster.own_node> or <own_zone.0.0>.

1.5  Abbreviations

   o  MAC    - Message Authentication Code [RFC2104]
   o  MTU    - Maximum Transmission Unit
   o  API    - Application Programming Interface
   o  RTT    - Round Trip Time, the elapsed time from the moment a
      message is sent to a destination to the moment it arrives back to
      the sender, provided the message is immediately bounced back from



Maloy, et al.            Expires April 27, 2005                [Page 14]


Internet-Draft                    TIPC                      October 2004


      the sender.  Typically on the order of 100 usecs,
      process-to-process, between 2 Ghz CPUs via a 100 Mbps Ethernet
      switch.
















































Maloy, et al.            Expires April 27, 2005                [Page 15]


Internet-Draft                    TIPC                      October 2004


2.  Mapping ForCES/PL to TIPC/TML

2.1  Fulfilment of TML Requirements

   o  Reliability: TIPC is a protocol guaranteeing sequential,
      loss-free, non-duplicated delivery of checksummed messages, as
      described in 3.7.3.  Whenever needed, each individual socket can
      be set to be "unreliable", meaning that all messages sent from
      that socket has the "drop"-bit (see 4.1.2) set.
   o  Security: For now, TIPC can only guarantee message and endpoint
      authenticity for closed networks, e.g.  a trusted LAN or bus.
      Since no router can yet forward TIPC/Ethernet packets it is
      impossible to inject spoofed packets into such a network.  How
      this should be handled when the nodes connected to the LAN or bus
      can not be trusted remains TBD.  The same is valid for message
      encryption.
   o  Congestion Control: TIPC provides three levels of congestion
      control, as described in in sections 3.5.5,3.7.6,3.7.7 and 3.9.6.
      The ForCES PL may receive indication of destination socket or node
      congestion when setting up a connection.  For established
      connections, socket congestion is handled transparently by the
      TIPC connection flow control scheme, while node congestion will
      result in connection abortion.  TIPC will also inform the PL layer
      about the reason for any connection abortion, such as node
      overload, node crash, or process crash.  When a connection, is
      aborted, the indication will be given to the PL immediately.  As
      connections require very few system resources, in particular
      regarding supervision timers, CE-FE connections can normally be
      established directly process to process, with no restraints on
      number of parallel connections.  Connections dedicated to traffic
      data transfer should be set to "non-reliable" in the FE-CE
      direction, to make it possible to fence off DoS attacks, while the
      CE-FE direction, as well as both directions of control data
      connections, should be established as "reliable".
   o  Uni/multi/broadcast: TIPC provides functional multicast, and
      broadcast as a special case of that, to the PL layer.  This
      function takes advantage of any broadcast transport facility in
      the bearer, such as Ethernet, and will use replicated unicast if
      this feature is missing, as with TCP.  Furtheremore, it is
      configurable when  L2 broadcast should be used, so that multicasts
      identified to have only a few destination nodes, as well as ditto
      retransmissions, in reality may be sent as replicated unicast.
   o  Timeliness: Messages are delivered without any delay whatsoever
      over L2 networks.  With Ethernet this will in practice mean a
      delivery time, process-to-process, in the order of 100
      microseconds of a typical one-packet message.  TIPC does not allow
      obsoleting messages.




Maloy, et al.            Expires April 27, 2005                [Page 16]


Internet-Draft                    TIPC                      October 2004


   o  HA considerations: L2 link failure detection and failover is
      handled transparently by TIPC, and does not affect the PL layer.
      If there is a complete communication failure between two nodes,
      the PL layer will be informed.  Any non-delivered messages will be
      returned to the sending PL, along with the failure reason, and it
      is up to the PL to handle such a situation as intelligently as
      possible.
   o  Encapsulations: The TIPC message formats are defined in section 4
      of this document.  There is no particular encapsulation
      distinguishing the PL layer from other users.
   o  TIPC provides four message importance priorities, instead of
      eight, as required in [ForCES].  However, the rationale for
      requiring as much as eigth levels seems weak; extensive experience
      from use of TIPC indicates that four levels is perfectly adequate.
      If it is decided that the ForCES PL must have eight levels, those
      will have to be mapped down 2-to-1 to the TIPC priorities.

2.2  Address Mapping

   [ForCES] decribes two address levels, the node level (CE/FE identity
   and multicast addresses), and the LFB level (type/instance tuple).
   These can easily be mapped down to the TIPC address concept of Port
   Name and Port Name Sequence.  The following example illustrates such
   a mapping:



























Maloy, et al.            Expires April 27, 2005                [Page 17]


Internet-Draft                    TIPC                      October 2004


        ----------------    ------------------
       | CE 7           |  | CE 8             |
       |                |  |                  |   | tml_bind(type=RSVP,
       |  ----------    |  |     ---------    |   |          inst=77)
       | |          |   |  |    |         |   |   |
       | | RSVP,66  |   |  |    |RSVP,77  |   | --v------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   | bind(type=RSVP,
       |  ----------    |  |     ---------    |   |      inst=77)
       |                |  |                  |   |
        ----------------    ------------------  --v------TIPC API


                                                --^------TIPC API
        ----------------    ------------------    |
       | FE 17          |  | FE 18            |   |  bind(type=Meter,
       |                |  |                  |   |       inst=44)
       |                |  |                  |   |
       |  ----------    |  |     ---------    | --^------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   |
       | | Shaper,33|   |  |    |Meter,44 |   |   |tml_bind(type=Meter,
       | |          |   |  |    |         |   |             inst=44)
       |  ----------    |  |     ---------    |
       |                |  |                  |
        ----------------    ------------------



            Figure 3: ForCES/PL to TIPC/TML address mapping

   An LFB wanting to send a message to a block in a CE would use the
   following call:


















Maloy, et al.            Expires April 27, 2005                [Page 18]


Internet-Draft                    TIPC                      October 2004


        ----------------    ------------------
       | CE 7           |  | CE 8             |
       |                |  |                  |
       |  ----------    |  |     ---------    |
       | |          |   |  |    |         |   |
       | | RSVP,66  |   |  |    |RSVP,77  |   |
       | |          |   |  |    |         <-------
       | |          |   |  |    |         |   |   |
       |  ----------    |  |     ---------    |   |
       |                |  |                  |   |
        ----------------    ------------------    |
                                                  |
                                                --^------TIPC API
        ----------------    ------------------    |
       | FE 17          |  | FE 18            |   |sendto(type=RSVP,
       |                |  |                  |   |       inst=77,
       |                |  |                  |   |       node=8)
       |  ----------    |  |     ---------    | --^------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   |
       | | Shaper,33|   |  |    |Meter,44 |   |   |tml_send(type=RSVP,
       | |          |   |  |    |         |   |             inst=77,
       |  ----------    |  |     ---------    |             CEID=8)
       |                |  |                  |
        ----------------    ------------------



                       Figure 4: FE-CE messaging

   Note that unless we coordinate the instance numbers for a block
   across the whole cluster, the constructed addresses are not
   guaranteed unique.  Such a coordination can be achieved either
   statically through configuration data, or dynamically through use of
   the TIPC topology subscription mechanism.  But this is not a
   prerequisite for this model to work, since the the current version of
   ForCES/PL layer anyway assumes that the sender knows the identity of
   the destination node.  However, the model becomes immensely more
   flexible if we can remove this assumption, and ensure that block
   addresses are cluster unique.  The message sender in the example
   above would never need to indicate the CEID, and would never need to
   be updated in case the configuration changes, e.g.  that RSVP number
   77 moves over to CE number 7.

   FE and CE Protocol Objects can be considered as just other LFB's and
   can connect to each other as in the next example.





Maloy, et al.            Expires April 27, 2005                [Page 19]


Internet-Draft                    TIPC                      October 2004


        ----------------    ------------------
       | CE 7           |  | CE 8             |
       |                |  |                  |   | tml_bind(type=1,
       |  ----------    |  |     ---------    |   |          inst=1)
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |ProtoObj |   | --v------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   | bind(type=1,
       |  ----------    |  |     ---------    |   |      inst=1)
       |                |  |                  |   |
        ----------------    ------------------  --v------TIPC API


                                                --^------TIPC API
        ----------------    ------------------    |
       | FE 17          |  | FE 18            |   |connect(type=1,
       |                |  |                  |   |        inst=1,
       |                |  |                  |   |        node=8)
       |  ----------    |  |     ---------    | --^------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |ProtoObj |   |   |tml_connect(type=1,
       | |          |   |  |    |         |   |                inst=1,
       |  ----------    |  |     ---------    |                CEID=8)
       |                |  |                  |
        ----------------    ------------------



                Figure 5: FE-CE Protocol Object connect

   Note that there is nothing stopping any LFB to establish its own
   connection to any block in a CE (or another FE).  Given that TIPC
   connections don't need individual heartbeating this should not be a
   problem, but whether this is desirable or anticipated by the authors
   of [ForCES] remains unclear.  Also note that FE/CE protocol objects
   don't need to start any heartbeating at all.  If they want a node
   failure detection time lower than the TIPC default value, they only
   need to configure (dynamically) the corresponding TIPC links
   accordingly.  Since TIPC does its heartbeating in driver mode,and
   also subscribes for low-level carrier failure detection, there is
   unlikely ForCES/PL can do this better.









Maloy, et al.            Expires April 27, 2005                [Page 20]


Internet-Draft                    TIPC                      October 2004


        ----------------    ------------------
       | CE 7           |  | CE 8             |
       |                |  |                  |   | tml_mcast(type=MeterMC,
       |  ----------    |  |     ---------    |   |           mc_group=X)
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |  RSPV   |   | --v------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   | mcast(type=MeterMC,
       |  ----------    |  |     ---------    |   |       lower=X,
       |                |  |                  |   |       upper=X)
        ----------------    ------------------  --v------TIPC API


                                                --^------TIPC API
        ----------------    ------------------    |
       | FE 17          |  | FE 18            |   |bind(type=MeterMC,
       |                |  |                  |   |     lower=X,
       |                |  |                  |   |     upper=X)
       |  ----------    |  |     ---------    | --^------TML API
       | |          |   |  |    |         |   |   |
       | |          |   |  |    |         |   |   |
       | |Meter,33  |   |  |    |Meter,44 |   |   |tml_join(type=MeterMC,
       | |          |   |  |    |         |   |             mc_group=X)
       |  ----------    |  |     ---------    |
       |                |  |                  |
        ----------------    ------------------



                       Figure 6: CE-FE Multicast

   Multicast addresses are mapped to TIPC port name sequences according
   to the figure above.  We have to assign a new type identity,"MeterMC"
   instead of "Meter" to avoid introducing a limitation: otherwise
   "ordinary" Meter instance numbers might collide with the value range
   of multicast addresses (see [ForCES]),and cause utter confusion.  Of
   course, the binding  of the multicast address can still be done to
   the same socket as the unicast address.













Maloy, et al.            Expires April 27, 2005                [Page 21]


Internet-Draft                    TIPC                      October 2004


3.  TIPC Features

3.1  Network Topology

   From a TIPC viewpoint the network is organized in a five-layer
   structure:



    ------------------------------------------------------  ----------
   | Zone <1>                                            | | Zone <2> |
   |  -----------------------    ----------------------  | |          |
   | | Cluster <1.1>         |  | Cluster <1.2>        | | |          |
   | |                       |  |                      | | |          |
   | |  -------              |  |  -------    -------  | | |          |
   | | |       |             |  | |       |  |       | | | |          |
   | | | Node  |             |  | | Node  +--+ Node  | | | |          |
   | | |<1.1.1>|    -------  |  | |<1.2.1>|  |<1.2.2>| | | |          |
   | | |       +---+       | |  | |       |  |       | | | |          |
   | |  ---+---    | Node  | |  |  --+----    -------  | | |          |
   | |     |       |<1.1.3>| |  |    |                 | | |          |
   | |  ---+---    |       | |  |  --+--               | | |          |
   | | |       +---+       | |  | |Seco.|              | | |          |
   | | | Node  |    -------  |  | |<1.2.|              | | |          |
   | | |<1.1.2>|             |  | |3333>|              | | |          |
   | | |       |             |  |  -----               | | |          |
   | |  -------              |  |                      | | |          |
   |  -----------------------    ----------------------  | |          |
   |                                                     | |          |
    -----------------------------------------------------   ----------


                    Figure 7: TIPC network topology


3.1.1  Network

   The top level is the TIPC network as such.  This is the ensemble of
   all zones interconnected via TIPC, i.e.  the domain where any node
   can reach any other node by using a TIPC network address.  The zones
   within such a network must be directly interconnected all-to-all via
   TIPC links, since there is no zone-level routing, i.e.  a message can
   not pass from one zone to another via an intermediate zone.  Any
   number of links between two zones is permitted, and normally there
   will be more than one for redundancy reasons.

   It is possible to create distinct, isolated networks, even on the
   same LAN, reusing the same network addresses, by assigning each



Maloy, et al.            Expires April 27, 2005                [Page 22]


Internet-Draft                    TIPC                      October 2004


   network a Network Identity.  This identity is not an address, and
   only serves the purpose of isolating networks from each other.
   Networks with different identities can not communicate with each
   other via TIPC.

3.1.2  Zone

   The next level in the hierarchy is the zone.  This is the largest
   scope of location transparency within a network, i.e.  the domain
   where a programmer does not need to worry about network addresses.
   The maximum number of zones in a network is 255, but this may be
   implementation dependent, and should be configurable.

3.1.3  Cluster

   The third level is the cluster.  Cluster nodes within a zone must be
   interconnected in a full mesh, but just as with zones, there is no
   need for fully meshed node links between clusters.  The maximum
   number of clusters within a zone is 4095, but this should be made
   configurable in the actual implementation.

3.1.4  Node

   The fourth level is the individual system node, or just node.  Nodes
   within a cluster must be interconnected all-to-all.  There may be up
   to 2047 system nodes in a cluster.

3.1.5  Secondary Node

   The fifth level is the secondary node.  Secondary nodes belong to a
   cluster, just like system nodes, and provide the same properties
   regarding location transparency and availability as system nodes.
   The difference is that secondary nodes don't need full physical
   connectivity to all other nodes in the cluster, -one link to one
   system node is sufficient, although there may be more for redundancy
   reasons.

   There may be up to 2047 secondary nodes in a cluster, the node part
   of their identities being within the range 2048-4095.  In fact, from
   a TIPC viewpoint this special address is the only thing
   distinguishing a secondary node from a system node.

   TIPC does not allow secondary nodes to establish links directly to
   each other, since they are supposed to play a subordinate role in the
   system.






Maloy, et al.            Expires April 27, 2005                [Page 23]


Internet-Draft                    TIPC                      October 2004


3.2  Addressing

3.2.1  Location Transparency

   TIPC provides two functional address types, Port Name and Port Name
   Sequence, to support location transparency, and two physical address
   types, Network Address and Port Identity, to be used when physical
   location knowledge is necessary for the user.

3.2.2  Network Address

   A physical entity within a network is identified internally by a TIPC
   Network Address.  This address is a 32-bit integer, structured into
   three fields, zone (8 MSB), cluster, (12 bits), and node (12 LSB).
   The address is only filled in with as much information as is relevant
   for the entity concerned, e.g.  a zone may be identified as
   0x03000000 (<3.0.0>), a cluster as 0x03001000 (<3.1.0>), and a node
   as 0x03001005 (<3.1.5>).  Any of these formats is sufficient for the
   TIPC routing function to find a valid destination for a message.

3.2.3  Port Identity

   This address is produced internally by TIPC when a port is created,
   and is only valid as long as that physical instance of the port
   exists.  It consists of two 32-bit integers.  The first one is a
   random number with a period of 2^31-1, the second one is a fully
   qualified network address with the internal format as described
   earlier.  A port identity may be used the same way as a port name,
   for connectionless communication or connection setup, as long as the
   user is aware of its limitations.  The main advantage with using this
   address type over a port name is that it avoids the potentially
   expensive binding operation in the destination port, something which
   may be desirable for performance reasons.

3.2.4  Port Name

   A port name is a persistent address typically used for connectionless
   communication and for setting up connections.  Binding a port name to
   a port roughly corresponds to binding a socket to a port number in
   TCP, except that the port name is unique and has validity for the
   whole publishing scope indicated in the bind operation, not only for
   a specific node.  This means that no network address has to be given
   by the caller when setting up a connection, unless he explicitly
   wants to reach a certain node, cluster or zone.

   A port name consists of two 32-bits integers.  The first integer is
   called the Name Type, and typically identifies a certain service type
   or functionality.  The second integer is called the Name Instance,



Maloy, et al.            Expires April 27, 2005                [Page 24]


Internet-Draft                    TIPC                      October 2004


   and is used as a key for accessing a certain instance of the
   requested service.

   The type/instance structure of a port name helps giving support for
   both service partitioning and service load sharing.

   When a port name is used as destination address for a message, it
   must be translated by TIPC to a port identity before it can reach it
   destination.  This translation is performed on a node within the
   lookup scope indicated along with the port name.

3.2.5  Port Name Sequence

   To give further support for service partitioning TIPC even provides
   an address type called  Port Name Sequence, or just Name Sequence.
   This is a three-integer structure defining a range of port names,
   i.e.  a name type plus the lower limit of and the upper boundary of
   the range.  By allowing a port to bind to a sequence, instead of just
   an individual port name, it is possible to partition the service's
   range of responsibility into sub-ranges, without having to create a
   vast number of ports to do so.

   There are very few limitations on how name sequences may be bound to
   ports.  One may bind many different sequences, or many instances of
   the same sequence, to the same port, to different ports on the same
   node, or to different ports anywhere in the cluster or zone.  The
   only restriction, in reality imposed by the implementation complexity
   it would involve, is that no partially overlapping sequences of the
   same name type may exist within the same publishing scope.






















Maloy, et al.            Expires April 27, 2005                [Page 25]


Internet-Draft                    TIPC                      October 2004


                                                ---------------
                                               | Partition B   |
                                               |               |
                                               O bind(type: 17 |
             -----------------                 |      lower:10 |
            |                 |                |      upper:19)|
            |send(type:    17 |                 ---------------
            |     instance:7) O------+
            |                 |      |          ---------------
            |                 |      |         | Partition A   |
             -----------------       |         |               |
                                     +-------->O bind(type: 17 |
                                               |      lower:0  |
                                               |      upper:9  |
                                                ---------------


     Figure 8: Functional addressing, using port name and port name
                                sequence

   When a port name is used as a destination address it is never used
   alone, contrary to what is indicated in Figure 8.  It has to be
   accompanied by a network address stating the scope and policy for the
   lookup of the port name.  This will be described later.

3.2.6  Multicast Addressing

   The concept of functional addressing is also used to provide
   multicast functionality.  If the sender of a message indicates a port
   name sequence instead of a port name, a replica of the message will
   be sent to all ports bound to a name sequence fully or partially
   overlapping with the sequence indicated.



















Maloy, et al.            Expires April 27, 2005                [Page 26]


Internet-Draft                    TIPC                      October 2004


                                                ---------------
                                               | Partition B   |
                                               |               |
                                     +-------->O bind(type: 17 |
             -----------------       |         |      lower:10 |
            |                 |      |         |      upper:19)|
            |send(type: 17    |      |          ---------------
            |     lower:7     O------+
            |     upper 13)   |      |          ---------------
            |                 |      |         | Partition A   |
             -----------------       |         |               |
                                     +-------->O bind(type: 17 |
                                               |      lower:0  |
                                               |      upper:9  |
                                                ---------------


        Figure 9: Functional multicast, using port name sequence

   Only one replica of the message will be sent to each identified
   target port, even if it is bound to more than one overlapping name
   sequence.

   This function will whenever possible and considered advantageous make
   use of the reliable cluster broadcast service also supported by TIPC.

3.2.7  Publishing Scope

   The default visibility scope of a published (bound) port name is the
   local cluster.  If the publication issuer wants to give it some other
   visibility he must indicate this explicitly when binding the port.
   The scopes available are:

      Value     Meaning
      -----     -------
      1         Visibility within whole own zone
      2         Visibility within whole own cluster
      3         Visibility limited to own node



3.2.8  Lookup Policies

   When a port name is looked up in the TIPC internal naming table for
   translation to a port identity the following rules apply:

   If indicated lookup domain is <Z.C.N>, the lookup algorithm must
   choose a matching publication from that particular node.  If nothing



Maloy, et al.            Expires April 27, 2005                [Page 27]


Internet-Draft                    TIPC                      October 2004


   is found on the given node, it must give up and reject the request,
   even if other matching publications exist within the zone.

   If the lookup domain is <Z.C.0>, the algorithm must select
   round-robin among all matching publications within that cluster,
   treating node local publications no different than the others.  If
   nothing is found within the given cluster, it must give up and reject
   the request, even if other matching publications exist within the
   zone.

   If the lookup domain is <Z.0.0>, the algorithm must select
   round-robin among all concerned publications within that zone,
   treating node or cluster local publications no different than the
   others.  If nothing is found, it must give up and reject the request.

   A lookup domain of <0.0.0> means that the nearest found publication
   must be selected.  First a lookup with scope <own zone.own
   cluster.own node> is attempted.  If that fails, a lookup with the
   scope <own zone.own cluster.0> is tried, and finally, if that fails,
   a lookup with the scope <own zone.0.0>.  If that fails the request
   must be rejected.

   Round-robin based lookup means that the algorithm must select equally
   among all the matching publications within the given scope.  In
   practice this means stepping the root pointer to a circular list
   referring to those publications between each lookup.

3.2.9  Name Translation

   Recommended Algorithm.

3.2.10  Distributed Naming Table

   The TIPC internal naming table is used for translation from a port
   name to a corresponding port identity, or from a port name sequence
   to a corresponding set of port identities.  In order to achieve
   acceptable translation times and fault tolerance, an instance of this
   table must exist on each node.  Each instance of the table must be
   kept consistent with all other instances within the same zone, and
   there must be no unnecessary delays in the update the neighbouring
   table instances when a port name sequence is published or withdrawn.
   Inconsistencies are only tolerated for the short timespan it takes
   for update messages to reach the neigbouring nodes, or for the time
   it takes for a node to detect that a neighbouring node has
   disappeared.

   When a node establishes contact with a new node in the cluster or the
   zone, it must immediately send out the necessary number of



Maloy, et al.            Expires April 27, 2005                [Page 28]


Internet-Draft                    TIPC                      October 2004


   NAME_DISTRIBUTOR/ PUBLICATION messages to that node, in order to let
   it update its local naming table instance.

   When a node looses contact with another node, it must immediately
   clean its naming table from all entries pertaining to that node.

   When a port name sequence is published on a node, TIPC must
   immediately send out a NAME_DISTRIBUTOR/PUBLICATION message to all
   nodes within the publishing scope, in order to have them update their
   tables.

   When a port name sequence is withdrawn on a node, TIPC must
   immediately send out a NAME_DISTRIBUTOR/WITHDRAWAL message to all
   nodes within the publishing scope, in order to have them remove the
   corresponding entry from their tables.

   Temporary table inconsistencies may occur, despite the above, and are
   handled as follows: If a successful lookup on one node leads to a
   non-existing port on another node, the lookup is repeated on that
   node.  If this lookup succeeds, but again leads to a non-existing
   port, another lookup is done.  This procedure can be repeated up to
   six times before giving up and rejecting the message.

3.3  Topology Services

   TIPC provides a mechanism for inquiring about or subscribing for the
   availability of  port names or ranges of port names.  The service is
   built on and uses the contents of the node local instance of the
   naming table.

3.3.1  Inquiry




















Maloy, et al.            Expires April 27, 2005                [Page 29]


Internet-Draft                    TIPC                      October 2004


                                                ---------------
                                               | Partition B   |
                                               |               |
                                               O bind(type: 17 |
    --------------------------                 |      lower:10 |
   |                          |                |      upper:19)|
   |is_published(type:    17  |                 ---------------
   |             instance: 7, O<-----+
   |             timeout:  0) |      |          ---------------
   |                          |      |         | Partition A   |
    --------------------------       |         |               |
                                     +---------O bind(type: 17 |
                                               |      lower:0  |
                                               |      upper:9  |
                                                ---------------


           Figure 11: Inquiry about existence of a port name

   Inquiries are synchronous requests to TIPC about a port name.  A
   timer value in msecs may be given along with the request, indicating
   that the call should not return until the port name has been
   published, or until the timer expires, whichever comes first,
   indicated in the return value of the call.  A timeout of zero
   instructs the call to return immediately, a timeout of 0xffffffff
   indicates that the call should not return until the port name
   requested has been published.

3.3.2  Subscriptions






















Maloy, et al.            Expires April 27, 2005                [Page 30]


Internet-Draft                    TIPC                      October 2004


                                                      ---------------
                                                     | Partition B   |
                                           <-17,10,13|               |
                                           +---------O bind(type: 17 |
             -----------------------       |         |      lower:10 |
            |                       |      |         |      upper:19)|
            |subscribe(type:    17  |      |          ---------------
            |          lower:    7, O<-----+
            |          upper:   13, |      |          ---------------
            |          timeout:100) |      |         | Partition A   |
             -----------------------       |         |               |
                                           +---------O bind(type: 17 |
                                           <-17,7,9  |      lower:0  |
                                                     |      upper:9  |
                                                      ---------------


  Figure 12: Subscription about existence of  sequences within a range

   A subscription is a non-blocking request to TIPC, telling it to
   indicate when a name sequence within the requested range is published
   or withdrawn.  Such events will be issued repeatedly for any changes
   pertaining to the range until the given timer expires.  The timer
   values are interpreted the same way as for inquiries.  Subscription
   for a particular port name is equivalent to indicating the same value
   in "lower" and "upper".

   Each event will indicate the overlapping part between the requested
   range and the actual published range, as it is also shown in the
   figure above.

3.3.3  Functional Topology

   The functional topology of the cluster can be continuously kept track
   of by subscribing for the relevant port names or sequences.

3.3.4  Physical Topology














Maloy, et al.            Expires April 27, 2005                [Page 31]


Internet-Draft                    TIPC                      October 2004


                                              -----------------------
                                             | Node <1.1.3>          |
                                             |                       |
                                       +-----O bind(type: 0,         |
    ------------------------------     |     |      lower:0x01001003,|
   |                              |    |     |      upper:0x01001003)|
   |subscribe(type:   0,          |    |      -----------------------
   |          lower:  0,          O<---+
   |          upper:  0x01001000, |    |      -----------------------
   |          timeout:0xffffffff) |    |     | Node <1.1.7>          |
    ------------------------------     |     |                       |
                                       +-----O bind(type: 0,         |
                                             |      lower:0x01001007,|
                                             |      upper:0x01001007)|
                                              -----------------------


    Figure 13: Subscription for physical topology  of cluster <1.1>

   The physical cluster topology can be considered a special case of the
   functional topology, and can be kept track of in the same way.
   Hence, to subscribe for the availability/disappearance of a specific
   node, a group of nodes, or a remote cluster, the user specifies a
   dedicated port name sequence, representing this "function".  In this
   particular case, TIPC will itself publish the corresponding port name
   as soon as it discovers or looses contact with a node.  The special
   name type 0 (zero) is used for this purpose.

3.4  Ports

3.4.1  Port State Machine




















Maloy, et al.            Expires April 27, 2005                [Page 32]


Internet-Draft                    TIPC                      October 2004


                    ---------------             -----------------
             create|               | connect   |                 |
             ----->|               |---------->|    CONNECTED/   |
             delete|     READY     |<----------|    CONFIRMED    |
             <-----|               | disconnect|                 |
                   |               |<--+       |                 |
                    --A--------+---    |        --+----A------A--
                      |        |       |      time| pro| probe|
                 withdraw     publish  |      out | be | reply|
                      |        |   disconnect     |    |      |
                    --+--------V---    |        --V----+------+--
                   |               |   +-------|                 |
                   |               |           |    CONNECTED/   |
                   |     BOUND     |           |    PROBING      |
                   |               |           |                 |
                   |               |           |                 |
                    ---------------             -----------------


                Figure 14: Port FSM for non-error events

   The port state machine is relatively simple for normal, non-error
   events, as illustrated in Figure 14.

   A port has three main STATES, as described below:

   READY: The port is in its basic state, and is ready to receive any
      normal state event.
   BOUND: The port has been bound to (published with) one or more port
      name sequences.
   CONNECTED: The port has been connected to some other port in the
      network, i.e.  it has stored the identity of that port, and a flag
      "connected" is set in the port.

   The CONNECTED state has two sub-states, reflecting its supervision of
   the connected peer:

   CONNECTED/CONFIRMED: The port has had confirmed that the other port
      exists, through reception a payload message or CONN_MANAGER
      message from the peer within the last timer interval.
   CONNECTED/PROBING: During the last timer expiration, it sent out a
      CONN_PROBE message to the peer, and now awaits the unconditional
      CONN_PROBE_REPLY message from the other end, or any data or
      CONN_PROBE message from the peer that can confirm the correct
      state of that port.  See the detailed description of how this is
      handled later in this section.

   The following EVENTS may occur to a port:



Maloy, et al.            Expires April 27, 2005                [Page 33]


Internet-Draft                    TIPC                      October 2004


      CREATE:        Trivial
      PUBLISH:       Bind a port name sequence to a port.
      WITHDRAW:      Unbind the relation between a port name sequence
                     and a port.
      CONNECT:       Connect the port to another port.
      DISCONNECT:    Disconnect the port from the port it is connected
                     to.
      TIMEOUT:       Check if a sent CONN_PROBE was reponded to. Order
                     new timer.
      PROBE:         Receive a CONN_PROBE from peer.
      PROBE_REPLY:   Receive a CONN_PROBE_REPLY from peer.
      SEND_CONN:     Send a data message of type CONN_MSG.
      SEND_CONNLESS: Send a data message of type NAMED_MSG or
                     DIRECT_MSG.
      REC_CONN:      Receive data message of type CONN_MSG.
      REC_DIRECT:    Receive a DIRECT_MSG data message.
      REC_NAMED:     Receive a NAMED_MSG data message.
      REC_CONN_ERR:  Receive CONN_MSG data message with error code.
      REC_CLESS_ERR: Receive DIRECT_MSG or NAMED_MSG with error code.
      LOST_NODE:     Receive indication that contact with peer node
                     lost.
      DELETE:        Not so trivial.

   A port may also take the following ACTIONS, depending on event:


      SEND_PRB:      Send a CONN_PROBE to peer.
      SEND_REPLY:    Send a CONN_PROBE_REPLY to peer.
      ABORT_REM:     Send one DATA_NON_REJECTABLE/CONN_MSG/
                     NO_REMOTE_PORT to peer.
      ABORT_SELF:    Send one DATA_NON_REJECTABLE/CONN_MSG to self,
                     with the appropriate error code, NO_REMOTE_NODE
                  or NO_REMOTE_PORT.
      DISCONNECT:    Disconnect.
      WITHDRAW:      Withdraw all publications pertaining to this port.
      REJ_CALL:      Reject user call with interface specific error
                     code.
      REJ_MSG:       Reject message with error code NO_REMOTE_PORT.
      DROP:          Drop message silently.

   The state machine in Figure 14 only covers the normal events and
   state transitions in a port.  The following table gives a more
   comprehensive picture.  If there is no arrow "->" in a field it means
   that the port remains it its current state.



   ---------------------------------------------------------------------



Maloy, et al.            Expires April 27, 2005                [Page 34]


Internet-Draft                    TIPC                      October 2004


   Event:         |   READY   |   BOUND  |          CONNECTED
                  |           |          | CONFIRMED    |     PROBING
   ---------------|-----------+----------+--------------+---------------
   CREATE:        |   ->!     |    -     |      -       |     -
   ---------------|-----------+----------+--------------+---------------
   PUBLISH:       |  ->BOUND  |          |           REJ_CALL
   ---------------|-----------+----------+--------------+---------------
   WITHDRAW:      | REJ_CALL  | ->READY  |           REJ_CALL
   ---------------|-----------+----------+--------------+---------------
   CONNECT:       |->CONN/CONF| REJ_CALL |           REJ_CALL
   ---------------|-----------+----------+--------------+---------------
   DISCONNECT:    | REJ_CALL  | REJ_CALL |          -> READY
   ---------------|-----------+----------+--------------+---------------
   TIMEOUT:       |    -      |   -      | SEND_PRB ->  | ABORT_SELF ->
                  |           |          |      PROBING |       READY
   ---------------|-----------+----------+--------------+---------------
   PROBE:         |SEND_REPLY |  ABORT   |   SEND_REPLY -> CONFIRMED
   ---------------|-----------+----------+--------------+---------------
   PROBE_REPLY:   | ABORT_REM |  ABORT   |              | ->CONFIRMED
   ---------------|-----------+----------+--------------+---------------
   SEND_CONN:     | REJ_CALL  | REJ_CALL |              |
   ---------------|-----------+----------+--------------+---------------
   SEND_CONNLESS: |           | ->READY  |           REJ_CALL
   ---------------|-----------+----------+--------------+---------------
   REC_CONN:      |->CONN/CONF|ABORT_REM |              | ->CONFIRMED
   ---------------|-----------+----------+--------------+---------------
   REC_DIRECT:    |           |          |           REJ_MSG
   ---------------|-----------+----------+--------------+---------------
   REC_NAMED:     |           |          |           REJ_MSG
   ---------------|-----------+----------+--------------+---------------
   REC_CONN_ERR:  |  DROP     |  DROP    |   DISCONNECT -> READY
   ---------------|-----------+----------+--------------+---------------
   LOST_NODE:     |    -      |    -     |   ABORT_SELF -> READY
   ---------------|-----------+----------+--------------+---------------
   REC_CLESS_ERR: |           |          |             DROP
   ---------------|-----------+----------+--------------+---------------
   DELETE:        |   ->0     | WITHDRAW |         ABORT_REM
   ---------------|-----------+----------+--------------+---------------


                      Figure 17: Complete port FSM

   The reason for having a background probing of connections is
   explained in Section 3.5.  The recommended timer interval for this
   probing is 3600 s, making it probable that the timer will never have
   to expire.





Maloy, et al.            Expires April 27, 2005                [Page 35]


Internet-Draft                    TIPC                      October 2004


3.5  Connections

   User Connections must be kept as lightweight as possible because of
   their potential huge number, and because it must be possible to
   establish and shut down thousands of connections per second on a
   node.

3.5.1  Connection Setup

   How a connection is established and terminated is not defined by the
   protocol, only how they are supervised, and if necessary, aborted.
   Instead, this is left to the end user to define, or to the actual
   implementation of the user API-adapter.  The following figures show
   two examples of this.



             -------------------                -------------------
            | Client            |              | Server            |
            |                   |              |                   |
            | (3)create(cport)  |              | (1)create(suport) |
            | (4)send(type:17,  |------------->0 (2)bind(type: 17, |
            |         inst: 7)  0<------+      |\        lower:0   |
            | (8)lconnect(sport)|       |      | \       upper:9)  |
            |                   |       |      | /                 |
            |                   |       |      |/(5)create(sport)  |
            |                   |       +------0 (6)lconnect(cport)|
            |                   |              | (7)send()         |
             -------------------                -------------------


   Figure 18: Example of user defined  establishment of a connection

   Figure 18 shows an example where the user himself defines how to set
   up the connection.  In this case, the client starts with sending one
   payload- carrying NAMED_MSG message to the setup port (suport)(4).
   The setup server receives the message, and reads its contents and the
   client port (cport) identity.  He then creates a new port (sport)(5),
   and connects it to the client port's identity(6).  The lconnect()
   call is a purely node local operation in this case, and the
   connection is not fully established until the server has fulfilled
   the request and sent a response payload-carrying CONN_MSG message
   back to the client port(7).  Upon reception of the response message
   the client reads the server port's identity and performs an
   lconnect() on it(8).  This way, a connection has been established
   without sending a single protocol message.





Maloy, et al.            Expires April 27, 2005                [Page 36]


Internet-Draft                    TIPC                      October 2004


             --------------------                -------------------
            | Client             |              | Server            |
            |                    |              | (1)create(suport) |
            | (4)create(cport)   |   "SYN"      | (2)bind(type: 17, |
            | (5)connect(type:17,|------------->0         lower:0   |
            | (9)        inst: 7)0<------+     /|         upper:9)  |
            |                    |       |    / | (3)accept()       |
            |                    |    (7)|    \ | (8)               |
            |                    |       |  (6)\|                   |
            |                    |       +------0 (9)recv()         |
            |                    |      "SYN"   |                   |
             --------------------                -------------------


                 Figure 19: TCP-style connection setup

   Figure 19 shows an example where the user API-adapter supports a
   TCP-style connection setup, using hidden protocol messages to fulfil
   the connection.  The client  starts with calling connect()(5),
   causing the API to send an empty NAMED_MSG message ("SYN" in TCP
   terminology) to the setup port.  Upon reception, the API-adapter at
   the server side creates the server port, peforms a local
   lconnect()(6) on it towards the client port, and sends an empty
   CONN_MSG ("SYN") back to the  client port (7).  The accept() call in
   the server then returns, and the server can start waiting for
   messages (8).  When the second SYN  message arrives in the client,
   the API-adapter performs a node local lconnect() and lets the
   original connect() call return (9).

   Note the difference between this protocol and the real TCP connection
   setup protocol.  In our case there is no need for SYN_ACK messages,
   because the transport media between the client and the server (the
   node-to-node link) is reliable.

   Also note from these examples that it is possible to retain full
   compatibility between these two very different ways of establishing a
   connection.  Before the connection is established, a TCP-style client
   or server should interpret a payload message from a user-controlled
   counterpart as an implicit SYN, and perform an lconnect() before
   queueing the message for reading by the user.  The other way around,
   a user-controlled client or server must perform an lconnect() when
   receiving the empty message from its TCP-style counterpart.

3.5.2  Connection Shutdown







Maloy, et al.            Expires April 27, 2005                [Page 37]


Internet-Draft                    TIPC                      October 2004


             -------------------                -------------------
            | Client            |              | Server            |
            |                   |              |                   |
            |                   |              |                   |
            |          lclose() 0              0 lclose()          |
            |                   |              |                   |
            |                   |              |                   |
            |                   |              |                   |
             -------------------                -------------------


      Figure 20: Example of user defined shutdown of  a connection

   Figure 20 shows the simplest possible user defined connection
   shutdown scheme.  If it inherent in the user protocol when the
   connection should be closed, both parties will know the right moment
   to perform a "node local close" (lclose()) and no protocol messages
   need to be involved.



             --------------------                -------------------
            | Client             |              | Server            |
            |                    |    "FIN"     |                   |
            |          (1)close()0------------->0(2)close()         |
            |                    |              |                   |
            |                    |              |                   |
            |                    |              |                   |
             --------------------                -------------------


                Figure 21: TCP-style connection shutdown

   In Figure 21 a TCP-style connection close() is illustrated.  This is
   simpler than the connection setup case, because the built-in
   connection abortion mechanism of TIPC can be used.  When the client
   calls close() (1) TIPC must delete the client port.  As will be
   described later, deleting a connected port has the effect that a
   DATA_NON_REJECTABLE/CONN_MSG ("FIN" in TCP terminology) with error
   code NO_REMOTE_PORT is sent to the other end.  Reception of such a
   message means that TIPC at the receiving side must shut down the
   connection, and this must be done already before the server is
   notified.  The server must then call close() (2), not to close the
   connection, but to delete the port.  TIPC does not send any "FIN"
   this time, the server port is already disconnected, and the client
   port is anyway gone.  If both endpoints call close() simultaneously,
   two "FIN" messages will cross each other, but at the reception they
   will have no effect, since there is no destination port, and they



Maloy, et al.            Expires April 27, 2005                [Page 38]


Internet-Draft                    TIPC                      October 2004


   must be discarded by TIPC.

   Note even here the automatic compatibility with a user-defined peer
   and a TCP-style ditto: Any user, no matter the user API, must at any
   moment be ready to receive a "connection aborted" indication, and
   this is what in reality happens here.

3.5.3  Connection Abortion

   When a connected port receives an indication from the TIPC link layer
   that it has lost contact with its peer node, it must immediately
   disconnect itself and send an empty CONN_MSG/NO_REMOTE_NODE to its
   owner process.

   When a connected port is deleted without a preceding disconnect()
   call from the user it must immediately disconnect itself and send an
   empty CONN_MSG/NO_REMOTE_PORT to its peer port.  This may happen when
   the owner process crashes, and the OS is reclaiming its resources.

   When a connected port receives a timeout call, and is still in
   CONNECTED/PROBING state since the previous timer expiration,it must
   immediately disconnect itself and send an empty
   CONN_MSG/NO_REMOTE_PORT to its owner process.

   When a connected port receives a rejected previously sent message, (a
   CONN_MSG with error code), it must immediately disconnect itself and
   deliver the message, with data contents, to its owner process.

   When a port participating in a multi-hop connection receives a
   CONN_MSG where the connection level sequence number does not fit, it
   must immediately disconnect itself, send an empty CONN_MSG/COMM_ERROR
   to its owner process, and another empty CONN_MSG/COMM_ERROR to its
   peer port.

   When a connected port receives a CONN_MSG from somebody else than its
   peer port, it must immediately send an empty CONN_MSG/NO_CONNECTION
   to the originating port.

   When TIPC in a node receives a CONN_MSG/TIPC_OK for which it finds no
   destination port,  it must immediately send an empty
   CONN_MSG/NO_REMOTE_PORT back to the originating port.

   When a bound port receives a CONN_MSG from anybody,it must
   immediately send an empty CONN_MSG/NO_CONNECTION to the originating
   port.






Maloy, et al.            Expires April 27, 2005                [Page 39]


Internet-Draft                    TIPC                      October 2004


3.5.4  Connection Supervision

   In almost all practical cases the mechanisms for resource cleanup
   after process failure, rejection of messages when no destination port
   is found, and supervision of peer nodes at the link level, is
   sufficient for immediate failure detection and abortion of
   connections.

   However, because of the non-specified connection setup procedure of
   TIPC, there exists cases when a connection may remain incomplete.
   This may happen if the client in a user-defined setup/shutdown scheme
   forgets to call lconnect() (see Figure 20), and then deletes itself,
   or if one of the parties fails to call lclose() (see Figure 21).
   These cases are considered very rare, and should normally have no
   serious consequenses for the availability of the system, so a slow
   background timer is judged sufficient to discover such situations.

   When a connection is established each port starts a timer, whose
   purpose is to check the status of the connection.  It does this by
   regularly (typical configured interval is once an hour) sending a
   CONN_PROBE message to the peer port of the connection.  The probe has
   two tasks; first, to inform that the sender is still alive and
   connected; second, to inquire about the state of the recipient.

   A CONN_PROBE or a CONN_PROBE_REPLY reply MUST be immediately
   responded to according to the following scheme:

























Maloy, et al.            Expires April 27, 2005                [Page 40]


Internet-Draft                    TIPC                      October 2004


   ---------------------------------------------------------------------
   |                              |        Received Message Type       |
   |                              |-----------------+------------------|
   |                              |   CONN_PROBE    | CONN_PROBE_REPLY |
   |                              |                 |                  |
   |==============================|====================================|
   |     |             Multi-hop  |        DATA_NON_REJECTABLE+        |
   |     |             seqno wrong|        TIPC_COMM_ERROR             |
   |     |            ------------|-----------------+------------------|
   |     | Connected   Multi-hop  |                 |                  |
   |     | to sender   seqno ok   |                 |                  |
   |     | port       ------------|                 |                  |
   |     |             Single hop | CONN_PROBE_REPLY|  No Response     |
   |     |------------------------|                 |                  |
   |     | Not connected,         |                 |                  |
   |Rece-| not bound,             |                 |                  |
   |ving |------------------------|-----------------+------------------|
   |Port | Connected to           |                                    |
   |State| other port             |        DATA_NON_REJECTABLE+        |
   |     |------------------------|        TIPC_NOT_CONNECTED          |
   |     | Bound to               |                                    |
   |     | port name sequence     |                                    |
   |     |------------------------|------------------------------------|
   |     | Recv. node available,  |        DATA_NON_REJECTABLE+        |
   |     | recv. port non-existent|        TIPC_NO_REMOTE_PORT         |
   |     |------------------------|------------------------------------|
   |     | Receiving node         |        DATA_NON_REJECTABLE+        |
   |     | unavailable            |        TIPC_NO_REMOTE_NODE         |
   ---------------------------------------------------------------------


       Figure 22: Response to probe/probe replies vs port state.

   If everything is well then the receiving port will answer with a
   probe reply message, and the probing port will go to rest for another
   interval.  It is inherent in the protocol that one of the ports - the
   one connected last - normally will remain passive in this
   relationship.  Each time its timer expires it will find that it has
   just received and replied to a probe, so it will never have any
   reason to explicitly send a probe itself.

   When an error is encountered, one or two empty CONN_MSG data are
   generated, one to indicate a connection abortion in the receiving
   port, if it exists, and one to do the same thing in the sending port.

   The state machine for a port during this message exchange is
   described in Section 3.5.




Maloy, et al.            Expires April 27, 2005                [Page 41]


Internet-Draft                    TIPC                      October 2004


3.5.5  Flow Control

   The mechanism for end-to-end flow control at the connection level has
   as its primary purpose to stop a sending process from overrunning a
   slower receiving process.  Other tasks, such as bearer, link,
   network, and node congestion control, are handled by other mechanisms
   in TIPC.  Because of this, the algorithm can be kept very simple.  It
   works as follows:
   1.  The message sender (the API-adapter) keeps one counter, SENT_CNT,
       to count messsages he has sent, but which has not yet been
       acnkowledged.  The counter is incremented for each sent message.
   2.  The receiver counts the number of messages he delivers to the
       user, ignoring any messages pending in the process in-queue.  For
       each N message, he sends back a CONN_MANAGER/ACK_MSG containing
       this number in its data part.
   3.  When the sender receives the acknowledge message, he subtracts N
       from SENT_CNT, and stores the new value.
   4.  When the sender wants to send a new message he must first check
       the value of SENT_CNT, and if this exceeds a certain limit, he
       must abstain from sending the message.  A typical measure to take
       when this happens is to block the sending process until SENT_CNT
       is under the limit again, but this will be API-dependent.

   The recommended value for the send window N is at least 200 messages,
   and the limit for SENT should be at least 2*N.

3.5.6  Sequentiality Check

   Inter-cluster connection-based messages, and intra-cluster messages
   between cluster nodes and secondary nodes, may need to be routed via
   intermediate nodes if there is no direct link between the two.  This
   implies a small, but not negligeable risk that messages may be lost
   or re-ordered.  E.g.  an intermediate node may crash, or it may have
   changed its routing table in the interval between the messages.  A
   connection level sequence number is used to detect such problems, and
   this must be checked for each message received on the connection.  If
   the sequence number does not fit in sequence, no attempts of
   re-sequencing should be done.  The port discovering the sequence
   error must immediately abort the connection by sending one empty
   CONN_MSG/COMM_ERROR message to itself, and one to the peer port.

   The sequence number must not be checked on single-hop connections,
   where the link protocol guarantees that no such errors can occur.

   The first message sent on a connection has the sequence number 42.






Maloy, et al.            Expires April 27, 2005                [Page 42]


Internet-Draft                    TIPC                      October 2004


3.6  Neighbour Detection

3.6.1  Link Requests

   At startup, or when otherwise told to, TIPC will send out Link
   Request messages to its neighbouring nodes, informing about its
   existence, and requesting to have a set of links set up from the
   destination domain towards itself.  The structure of Link
   Configuration messages is described in Section 4.3.9.

   A node receiving a Link Request, first checks whether it belongs to
   the destination  domain stated in the message, and if the Network
   Identity of the message is equal to its own.  If that is not the
   case, it ignores the message.

   Otherwise, depending on the destination domain and the sender node
   address, message, the node goes through one of the following steps:
   o  If the destination domain is exactly the own node, and if a link
      does not already exist, it creates a link to the sender node.  A
      response is sent back to the sender node if the Response Expected
      field is set.
   o  Otherwise, if the sender node belongs to the own cluster, and if a
      link does not already exist, it creates a link to the sender node.
      A response is sent back to the sender node if the Response
      Expected field is set.
   o  Otherwise, if the destination domain comprises, but is larger than
      the own node, and the sender node belongs to a remote cluster, the
      node initiates the Inter Cluster Link Setup algorithm.

3.6.2  Inter-Cluster Link Setup

3.6.2.1  Link Entropy

   The inter-cluster link setup algorithm has the goal of setting up a
   specified number of links from each node in one cluster, to a
   correponding number of different nodes in another cluster.  Dual
   links between a node pair are not permitted.  The algorithm takes all
   measures necessary to ensure that exactly the requested number of
   links are created and maintained at any time; nothing less, nothing
   more.  We call this the Link Entropy Check Algorithm.

   The algorithm also ensures that all created links are distributed
   smoothly over the two clusters.  The following applies:
   o  If two clusters are of equal size, each of the nodes in the two
      clusters will have exactly the number of links specified.
   o  If two clusters are of different size, all nodes in the bigger
      cluster will have exactly the specified number of links.  Some
      nodes in the smaller cluster will have more links than specified,



Maloy, et al.            Expires April 27, 2005                [Page 43]


Internet-Draft                    TIPC                      October 2004


      to fulfil the requirement for the bigger one.  These extra links
      will be smoothly distributed, so that no node in the smaller
      cluster will have more than one link more than any of the others.
   o  If a node is added to the smaller cluster, some of the existing
      extra links will be moved to the new node, to keep the optimal
      distribution.
   o  If a node is added to a bigger or equal-sized cluster, its new
      links will be established to nodes in the other clusters having
      the fewer links.
   o  If a node disappears or is removed from a cluster, the connected
      nodes in the other re-establish links to some of the remaining
      nodes, in such a way that smooth distribution is maintained, and
      the nodes regains the specified number of links.
   o  Whenever a new link is established beyond the specified link
      number for a node, it checks link entropy by sending a
      CHECK_LINK_COUNT message to all its peer nodes.  If any of the
      receiving nodes finds it has more inter-cluster links than its
      specified number, it knows that the link to the message sender is
      redundant, and terminates it.

   The recommended specified number of inter-cluster links per-node is
   two.

3.6.2.2  Initiation of Setup

   The first inter cluster contact may be established in two ways:
   o  A node is ordered through the management interface to send a Link
      Request to a specific node in the other cluster, hence creating a
      Pilot Link.  This link request must have the Response Expected
      field set to non-zero.
   o  Both clusters are within a bearer multicast/broadcast domain, e.g.
      the same LAN, and the neigbour detection broadcasts are configured
      to be accepted by foreign clusters, i.e.  destination domain is
      <Z.0.0> or <0.0.0>.  These link requests must have the Response
      Expected field set to non-zero.  Possible redundant links created
      this way will be removed later through the link entropy check.

   Thereafter the following sequence of events follows:
   o  When a first inter-cluster link comes up, all the other nodes in
      both clusters will immediately become aware of it.  This is
      because of the distribution of ROUTE_ADDITION (see Section 3.8)
      messages from the establishing nodes.
   o  Any node in a cluster becoming aware of the existence of a new
      cluster, immediately sends a GET_NODE_INFO message to the router
      node, in order to obtain the bearer level address (IP- or
      Ethernet) needed to reach the remote node.
   o  When the bearer address is obtained, the nodes send a Link Request
      message to the identified remote node.  This Link Request must be



Maloy, et al.            Expires April 27, 2005                [Page 44]


Internet-Draft                    TIPC                      October 2004


      "open", i.e.  both the node part of the destination domain field
      and the contents of the Response Expected field must be zero.
      When the romote node receive the request, it does not immediately
      establish a link, but follows a procedure described in the
      following section.
   o  Until the first own inter-cluster link is established, each node
      repeatedly, but using the same exponential backoff algorithm as
      for broadcasted Link Requests (see description later), send out
      new Link Requests to the other cluster.  Duplicates of such
      requests are detected and dropped, as will be described later.

   As can be seen from this description, and from the following
   sections, the scenario for setting up inter-cluster links is
   extremely chaotic in the beginning, with all nodes in both clusters
   simultaneously trying to set up links to the opposite cluster.  The
   way Link Requests and link establishments are handled, still ensures
   that this phase will eventually settle down to a link pattern with
   the optimal number and distribution of links, and remain that way as
   long as the two clusters are in contact.

3.6.2.3  Handling of Inter Cluster Link Requests

   When a node receives a Link Request from a remote cluster, and the
   request contains a domain larger than the node itself, it initiates
   the following sequence of events.
   o  The node creates a Link Probe message, whose task it is to roam
      around in the cluster to find the most suitable node to establish
      a link back to the original remote node.  The structure of Link
      Probe Messages is described in Section 4.
   o  If there is no previous contact with the remote cluster, i.e.  the
      node's routing table shows that there are no other nodes having
      links to the sender's cluster, and the Response Expected field of
      the request is non-zero,the node creates a link, called the Pilot
      Link, and sends a response back to the originating node.  While
      the link activation is ongoing, i.e.  until the pilot link is up
      in WORKING_WORKING state, or is found to have failed, the link
      probe is parked at the receiving node, and all subsequent link
      requests from the originating cluster are ignored.
   o  If there is previous contact, or when the pilot link comes up, the
      link probe is sent on a tour in the cluster, using the Sequence
      Tag to determine each next hop.  At each node on the tour it
      counts the number of links back to the originating cluster, and
      stores that value if it is lower than the Lowest Link Count This
      Tour field in the probe.  If at any node it discovers a working
      link back to the requesting node, it decrements the Requested
      Links field with one and continues to the next node in the
      sequence.  If the Requested Links field reaches zero, or a parked
      probe from the same remote node is found, the link probe is



Maloy, et al.            Expires April 27, 2005                [Page 45]


Internet-Draft                    TIPC                      October 2004


      dropped as a duplicate, and no more action is taken on behalf of
      the original link request.
   o  After a tour, i.e.  when the probe is back at the node receiving
      the original link request, the value of Lowest Link Count This
      Tour field is stored in the Lowest Link Count field, and the probe
      is sent out on a new tour, comprising the first node.
   o  When the probe encounters a node having the same number of links
      to the remote cluster as Lowest Link Count, and where there is no
      previous link towards the requesting node, the probe is parked on
      that node, a link endpoint is created, and a "reverse" Link
      Request message is sent directly back to the requesting node.  In
      this request, the destination domain field must be fully
      specified, and the Response Expected field set to zero.  No link
      endpoint is created yet.
   o  When the requesting node in the other cluster receives the reverse
      link request, there are four possible responses:
      1.  It finds it already has enough (i.e.  the requested number of)
          links to the responding cluster.  It sends back a
          DROP_LINK_REQUEST message to the responding node, instructing
          it to delete its link endpoint and the parked link probe.
      2.  It may find it already has a link, working or in progress, to
          the responding  node.  If this race condition is true, it
          returns a LINK_REQUEST_REJECTED message to that node.  This
          forces the responding node to to pass the parked link probe to
          the next node in the cluster sequence.
      3.  It may find that itself has a parked link probe, trying to
          establish a link to the responder.  If this race condition is
          true, it must decide which of the mutual setup attempts should
          be aborted.  Hence, if the numerical value of the own node
          address is lower than that of the responding node, it returns
          a LINK_REQUEST_REJECTED message to that node.  This forces the
          reponding node to pass the parked link probe to the next node
          in the cluster sequence.
      4.  None of the previous conditions are true.  It returns a
          LINK_REQUEST_ACCEPTED message to the reponding node.  That
          node creates its link endpoint, decrements the Requested Links
          counter of the parked link probe, and if it is still non-zero,
          it passes it on to the next node in the cluster.

   Note that the three new message types introduced here are sent as
   ordinary TIPC messages, using an ordinary TIPC port.  This can be
   done because there is already at least a pilot link between the two
   clusters.  The address used is the port name <1,msg_type>, using the
   responder node as lookup domain.

   When the probe has established all the requested links, or after a
   maximum of 10 complete cluster tours, it is dropped.




Maloy, et al.            Expires April 27, 2005                [Page 46]


Internet-Draft                    TIPC                      October 2004


3.6.3  Multicast Link Setup

   When the bearer media makes it possible, TIPC uses a special
   auto-configuration protocol for neighbour detection and link
   creation.  This is the case e.g.  with Ethernet and DCCP, where
   multicast transmission of packets is possible.  The protocol for this
   is fairly simple: Immediately after start, or when it detects that a
   new interface has become active, a node starts to repeatedly
   broadcast Link Request messages to other presumed members of the
   network.

   The Destination Domain field in these messages should be configurable
   when TIPC is started, but is typically set to
   <own_zone.own_cluster.0>.

   The broadcast interval has a start value of 125 msec, and is
   multiplied by a factor 4 at each transmission, until it reaches a
   configurable upper limit, default set to 32000 msec, at which point
   it stops increasing.  The broadcasts continue at this rate as long as
   the node is up.  A node receiving a Link Request, checks whether it
   belongs to the destination domain stated in the message, and if the
   Network Identity of the message is equal to its own.  If that is the
   case, if a link does not already exist, and the Response Expected
   field is non-zero, it creates its end of the link.  Thereafter, it
   answers with a unicast Link Request back to the requesting node.
   This will in its turn create the other end of the link, if there is
   not one already, and the next phase, the link activation phase,
   begins.























Maloy, et al.            Expires April 27, 2005                [Page 47]


Internet-Draft                    TIPC                      October 2004


                                                      -------------
                                                     | <1.1.3>     |
                                                     |             |
                     ucast(dest:<1.1.1>,orig:<1.1.3> |             |
                    <------------------------------- |             |
                                                     |             |
                                                      -------------
    --------------
   | <1.1.1>      |
   |              | bcast(orig:<1.1.1>,dest:<1.1.0>)
   |              |-------------------------------->
   |              |
   |              |
    --------------
                                                      -------------
                     ucast(dest:<1.1.1>,orig:<1.1.2> | <1.1.2>     |
                    <------------------------------- |             |
                                                     |             |
                                                     |             |
                                                     |             |
                                                      -------------

                     Figure 23: Neighbour Detection

   There are two reasons for the continuous broadcasting decribed above.
   First, it should always be possible for two nodes to discover each
   other, even if the communication media between them is non-functional
   at the start moment.  E.g.  in a dual-switch system, one of the node
   cables may have been faulty or disconnected from the beginning, while
   the cluster is still fully connected and functional via the other
   switch.  In such cases one should not be forced to restart one of the
   nodes to set up the links, and any more manual intervention than
   plugging in a working cable should be unnecessary.  Second, it should
   be possible to replace (hot-swap) an interface card with one having a
   different MAC address, still without having to restart the node.
   When a node receives a Link Request its originating MAC address is
   always checked against the one previously stored for that
   destination, and if they differ the old one is replaced.  This way, a
   replaced interface board will be detected and taken into traffic
   within 32 seconds of the replacement.

   The structure and semantics of Link Request messages is described in
   Section 4.3.9.

3.7  Links






Maloy, et al.            Expires April 27, 2005                [Page 48]


Internet-Draft                    TIPC                      October 2004


3.7.1  Link Activation

   Link activation and supervision is completely handled by the generic
   part of the protocol, in contrast to the partially media-dependent
   neighbour detection protocol.

   The following FSM describes how a link is activated and supervised.



     ---------------                               ---------------
    |               |<--(CHECKPOINT == LAST_REC)--|               |
    |               |                             |               |
    |Working-Unknown|----TRAFFIC/ACTIVATE_MSG---->|Working-Working|
    |               |                             |               |
    |               |-------+      +-ACTIVATE_MSG>|               |
     ---------------         \    /                ------------A--
        |                     \  /                   |         |
        | NO TRAFFIC/          \/                 RESET_MSG  TRAFFIC/
        | NO PROBE             /\                    |      ACTIVATE_MSG
        | REPLY               /  \                   |         |
     ---V-----------         /    \                --V------------
    |               |-------+      +--RESET_MSG-->|               |
    |               |                             |               |
    | Reset-Unknown |                             |  Reset-Reset  |
    |               |----------RESET_MSG--------->|               |
    |               |                             |               |
     -------------A-                               ---------------
      |           |
      | BLOCK/    | UNBLOCK/
      | CHANGEOVER| CHANGEOVER END
      | ORIG_MSG  |
     -V-------------
    |               |
    |               |
    |    Blocked    |
    |               |
    |               |
     ---------------


                  Figure 24: Link finite state machine

   A link enpoint's state is defined by the own endpoint's state,
   combined with what is known about the other endpoint's state.  The
   following states exist:





Maloy, et al.            Expires April 27, 2005                [Page 49]


Internet-Draft                    TIPC                      October 2004


   Reset-Unknown

      Own link endpoint reset, i.e.  queues are emptied and sequence
      numbers are set back to their initial values.  The state of the
      peer endpoint is unknown.  LINK_PROTOCOL/RESET_MSG messages are
      sent periodically at CONTINUITY_INTERVAL to inform peer about the
      own endpoint's state, and to force it to reset its own enpoint,if
      this has not already been done.  If the peer endpoint is
      rebooting, or has reset for some other reason, it will sooner or
      later also reach the state Reset-Unknown, and start sending its
      own RESET_MSG messages periodically.  At least one of the
      endpoints, and often both, will eventually receive a RESET_MSG and
      transfer to state Reset-Reset.  If the peer is still active, i.e.
      in one of the states Working-Working or Working-Unknown, and has
      not yet detected the disturbance causing this endpoint to reset,
      it will sooner or later receive a RESET_MSG, and transfer directly
      to state Reset-Reset.  If a LINK_PROTOCOL/ ACTIVATE_MSG message is
      received in this state, the link endpoint knows that the peer is
      already in state Reset-Reset, and can itself move directly on to
      state Working-Working.  Any other messages are ignored in this
      state.  CONTINUITY_INTERVAL is calculated as the smallest value of
      LINK_TOLERANCE/4 and 0.5 sec.
   Reset-Reset

      Own link endpoint reset, peer endpoint known to be reset, since
      the only way to reach this state is through receiving a RESET_MSG
      from peer.  The link endpoint is periodically at
      CONTINUITY_INTERVAL sending ACTIVATE_MSG messages.  This will will
      eventually cause peer to transfer to state Working-Working.  The
      own endpoint will also transfer to state Working-Working as soon
      as any message which is not a RESET_MSG is received.
   Working-Working

      Own link endpoint working.  Peer link endpoint known to be
      working, i.e.  both can send and receive traffic messages.  A
      periodic timer with the interval CONTINUITY_INTERVAL checks if
      anything has been received from the peer during the last interval.
      If not,state transfers to state Working-Unknown.
   Working-Unknown

      Own link endpoint working.  Peer link endpoint in unknown state.
      LINK_PROTOCOL/STATE_MSG messages with the PROBE bit set are sent
      at an interval of CONTINUITY_INTERVAL/4 to force a response from
      peer.  If a calculated number of probes
      (LINK_TOLERANCE/(CONTINUITY_INTERVAL/4) remain unresponded, state
      transfers to Reset-Unknown.  Own link endpoint is reset, and the
      link is considered lost.  If, instead, any kind of message, except
      LINK_PROTOCOL/RESET_MSG and LINK_PROTOCOL/ACTIVATE_MSG is



Maloy, et al.            Expires April 27, 2005                [Page 50]


Internet-Draft                    TIPC                      October 2004


      received, state transfers back to Working-Working.  Reception of a
      RESET_MSG in this situation brings the link to state Reset-Reset.
      ACTIVATE_MSG will never received in this state.
   Blocked

      The link endpoint is blocked from accepting any packets in either
      direction, except incoming, tunneled CHANGEOVER_PROTOCOL/ORIG_MSG.
      This state is entered upon the arrival of the first such message,
      and left when the last has been counted in and delivered.  See
      description about the changeover procedure later in this section.
      The Blocked state may also be entered and left through the
      management commands BLOCK and UNBLOCK.  This is also described
      later.

   A newly created link endpoint starts from the state Reset-Unknown.
   The recommended default value for LINK_TOLERANCE is 0.8 sec.

3.7.2  Link Continuity Check

   During normal traffic both link enpoints are in state
   Working-Working.  At each expiration point, the background timer
   checkpoints the value of the Last Received Sequence Number.  Before
   doing this, it compares the check- point from the previous expiration
   with the current value of Last Received Sequence Number, and if they
   differ, it takes the new checkpoint and goes back to sleep.  If the
   two values don't differ, it means that nothing was received during
   the last interval, and the link endpoint must start probing, i.e.
   move to state Working-Unknown.

   Note here that even LINK_PROTOCOL messages are counted as received
   traffic, altough they don't contain valid sequence numbers.  When a
   LINK_PROTOCOL message is received, the checkpoint value is
   moved,instead of Last Received Sequence Number, and hence the next
   comparison gives the desired result.

3.7.3  Sequence Control and Retransmission

   Each packet eligible to be sent on a link is assigned a Link Level
   Sequence Number, and appended to a send queue associated with the
   link endpoint.  At the moment the packet is sent, its field Link
   Level Acknowledge Number is set to the value of Last Received
   Sequence Number.

   When a packet is received in a link endpoint, its send queue is
   scanned, and all packets with a sequence number lower than the
   arriving packet's acknowledge number (modulo 2^16-1) are released.

   If the packet's sequence number is equal to Last Received Sequence



Maloy, et al.            Expires April 27, 2005                [Page 51]


Internet-Draft                    TIPC                      October 2004


   Number + 1 (mod 2^16-1), the counter is updated, and the packet is
   delivered upwards in the stack.  A counter, Non Acknowledged Packets,
   is incremented for each message received, and if it reaches the value
   10, a LINK_PROTOCOL/STATE_MSG is sent back to the sender.  For any
   message sent, except BCAST_PROTOCOL messages, the Non Acknowledged
   Packets counter is set to zero.

   Otherwise, if the sequence number is lower, the packet is considered
   a duplicate, and is silently discarded.

   Otherwise,if a gap is found in the sequence, the packet is sorted
   into the Deferred Incoming Packets Queue associated to the link
   endpoint, to be re-sequenced and delivered upwards when the missing
   packets arrive.  If that queue is empty,the gap is calculated and
   immediately transferred in a LINK_PROTOCOL/STATE_MSG back to the
   sending node.  That node must immediately retransmit the missing
   packets.  Also, for each 8 subsequent received out-of-sequence
   packets, such a message must be sent.

3.7.4  Message Bundling

   Sometimes a packet can not be sent immediately over a bearer, due to
   network or recipient congestion (link level send window overflow), or
   due to bearer congestion.  In such situations it is important to
   utilize the network and bearer as efficiently as possible, and not
   stop important users from sending messages before this is absolutely
   unavoidable.  To achieve this, messages which can not be transmitted
   immediately are bundled into already waiting, packets whenever
   possible, i.e.  when there are unsent packets in the send queue of a
   link.  When the packet finally arrives at the receiving node it is
   split up to its individual messages again.  Since the bundling layer
   is located below the fragmentation layer in the functional model of
   the stack, even message fragments may be bundled with other messages
   this way, but this can only happen to the last fragment of a message,
   the only one normally not filling an entire packet by itself.

   It must be emphasized that message transmissions never are delayed in
   order to obtain this effect.  In contrast to TCP's Nagle Algorithm,
   the only goal of the TIPC bundling mechanism is to overcome
   congestion situations as quickly and efficiently as possible.

3.7.5  Message Fragmentation

   When a message is longer than the identified MTU of the link it will
   use, it is split up in fragments, each being sent in separate packets
   to the destination node.  Each fragment is wrapped into a packet
   headed by an TIPC internal header (see Section 4.2).  The User field
   of the header is set to MSG_FRAGMENTER, and each fragment is assigned



Maloy, et al.            Expires April 27, 2005                [Page 52]


Internet-Draft                    TIPC                      October 2004


   a Fragment Number relative to the first fragment of the message.
   Each fragmented message is also assigned a Fragmented Message Number,
   to be present in all fragments.  Fragmented Message Number must be a
   sequence number with the period of 2^16-1.  At reception the
   fragments are reassembled so that the original message is recreated,
   and then delivered upwards to the destination port.

3.7.6  Link Congestion Control

   TIPC uses a common sliding window protocol to handle traffic flow at
   the signalling link level.  When the send queue associated to each
   link endpoint reaches a configurable limit, the Send Window Limit,
   TIPC stop sending messages over that link.  Packets may still be
   appended to or bundled into waiting packets in the queue, but only
   after having been subject to a filtering function, selecting or
   rejecting user calls according to the sent message's importance
   priority.  DATA_LOW messages are not accepted at all in this
   situation.  DATA_NORMAL messages are still accepted, up to a
   configurable limit set for that user.  All other users also have
   their individually configurable limits, recommended to be assigned
   values in the following ascending order: DATA_LOW, DATA_NORMAL,
   DATA_HIGH, DATA_NONREJECTABLE, CONNECTION_MANAGER,BCAST_PROTOCOL,
   ROUTE_DISTRIBUTOR, NAME_DISTRIBUTOR, MSG_FRAGMENTER.  MSG_BUNDLER
   messages are not filtered this way, since those are packets created
   at a later stage.  Whether to accept a message due for fragmentation
   or not is decided on its original importance, set before the
   fragmentation is done.  Once such a message has been accepted, its
   individal fragments must be handled as being more important than the
   original message.

   When the part of the queue containing sent packets again is under the
   Send Window Limit, the waiting packets must immediately be sent, but
   only until the Send Window Limit is reached again.

3.7.7  Bearer Congestion Control

   When the local bearer media becomes overloaded, e.g.  when an
   Ethernet circuit runs out of send buffers, the Bearer Congestion
   Control function may be activated.  This function keeps track of the
   current state of the bearer, and stops accepting any packet send
   calls until the bearer is ready for it again.  During this interval
   TIPC users may still perform send calls, and packets will be
   accumulated in the affected links send queues according to the same
   rules as for Link Congestion Control, but all actual transmission is
   stopped.

   When the congestion has abated, the bearer opens up for traffic
   again, and the links having packets waiting to be sent are scheduled



Maloy, et al.            Expires April 27, 2005                [Page 53]


Internet-Draft                    TIPC                      October 2004


   round-robin for sending their unsent packets.  This level of
   congestion control is optional, and its activation should be
   configurable.

3.7.8  Link Load Sharing vs Active/Standby

   When a link is created it is assigned a Link Priority, used to
   determine its relation to a possible parallel link to the same node.
   There are two possible relations between parallel working links.
   Load Sharing

      Load Sharing is used when the links have the same priority
      value.Payload traffic is shared equally over the two links, in
      order to take full advantage of available bandwidth.  The
      selection of which link to use must be done in a deterministic
      way, so that message sequentiality can be preserved for each
      individual sender port.  To obtain this a Link Selector is used.
      This must be value correlated to the sender in such a way that all
      messages from that sender choose the same link, while guaranteeing
      a statistically equal possibility for both links to be selected
      for the overall traffic between the nodes.  A simple example of a
      link selector with the right properties is the last two bits of
      the random number part of the originating port's identity, another
      is the same bits in Fragmented Message Number in message
      fragments.
   Active/Standby

      When the priority of one link has a higher numeral value than that
      of the other, all traffic will go through that link, denoted the
      Active Link.  The other link is kept up and working with the help
      of the continuity timer and probe messages, and is called the
      Standby Link.  The task of this link is to take over traffic in
      case the active link fails.

   Link Priority has a value within the range [1,31].  When a link is
   created it inherits a default priority from its corresponding bearer,
   and this should normally not need to be changed thereafter.  However,
   Link Priority must be reconfigurable in run-time.

3.7.9  Link Changeover

   When the link configuration between two nodes changes, the moving of
   traffic from one link to another must be performed in such a way that
   message sequentiality and cardinality per sender is preserved.  The
   following situations may occur:






Maloy, et al.            Expires April 27, 2005                [Page 54]


Internet-Draft                    TIPC                      October 2004


   Active Link Failure

      Before opening the remaining link for messages with the failing
      link's selector, all packets in the failing link's send queue must
      wrapped into messages (tunneling messages) to be sent over the
      remaining link, irrespective of whether this is a load sharing
      active link or a standby link.  These messages are headed by a
      TIPC Internal Header, the User field set to CHANGEOVER_PROTOCOL,
      Message Type set to ORIG_MSG.  On the tunneling link the messages
      are subject to congestion control, fragmentation and bundling,
      like any other messages.  Upon arrival in the arriving node, the
      tunneled packets are unwrapped, and moved over to the failing
      links receiving endpoint.  This link endpoint must now be reset,
      if it has not already been done, and itself initiate tunneling of
      its own queued packets in the opposite direction.  The unwrapped
      packets' original sequence numbers are compared to Last Received
      Sequence Number of the failed links receiving endpoint, and are
      delivered upwards or dropped according to their relation to this
      value.  There is no need for the failing link to consider packet
      sequentiality or possible losses in this case, - the tunneling
      link must be considered a reliable media guaranteeing all the
      necessary properties.  The header of the first ORIG_MSG sent in
      each direction must contain a valid number in the Message Count
      field, in order to let the receiver know how many packets to
      expect.  During the whole changeover procedure both link endpoints
      must be blocked for any normal message reception, to avoid that
      the link is inadvertently activated again before the changeover is
      finished.  When the expected number of packets has been received,
      the link endpoint is deblocked, and can go back to the normal
      activation procedure.
   Standby Link Failure

      This case is trivial, as there is no traffic to redirect.
   Second Link With Same Priority Comes Up

      When a link is active, and a second link with the same priority
      comes up, half of the traffic from the first link must be taken
      over by the new link.  Before opening the new link for new user
      messages, the packets in the existing link's send queue with a
      link selector corresponding to the new link must be transmitted
      over that link.  This is done by wrapping copies of these packets
      into messages (tunnel messages) to be sent over the new link.  The
      tunnel messages are headed by a TIPC Internal Header, the User
      field set to CHANGEOVER_PROTOCOL, Message Type set to
      DUPLICATE_MSG.  On the tunneling link the messages are subject to
      congestion control, fragmentation and bundling, just like any
      other messages.  Upon arrival in the arriving node, the tunneled
      packets are unwrapped, and delivered to the original links



Maloy, et al.            Expires April 27, 2005                [Page 55]


Internet-Draft                    TIPC                      October 2004


      receiving endpoint, just like any other packet arriving over that
      link's own bearer.  If the original packet has already arrived
      over that bearer, the tunneled packet is dropped as a duplicate,
      otherwise the tunneled packet will be accepted, and the original
      packet dropped as a duplicate when it arrives.
   Second Link With Higher Priority Comes Up

      When a link is active, and a second link with a higher numerical
      priority comes up, all traffic from the first link must be taken
      over by the new link.  The handling of this case is identical to
      the case when a link with same priority comes up.  After the
      traffic takeover has finished, no more senders will select the old
      link, but this does not affect the takeover procedure.

3.8  Routing

   TIPC support routing of packets when necessary, both between
   clusters, between zones, and between system nodes and secondary
   nodes.

3.8.1  Routing Algorithm

   Available routes to a remote zone, cluster or secondary node are
   explored in the following order:

   First, look for any direct links to the destination node, and if
   there are any, select one using the normal link selection algorithm.

   Second, if no direct link is found, and if the message is an
   inter-cluster message, look for a direct link to any node within the
   remote zone/cluster, and send the message there.  This selection must
   be performed in a deterministic way, to minimize the risk for
   disordered message arrivals.  If the destination or sender of the
   message is a secondary node, this step of the lookup is omitted.

   Third, if there are no such direct links,the algorithm must look for
   a node within the own cluster, known to have a direct link to the
   destination node.  Selection of this intermediate node must also be
   done in a deterministic way, e.g.  using the Link Selector.  If the
   destination or origin of the message is a secondary node, and if no
   router node is found, the message must be rejected with error code
   TIPC_NO_REMOTE_NODE.

   As last resort, if all previous attempts have failed, the algorithm
   will look for any cluster local node known to have a link to any node
   within the remote zone/cluster.  If no such router node is found, the
   message must be rejected with error code TIPC_NO_REMOTE_NODE.




Maloy, et al.            Expires April 27, 2005                [Page 56]


Internet-Draft                    TIPC                      October 2004


3.8.2  Routing Table

   Each node in a cluster must be kept up-to-date about all available
   routes to secondary nodes, external clusters, and external zones.
   This is done by letting each node broadcast any change in its direct
   connectivity to all the other nodes in the cluster, the same way the
   nameing table is kept updated.  Because of the multiplicative effect,
   the number of potential routes between two big zones may be very
   large, and the routing table structure must reflect this.  As an
   extreme scenario, take a zone with 1000 nodes, divided into five
   clusters with 200 nodes each, each node having two links to two
   different nodes in the foreign clusters, and dual links to all nodes
   within the local cluster.  Each node would have to store knowledge
   about 999 external  nodes, 800 of those representing nodes in the
   remote clusters.  For each of these 800 nodes, information about 2
   routes must be stored, apart from the four direct inter-cluster
   links.

   To continue this example, we see that each node would have 199 x 2 +
   5 x 2 = 308 links to maintain.  With a continuity timer for each link
   expiring e.g.  every 400 msec, corresponding to a link tolerance of
   1.6 sec, there would be 600 expiring timers per second on each node.
   Again, assuming a a worst-case scenario with idle links, 1200 probe
   messages would have to be sent and received per second per node.  To
   handle a kernel timer and send a probe message takes less than 5
   usecs of CPU-time on a single 2 Ghz CPU, and to receive and handle a
   probe at kernel level takes about the same time.  Hence, the
   background load for maintaining all necessary links even within such
   a huge zone would not exceed 2.5%.

3.8.3  Routing Table Updates

   There are five different cases when a node's routing table must be
   updated:

   A link towards a zone/cluster external node comes up:
   o  Broadcast a ROUTE_ADDITION message to all system nodes within the
      own cluster, informing them that the new destination can be
      reached via this node.

   A link towards a secondary node comes up:
   o  Broadcast a ROUTE_ADDITION message to all system nodes within the
      own cluster, informing that the new destination can be reached via
      this node.
   o  Send a LOCAL_ROUTING_TABLE message to the secondary node,informing
      about existence of all system nodes within cluster.

   A new cluster local system node becomes available:



Maloy, et al.            Expires April 27, 2005                [Page 57]


Internet-Draft                    TIPC                      October 2004


   o  Send a SEC_ROUTING_TABLE message to the new node, containing
      information about all cluster secondary nodes which can be reached
      via this node.
   o  Send an EXT_ROUTING_TABLE message to the new node, containing
      information about all cluster external nodes which can be reached
      via this node.
   o  Send a ROUTE_ADDITION message to all directly connected secondary
      nodes, informing about the existence of the new node.

   The direct contact with a zone/cluster external node or secondary
   node is lost:

   A cluster local system node becomes unavailable:
   o  Remove all references to this node from the local routing tables.
      This is a completely node local operation.
   o  Send ROUTE_REMOVAL messages to all directly connected secondary
      nodes, informing them about the loss of the node.

   The specific message structure used in these situations is described
   in Section 4.

3.9  Multicast Transport

   To effectively support the functional multicast service described in
   a previous section, a reliable cluster broadcast service is provided
   by TIPC.

   Although seen as a broadcast service from a TIPC viewpoint, at the
   bearer level this service is typically implemented as a multicast
   group comprising all nodes in the cluster.

   At the multicast/broadcast sending node a sequence of actions is
   followed:
   o  When a functional multicast is requested, TIPC first looks up all
      matching destinations in its name translation table.
   o  The destination addresses are filtered down to a list containing
      the network addresses of and the exact number of destination nodes
      in the indicated lookup domain.
   o  If the own node is on the list, a replica is sent to the
      functional multicast receive function in the own node.
   o  If the lookup domain indicated goes beyond the own cluster, a
      replica of the message is sent to the identified external zones or
      clusters.  There the message is subject to a new multicast lookup
      and cluster broadcast, if necessary.

3.9.1  Conditional Cluster Broadcast

   If the lookup domain comprises the node's own cluster, and if the



Maloy, et al.            Expires April 27, 2005                [Page 58]


Internet-Draft                    TIPC                      October 2004


   number of identified target nodes in the cluster is less than the
   configurable parameter Broadcast Limit(recommended default value: 8),
   or if no cluster broadcast is supported, a replica of the multicast
   message is sent via a link to each of these nodes.  Since a link can
   be considered a reliable media, performing fragmentation and
   retransmission if necessary, we can trust that the replicas will
   reach their destinations, and no more action need to be taken.

   Otherwise, if the number of destination nodes exceeds Broadcast
   Limit, and a cluster broadcast service is available, the message is
   sent as one or more broadcast packets to all nodes in the cluster.
   If necessary the message is fragmented into packets to fit into the
   MTU of the media.  Each packet is assigned a broadcast sequence
   number and added to a broadcast transmission/retransmission queue
   before being sent.  If there is no congestion in the bearer, or the
   transmission queue has not exceeded the Broadcast Window Limit, the
   packet or packets are sent immediately.  If there is any congestion,
   the packets are queued according to their importance, bundled into
   waiting buffers if possible.  Broadcast Window Limit should be
   configurable, with a recommended default value of 48.  All this is
   analogous to how the node-to-node link protocol works.

   If not already running, a background timer is started, to expire at a
   Broadcast Continuity Interval.  Broadcast Continuity Interval is
   recommended to be 16 * RTT, or, since most OS:es don't support timers
   of that resolution, as fast as the OS can support.  As long as there
   are non-acknowledged packets in the broadcast out-queue, the timer
   continues to run, but no more timers are started.

   At next expiration the timer checks the acknowledge status for each
   destination node, and releases all buffers which have been
   acknowledged by all nodes in the cluster.  In order to keep the
   send-queue as short as possible at any time, this step is also
   performed at each attempted sent broadcast, at each received
   multicast, and at each received BCAST_PROTOCOL/STATE_MSG.

3.9.2  Conditional Tunneled Retransmission

   For the still unacknowledged packets sent before the previous timer
   expiration, and for packets older than 16 positions from the last
   sent packet in the send queue, the timer function calculates how to
   do the retransmission.  If the number of missing nodes in the
   acknowledge list for a message is less than Broadcast Limit, it sends
   a replica of that packet via a link to each of the missing nodes.
   Because the packet must be recognized as a missing broadcast at the
   receiving node, it is tunneled over the link, i.e.  wrapped into a
   packet of type BCAST_PROTOCOL/BCAST_MSG.  Since a link can be trusted
   as a reliable media, the original packet is now discarded.  This step



Maloy, et al.            Expires April 27, 2005                [Page 59]


Internet-Draft                    TIPC                      October 2004


   is also performed at each received BCAST_PROTOCOL/STATE_MSG
   containing a non-zero sequence gap field.

   Otherwise, If the number of missing acknowledging nodes is larger
   than Broadcast Limit, the unacknowledged packets are re-broadcast
   again.  Note that packets sent after the previous timout expiration
   are not retransmitted, because those may potentially have been sent
   immediately before the current timer expired.

   In order to keep all receiving nodes updated about sent broadcast
   packets, even during low traffic, each sent LINK_PROTOCOL/STATE_MSG
   contains the sequence number of the "next sent broadcast packet from
   this node".  This gives the receiving nodes an opportunity of early
   detection of lost packets,and to send a BCAST_PROTOCOL/STATE_MSG
   demanding retransmission.

   When receiving a cluster broadcast, or a tunneled retransmission,the
   following is is done at the receiving node:
   o  The Broadcast Sequence Number is checked against the Next Expected
      Broadcast Packet counter for the corresponding sender node, and if
      it fits, this counter is updated.
   o  Otherwise, if the packet turns out to be a duplicate, it is
      silently discarded.
   o  Otherwise, if a gap is found in the sequence, the packet is queued
      in a Deferred Incoming Multicast Packets Queue, to be resequenced
      and delivered upwards later.  If the last 4 bits of the sequence
      number are equal to the last 4 bits of the node's Sequence Tag,
      AND if this queue was empty,the gap is calculated and immediately
      transferred in a BCAST_PROTOCOL/STATE_MSG back to the sending node
      This must immediately retransmit the missing packets.  Also, for
      each 8 received out-of-sequence packet fulfilling the sequence
      criteria described above, such a message must be sent.  All this
      is analogous to how the node-to-node link protocol works.

3.9.3  Piggybacked Acknowledge

   All packets, without exception, passed from one node to another,
   contain a valid value in the field Acknowledged Bcast Number.  Since
   there is always some traffic going on between all nodes in the
   cluster (in the worst case only link supervision messages), the
   receiving node can trust that the Last Acknowledged Bcast counter it
   has for each node is kept well up-to-date.  This value will under no
   circumstances be older than one CONTINUITY_INTERVAL, so it will
   inhibit a lot of unnecessary retransmissions of packets which in
   reality have already be received at the other end.






Maloy, et al.            Expires April 27, 2005                [Page 60]


Internet-Draft                    TIPC                      October 2004


3.9.4  Coordinated Acknowledge Interval

   If the received packet fits in sequence as described above, AND if
   the last four bits of the sequence number of the packet received are
   equal to the last four bits of the own node's Sequence Tag, AND if
   the difference to the last sent piggybacked Acknowledged Bcast Number
   to that node is more than 8, a BCAST_PROTOCOL/STATE_MSG is generated
   and sent back to the receiving node, acknowledging the packet, and
   implicitly all previously received packets.  This means that e.g.
   node <Z.C.1> will only explicitly acknowledge packet number 1, 17,
   33, and so on, node number <Z.C.2> will acknowledge packet number 2,
   18, 34, etc.  This condition significantly reduces the number of
   explicit acknowledges needing to be sent, taking advantage of the
   normally ongoing traffic over each link.

   A node's Sequence Tag is the node's sequence number in relation to
   the network address of the other nodes in the cluster.  E.g, if a
   cluster consists of the nodes <1.1.17>, <1.1.123> and <1.1.555>,
   those will assign themselves the sequence tags 1, 2, and 3,
   respectively.  This way, we make the protocol behaviour independent
   of how node addresses are assigned in the cluster.  The sequence tags
   must be recalculated locally on each node when the cluster topology
   changes.

3.9.5  Replicated Delivery

   When an in-sequence functional multicast is finally is delivered
   upwards in the stack, be it via cluster broadcast,replicated
   multicast, or tunneled broadcast retransmission, TIPC looks up in the
   naming table and finds all node local destination ports.  The
   destination list created this way is stripped of all duplicates, so
   that only one message replica is sent to each identified destination
   port.

3.9.6  Congestion Control

   Messages sent over the "broadcast link" are subject to the same
   congestion control mechanisms as point-to-point links, with
   prioritized transmission queue appending, message bundling, and as
   last resort a return value to the sender indicating the congestion.
   Typically this return value is taken care of by the socket layer
   code, blocking the sending process until the congestion abates.
   Hence, the sending application should never notice the congestion at
   all.

3.10  Fault Handling

   Most functions for improving system fault tolerance are described



Maloy, et al.            Expires April 27, 2005                [Page 61]


Internet-Draft                    TIPC                      October 2004


   elswhere, under the repective functions, but some aspects deserve
   being mentioned separately.

3.10.1  Fault Avoidance

   Strict source address check

      After the neighbour detection phase, a message arriving to a node
      must have a not only a valid Pevious Node address, but this must
      belong to one of the nodes known having a direct link to the
      destination.  The node may in practice be aware of at most a few
      hundred such nodes, while a network address is 32 bits long.  The
      risk of accepting a garbled message having a valid address within
      that range, a sequence number that fits into the reception window,
      and otherwise valid header fields, is extremely small, no doubt
      less than one to several billions.

   Sparse port address space

      As an extra measure, TIPC uses a 32-bit pseudo-random number as
      the first part of a port identity.  This gives an extra protection
      against corrupted messages, or against obsolete messages arriving
      at a node after long delays.  Such messages will not find any
      destination port, and be attempted returned to the sender port.
      If there is no valid sender port, the message should be quietly
      discarded.

   Name Table Keys

      When a naming table is updated with a new publication, each of
      those are qualified with a Key field, that is only known by the
      publishing port.  This key must be presented and verified when the
      publication is withdrawn, in all instances of the naming table.
      If the key does not fit, the withdrawal is refused.

   Link Selectors

      Whenever a message/packet is sent or routed, the link used for the
      next-hop transport is always selected in a deterministic way,
      based on the sender port's random number.  The risk of having
      packets arriving in disorder is hence non-existent for single-hop
      messages, and extremely low for multi-hop messages.

   Repeated Name Lookups

      If a lookup in the naming table has returned a port identity that
      later turns out to be false, TIPC performs up to 6 new lookups
      before giving up and rejecting the message.



Maloy, et al.            Expires April 27, 2005                [Page 62]


Internet-Draft                    TIPC                      October 2004


   Routing Counter

      To eliminate the risk of having messages roaming around in the
      network a routing counter is present in the TIPC header.  This
      counter is updated for each inter-node hop and for each naming
      table lookup the message is subject to.  If this counter reaches
      the upper limit, seven, the message is rejected back to the sender
      port.

3.10.2  Fault Detection

   The mechanisms for fault detection have been described in previous
   sections, but some of them will be briefly repeated here:
   o  Transport Level Sequence Number, to detect disordered multi-hop
      packets.
   o  Connection Supervision and Abortion mechanism.
   o  Link Supervision and Continuation control.

3.10.3  Fault Recovery

   When a failure has been detected, several mechanisms are used to
   eliminate the impact from the problem, or when that is impossible, to
   help the application to recover from it:
   Link Changeover

      When a link fails, its traffic is directed over to the redundant
      link, if any, in such a way that message sequentiality and
      cardinality is preserved.  This feature is described in Section
      3.7.9.
   Returning Messages to Sender

      When no destination is found for a message, the 1024 first bytes
      of it is returned to the sner port, along with an approriate error
      code.  This helps the application to identify the exact instant of
      failure, and if possible, to find a new destination for the failed
      call.  The complete list of error codes and their significance is
      described in Section 4.

3.10.4  Overload Protection

   To overcome situations where the congestion/flow control mechanisms
   described earlier in this section are inadequate or insufficient,
   TIPC must provide two additional overload protection services:
   Node Overload Protection

      TIPC must maintain a global counter on each node, keeping track of
      the total number of pending, unhandled payload messages on the
      node.  When this counter reaches a critical value, which should be



Maloy, et al.            Expires April 27, 2005                [Page 63]


Internet-Draft                    TIPC                      October 2004


      configurable, TIPC must selectively reject new incoming messages.
      Which messages to reject must be based on the following criteria:
      *  Message importance.  DATA_LOW messages should be rejected at a
         lower threshold than DATA_NORMAL messages, which should be
         rejected before DATA_HIGH messages.  DATA_NONREJECTABLE should
         not be rejected at all.
      *  Message type.  Connectionless messages should be rejected
         earlier than connection oriented messages.  Rejecting such
         messages normally means rejecting a service request form the
         beginning, causing less disturbances than interrupting a
         transaction already in progress, e.g.  an ongoing phone call.
   Process Overload Protection

      TIPC must maintain a counter for each process, or if this is
      impossible, for each port, keeping track of the  total number of
      pending, unhandled payload messages on that process or port.  When
      this counter reaches a critical value, which should be
      configurable, TIPC must selectively reject new incoming messages.
      Which messages to reject should be based on the same criteria as
      for the node overload protection mechanism, but all thresholds
      must be set significantly lower.  Empirically a ratio 2:1 between
      the node global thresholds and the port local thresholds has been
      working well.




























Maloy, et al.            Expires April 27, 2005                [Page 64]


Internet-Draft                    TIPC                      October 2004


4.  TIPC Packet Format

   A TIPC packet is composed of a header and a user data part.  Two
   different header formats are used, one for payload (user-to-user)
   messages, and one for TIPC internal protocol messages.

4.1  TIPC Payload Message Header

4.1.1  Payload Message Header Format


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0:|vers | user  |hdr sz |n|d|rsv|          message size           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1:|mstyp| error |rer cnt|lsc|opt p|      broadcast ack no         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w2:|        link level ack no      |   broadcast/link level seq no |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w3:|                       previous node                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4:|                      originating port                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w5:|                      destination port                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6:|                      originating node                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7:|                      destination node                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8:|             name type / transport sequence number             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w9:|              name instance/multicast lower bound              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   wA:|                    multicast upper bound                      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      /                                                               /
      \                           options                             \
      /                                                               /
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


             Figure 25: TIPC payload message header format

   Each 32-bit word in the header is transmitted as an integer coded in
   network byte order.

   The first four words of the header have an identical format in all



Maloy, et al.            Expires April 27, 2005                [Page 65]


Internet-Draft                    TIPC                      October 2004


   messages, independently of whether they are internal or payload
   messages.

4.1.2  Payload Message Header Field Descriptions

   Version: 3 bits

      To enable future upgrades the protocol version must be present in
      the header.The current version of the protocol is 2.

   User: 4 bits

      This field not only indicates whether the message is a protocol
      message or a data (payload) message, but in the latter case even
      the importance priority of the user.  The following values are
      used:

      ID Value    User
      --------    ----------
      0           Low Priority Payload Data    (DATA_LOW)
      1           Normal Priority Payload Data (DATA_NORMAL)
      2           High Priority Payload Data   (DATA_HIGH)
      3           Non-Rejectable Payload Data  (DATA_NON_REJECTABLE)

   Header Size: 4 bits

      Since header size both is variable and has been given a semantic
      meaning it must be present in the header.  This feature also
      provides forward compatibility in case we need to extend the
      header format in the future.  If options are present the header
      size will comprise the options size.  The unit used is 32 bit
      words, implying that the maximum allowed header size is 60 bytes.

   Non-sequenced: 1 bit

      Indicates whether a packet belongs to the sequence of a link or
      not.  Broadcast packets and neighbour detetction packet have this
      bit set.  If the packet is a broadcast it must be subject to
      special treatment at the receiving node, with some of the fields
      having a different meaning than in the unicast case.  This is
      described in section 3.

   Drop: 1 bit

      If this bit is set,the message should be dropped in case of
      congestion or overload,instead of being returned to the sender.  A
      connection may have this bit set to 1 for messages going in one
      direction, and to 0 for those going in the opposite direction.



Maloy, et al.            Expires April 27, 2005                [Page 66]


Internet-Draft                    TIPC                      October 2004


      Default value is 0.

   Reserved: 2 bits

      Unused in this protocol version, but must be set to zero in order
      to facilitate compatibility if used in the future.

   Message Size: 17 bits

      This field indicates the size of the complete message, end-to-end,
      inclusive header size.  The maximum size of a data message is
      internally set to 66060 bytes, i.e.  66000 bytes of user data plus
      plus a maximal header, inluding options.  The limit of 66000 was
      chosen to make it possible to tunnel maximum-size IP-messages
      through TIPC, but technically this can easily be extended, since
      there is an adjacent unused field of three bits.

   Message Type: 3 bits

      ID Value    User
      --------    --------------------------
      0           Message sent on connection             (CONN_MSG)
      1           Logical multicast message              (MCAST_MSG)
      2           Message with port name destination     (NAMED_MSG)
                  address
      3           Message with port identity destination (DIRECT_MSG)
                  address

   Error Code: 4 bits

      ID Value    Meaning
      --------    ----------
      0           No error                        (TIPC_OK)
      1           Destination port name unknown   (TIPC_ERR_NO_NAME)
      2           Destination port does not exist (TIPC_ERR_NO_PORT)
      3           Destination node unavailable    (TIPC_ERR_NO_NODE)
      4           Destination node overloaded     (TIPC_ERR_OVERLOAD)
      5           Connection Shutdown (No error)  (TIPC_CONN_SHUTDOWN)

   Reroute Counter: 4 bits

      This counter has the purpose of stopping messages from roaming
      around in the system.  This may, at least theoretically, happen in
      case of temporary naming table or routing table inconsistency.
      The counter is incremented each time a lookup is done in the
      naming table, and each time the message makes an inter-node hop.
      When the counter reaches a limit, (seven in the current
      implementation) the counter is reset and the message is rejected



Maloy, et al.            Expires April 27, 2005                [Page 67]


Internet-Draft                    TIPC                      October 2004


      with the appropriate error code.

   Lookup Scope: 2 bits

      When a port name has been successfully translated to a port
      identity, the field "Destination Node" is filled with a complete
      node address.  This also means that the scope of the original
      lookup domain is lost, since this is indicated by the value of
      this field before the lookup.  Sometimes, e.g.  because of
      temporary inonsistency ot the naming table during update, the
      destination port turns out to not exist, and one or more new
      lookups must be performed.  In order to do this correctly, the
      original lookup scope must be preserved in the message, and that
      is done in this field.  The following values apply:

      ID Value    Meaning
      --------    ----------
      1           Zone Scope
      2           Cluster Scope
      3           Node Scope

      The lookup domain is recreated based on the complete destination
      node address and the lookup scope.

   Options Position: 3 bits

      The position of the first word of the options field, if any.  If
      zero there are no options.  Otherwise it will have values between
      1 (word 8/byte 32) and 7 (word 14/byte 56).

   Broadcast Acknowledge Number: 16 bits

      All messages, irrespective of user and type, carry a valid value
      in this field.  It informs the recipient about the last
      in-sequence broadcast packet received from the recipient node in
      the sender node.  This gives the recipient node a chance to
      release sent broadcast buffers, or to retransmit broadcast packets
      in case it discovers a lag-behind for this node.

   Link Level Acknowledge Number: 16 bits

      All messages, except broadcast messages and LINK_PROTOCOL messages
      of type RESET_MSG and ACTIVATE_MSG, carry a valid value in this
      field.  It informs the recipient about the last in-sequence packet
      received on this link in the sender node.  This gives the
      recipient node a chance to release sent buffers.

   Link Level Sequence Number: 16 bits



Maloy, et al.            Expires April 27, 2005                [Page 68]


Internet-Draft                    TIPC                      October 2004


      All non-broadcast packets, except LINK_PROTOCOL, contain a
      sequence number valid for their particular link, in order to keep
      track of message flow, detect packet losses, duplicates or
      out-of-sequence packets.  The first packet sent after a link reset
      has the sequence number 0.

   Broadcast Sequence Number: 16 bits

      All broadcast packets contain a sequence number valid for the
      sending node, in order to keep track of message flow, detect
      packet losses, duplicates or out-of-sequence packets.  Since such
      packets are unrelated to any links, and are easily identified by
      the 'broadcast' bit, we can reuse the physical area of the 'Link
      Level Sequence Number' for this field.

   Previous Node: 32 bits

      The network address of the last node visited by the message.  In
      the case of intra-cluster messages this is most often, but not
      always, identical to the node from which the message originates.

   Originating Port: 32 bits

      This field is specific for data messages, and contains the random
      number identifying the originating port locally on the originating
      node.

   Destination Port: 32 bits

      The random number identifying the destination port on the
      destination node.  For  NAMED_MSG and MCAST_MSG messages this
      field is set to zero until a lookup in the naming table has found
      a destination.  As long as the value remains zero new lookups will
      be performed until a destination is found, or until 'Reroute
      Counter' reaches the upper limit.

   Originating Node: 32 bits

      The node from which the message originally was sent.

   Destination Node: 32 bits

      The final destination node for a message, when known.  For port
      name addressed messages this field has a slightly different
      meaning before and after the final destination is determined.  See
      Section 3.2.8.

   Transport Level Sequence Number: 32 bits



Maloy, et al.            Expires April 27, 2005                [Page 69]


Internet-Draft                    TIPC                      October 2004


      For port named messages a connection sequence number has no
      meaning, just as connection based messages never contain a port
      name.  Because of this mutual exclusion we can use the same
      physical space for both these fields.  The connection sequence
      number is only defined and checked for potentially routed
      messages, i.e.  for messages passed between different clusters, or
      between secondary nodes and system nodes.  The first message sent
      in each direction on such a connection has the sequence number 42.

   Port Name Type: 32 bits

      The type part of a destination port name or port name sequence.

   Port Name Instance: 32 bits

      The instance part of a destination port name.

   Port Name Sequence Lower: 32 bits

      The 'lower' boundary of a destination port name sequence.  Uses
      the same physical field as 'Port Name Instance' described above.

   Port Name Sequence Upper: 32 bits

      The 'upper' boundary of a destination port name sequence.

4.1.3  Payload Message Header Size

   The header is organized so that it should be possible to omit certain
   parts of it, whenever any information is dispensable.  The following
   header sizes are used:

   Cluster Internal Connection Based Non-Routed Messages:

      Such messages per definition do only one hop over an inherently
      reliable link, so all fields from word 6 and onwards are redundant
      or irrelevant.  The message header can be limited to 24 bytes.  By
      ensuring that no other messages have this particular header size,
      this can indeed be used as a test that we are dealing with that
      kind of message, and some code optimization can be done based on
      this knowledge.

   Direct Addressed Messages:

      These are connection-less messages containing a port identity as
      destination address, i.e.  the fields 'destination port' and
      'destination node' are filled in and non-zero.  All fields from
      word 7 and onwards are irrelevant, and the message size can be set



Maloy, et al.            Expires April 27, 2005                [Page 70]


Internet-Draft                    TIPC                      October 2004


      to 32.

   Connection Based Potentially Routed Messages:

      Inter cluster connection based messages, and intra-cluster
      messages between cluster nodes and secondary nodes, may need to be
      routed via intermediate nodes if there is no direct link between
      the two.  'Originating node' may hence differ from 'previous
      node', so this field must be present.  Since there is now a small,
      but not negligeable risk that messages may be lost or arrive in
      disorder (the intermediate node may crash), a transport level
      connection sequence number is needed for problem detection.  A
      header size of 36 bytes is required.

   Port Name Addressed Messages:

      These are connection-less messages containing a port name as
      destination address, i.e.  the fields 'name type' and 'name
      instance' have valid values, while 'destination port' is zero
      before the name table lookup, and non-zero after a sucessful
      lookup.  'Destination node' may be zero or have a valid value
      before lookup,but has a valid value after a sucessful lookup.  The
      header size is set to 40.

   Multicast Messages:

      Multicast messages are similar to port name addressed messages,
      except that the destination address is a range (port name
      sequence) rather than a port name.  An extra word, the 'upper'
      part of the port name sequence must be present, so the header size
      ends up at 44.

   Messages with Options:

      All message headers may exceptionally be appended with an
      'options' field, e.g.  containing tracing information or a time
      stamp.  In this case the total header length may take any
      four-byte aligned value up to the maximum, 60.  There is one
      anomaly here, however.  The non-routed 24-byter header can not
      take any option without invalidating the semantic meaning of the
      header size.  Hence, such headers must be expanded to the full 36
      bytes before any options can be added.

4.2  TIPC Internal Header

4.2.1  Internal Message Header Format





Maloy, et al.            Expires April 27, 2005                [Page 71]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0:|vers |msg usr|hdr sz |n|resrv|            packet size          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1:|m typ|bcstsqgap| sequence gap  |       broadcast ack no        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w2:|        link level ack no      |   broadcast/link level seq no |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w3:|                       previous node                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4:|  next sent broadcast/fragm no | next sent pkt/ fragm msg no   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w5:|          session no           | res |r|berid|link prio|netpl|p|
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6:|                      originating node                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7:|                      destination node                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8:|                  transport sequence number                    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w9:|          msg count            |       link tolerance          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                     User Specific Data                        /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


            Figure 30: TIPC Internal Message Header  Format

   The internal header has one format and one size, 40 bytes.  Some
   fields are only relevant to some users, but for simplicity in
   understanding and implementation a we present it as single header
   format.

4.2.2  Internal Message Header Fields Description

   The first four words are almost identical to the corresponding part
   of the data message header.  The differences are described next.

   User: 4 bits

      For TIPC internal messages this field has a different set of
      values than for data messages.  The following values are used:






Maloy, et al.            Expires April 27, 2005                [Page 72]


Internet-Draft                    TIPC                      October 2004


      ID Value    User
      --------    ----------
      5           Broadcast Maintenance Protocol  (BCAST_PROTOCOL)
      6           Message Bundler Protocol        (MSG_BUNDLER)
      7           Link State Maintenance Protocol (LINK_PROTOCOL)
      8           Connection Manager              (CONN_MANAGER)
      9           Routing Table Update Protocol   (ROUTE_DISTRIBUTOR)
      10          Link Changeover Protocol        (CHANGEOVER_PROTOCOL)
      11          Name Table Update Protocol      (NAME_DISTRIBUTOR)
      12          Message Fragmentation Protocol  (MSG_FRAGMENTER)

   Packet Size:   17 bits.  Used by: All users

      This is physically the same field as 'Message Size' in data
      messages, but now indicates the actual size of the packet it
      envelopes.

   Message Type:   4 bits.  Used by: All users

      The values in this field is defined per user.  See the section
      describing each separate user below.

   Broadcast Sequence Gap: 5 bits.  Used by: LINK_PROTOCOL

      The field 'Error Code','Reroute Count', 'Lookup Scope' and
      'Options Position' fields have no relevance for
      LINK_PROTOCOL/STATE_MSG messages, so these 13 bits can be recycled
      in such messages.  'Broadcast Sequence Gap' informs the recipient
      about the size of a gap detected in the sender's received
      broadcast packet sequence, from 'Broadcast Acknowledge Number' and
      onwards.  The receiver of this information must immediately
      retransmit the missing packets.

   Sequence Gap:  8 bits.  Used by: LINK_PROTOCOL

      The field 'Error Code','Reroute Count', 'Lookup Scope' and
      'Options Position' fields have no relevance for
      LINK_PROTOCOL/STATE_MSG messages, so these 13 bits can be recycled
      in such messages.  'Sequence Gap' informs the recipient about the
      size of a gap detected in the sender's received packet sequence,
      from 'Link Level Acknowledge Number' and onwards.  The receiver of
      this information must immediately retransmit the missing packets.

   Next Sent Broadcast: 16 bits.Used by: LINK_PROTOCOL

      In order to speed up detection of lost broadcasts packets all
      LINK_PROTOCOL/STATE_MSG  messages contain this information from
      the sender node.  If the receiver finds that this is not in



Maloy, et al.            Expires April 27, 2005                [Page 73]


Internet-Draft                    TIPC                      October 2004


      accordance with what he has received, he immediately sends a
      BCAST_PROTOCOL/STATE_MSG back to the sender, with Sequence Gap set
      appropriately.

   Fragment Number: 16 bits.Used by: MSG_FRAGMENTER

      Occupying the same space as 'Next Sent Broadcast' this value
      indicates the number of a message fragment within a fragmented
      message, starting from 1.

   Next Sent Packet:  16 bits.  Used by: LINK_PROTOCOL

      Link protocol messages bypass all other packets in order to
      maintain link integrity, and hence can not have sequence numbers
      valid for the ordinary packet stream.  But all receivers are
      dependent of this information to detect packet losses, and cannot
      completely rely on the assumption that a sequenced packet will
      arrive within acceptable time.  To guarantee a worst case packet
      loss detection time, even on low-traffic links,the equivalent
      information to a valid sequence number has to be conveyed by the
      link continuity check (STATE_MSG) messages, and that is the
      purpose of this field.

   Fragment Number: 16 bits.Used by: MSG_FRAGMENTER

      Occupying the same space as 'Next Sent Packet', this value
      identifies a a fragmented message on the particular link where it
      is sent.

   Session Number: 16 bits.  Used by: LINK_PROTOCOL

      The risk of packets being reordered by the router is particularly
      elevated at the moment of first contact between nodes, so a check
      of sequentiality is needed even for LINK_PROTOCOL/RESET_MSG
      messages.  The session number starts from a random value, and is
      incremented each time a link comes up.  This way, redundant
      RESET_MSG messages, delayed by the router and arriving after the
      link has been brought to a working state,can be identified and
      ignored.

   Reserved: 3 bits

      Must be set to zero.

   Redundant Link: 1 bit

      When set, this bit informs the other endpoint that the sender
      thinks it has a second working link towards the destination.  This



Maloy, et al.            Expires April 27, 2005                [Page 74]


Internet-Draft                    TIPC                      October 2004


      information is needed by the recipient on order know whether he
      should initiate a changeover procedure or not in case of link
      failure.  Under certain, extremely transient and rare,
      circumstances, it is not sufficient for an endpoint to know its
      own link view to perform a correct changeover.

   Bearer Identity: 3 bits

      When a bearer is registered with the link layer of TIPC in a node,
      it is assigned a unique ideitifying number in the range [0,7].
      This number will not necessarily be the same in different nodes,
      so a link endpoint to needs to know the other endpoint's assigned
      identity for the same bearer.  This is needed during the link
      changeover procedure, in order to identify the destination bearer
      instance of a tunneled packet.

   Link Priority: 5 bits.  Used by: LINK_PROTOCOL

      When there are more than one link between two nodes, one may want
      to use them in load sharing or active/standby mode.  Equal
      priority between links means load sharing, different priorities
      means that the link with the higher numerical value will take all
      traffic.  By offering a value range of 32 one can build in a
      default relation between different bearer types,(e.g.  DCCP is
      given lower priority than Ethernet), and no manual configuration
      of these values should normally be needed.

   Network Plane: 3 bits.  Used by: LINK_PROTOCOL

      When multiple parallel routers and multiple network interfaces are
      used it is useful, although not strictly needed by the protocol,
      to have a network pervasive identifier telling which interfaces
      are connected to which routers.  This relieves system managers
      from the burden of manually keeping track of the actual physical
      connectivity.  Typically, the identifier 0 would be presented to
      the operator as 'Network A', identity 1 as 'Network B' etc.  This
      identity must be agreed upon in the whole network, and therefore
      this field is present and valid in the header of all LINK_PROTOCOL
      messages.  The 'negotiation' consists of letting the node with the
      lowest numeral value of its network address, typically node
      <1.1.1>, decide the identities.  All others must strictly update
      their identities to the value received from any lower node.

   Probe: 1 bit.  Used by: LINK_PROTOCOL

      This one-bit field is used only by messages of type LINK_PROTOCOL/
      STATE_MSG.  When set it instructs the receiving link endpoint to
      immediately respond with a STATE_MSG.  The Probe bit MUST NOT be



Maloy, et al.            Expires April 27, 2005                [Page 75]


Internet-Draft                    TIPC                      October 2004


      set in the responding message.

   Originating Node: 32 bits.  Used by: NAME_DISTRIBUTOR

      The node from which the message originally was sent.

   Destination Node: 32 bits.  Used by: NAME_DISTRIBUTOR

      The final destination node for a message.

   Transport Level Sequence Number: 32 bits.  Used by: NAME_DISTRIBUTOR

      The sequence number of the NAME_DISTRIBUTOR message.  Only used
      and valid when the message needs routing.

   Message Count: 16 bits.  Used by: MSG_BUNDLER, CHANGEOVER_PROTOCOL

      This field is used for two different purposes.  First, the message
      bundling function uses it to indicate how many packets are bundled
      in a bundle packet.  Second, when a link goes down, the endpoint
      detecting the failure must send an ORIG_MSG to the other endpoint
      (tunneled through the remaing link) informing it about how many
      tunneled packets to expect.  This gives the other endpoint a
      chance to know when the changeover is finished, so it can return
      to the normal link setup procedure.

   Link Tolerance:  16 bits.  Used by: LINK_PROTOCOL

      Each link endpoint must have a limit for how long it can wait for
      packets from the other end before it declares the link failed.
      Initially this time may differ between the two endpoints, and must
      be negotiated.  At link setup all RESET_MSG messages in both
      directions carry the sender's configured value in this field, and
      the highest numerical value will be the one chosen by both
      endpoints.  In STATE_MSG messages this field is normally zero, but
      if the value is explicitly changed at one endpoint, e.g.  by a
      configuration command, it will be carried by the next STATE_MSG
      and force the other endpoint to also change its value.  Subsequent
      STATE_MSG messages return to the zero value.  The unit of the
      value is [ms].

4.3  Message Users

4.3.1  Broadcast Protocol

   User: 5 (BCAST_PROTOCOL).

   Message Types:



Maloy, et al.            Expires April 27, 2005                [Page 76]


Internet-Draft                    TIPC                      October 2004


      ID Value    Meaning
      --------    ----------
      0           Tunneled, retransmitted broadcast packet (BCAST_MSG)

   A BCAST_MSG message wraps a packet that was originally sent as
   broadcast, but which has retransmitted.  Depending on the number of
   missing acknowledges (see Section 3.9.4), the retransmission may be
   performed as a new broadcast, or as a limited sequence of BCAST_MSG
   unicasts to the nodes missing in the acknowledge list.

4.3.2  Message Bundler Protocol

   User: 6 (MSG_BUNDLER)

   Message Types: None

   A MSG_BUNDLER packet contains as many bundled packets as indicated in
   Message Count.  All bundled messages start at a 4-byte aligned
   position in the packet.  Each bundled packet is a complete packet,
   including header, but with the fields Broadcast Acknowledge Number,
   Link Level Sequence Number and Link Level Acknowledge Number left
   undefined.  Any kind of packets, except LINK_PROTOCOL and MSG_BUNDLER
   packets, may be bundled.

4.3.3  Link State Maintenance Protocol

   User: 7 (LINK_PROTOCOL)

      ID Value   Meaning
      --------   ----------
      0          Detailed state of a working link        (STATE_MSG)
                 endpoint
      1          Reset receiving endpoint, sender is     (RESET_MSG)
                 RESET_UNKNOWN
      2          Sender in RESET_RESET,ready to receive  (ACTIVATE_MSG)
                 traffic

   RESET_MSG messages must have a data part that must be a
   zero-terminated string.  This string is the name of the bearer
   instance used by the sender node for this link.  Examples of such
   names is "eth0","vmnet1" or "udp".  Those messages must also contain
   valid values in the fields Session Number, Link Priority and Link
   Tolerance.

   ACTIVATE_MSG messages do not need to contain any valid fields except
   Message User and Message Type.

   STATE_MSG messages may leave bearer name and Session Number



Maloy, et al.            Expires April 27, 2005                [Page 77]


Internet-Draft                    TIPC                      October 2004


   undefined, but Link Priority and Link Tolerance must be set to zero
   in the normal case.  If any of these values are non-zero, it implies
   an order to the receiver to change its local value to the one in the
   message.  This must be done when a management command has changed the
   corresponding value at one link endpoint, in order to enforce the
   same change at the other endpoint.  Network Identity must be valid in
   all messages.

   Link protocol messages must always be sent immediately, disregarding
   any traffic messages queued in the link.  Hence, they can not follow
   the ordinary packet sequence, and their sequence number must be
   ignored at the receiving endpoint.  To facilitate this, these
   messages should be given a sequence number guaranteed not to fit in
   sequence.  The recommended way to do this is to give such messages
   the next unassigned Link Level Sequence Number + 362768.  This way,
   at the reception the test for the user LINK_PROTOCOL needs to be
   performed only once, after the sequentiality check has failed, and we
   never need to reverse the Next Received Link Level Sequence Number.

4.3.4  Connection Manager

   Although a TIPC internal user, Connection Manager is special, because
   it uses the 36-byte header format of CONN_MSG payload messages
   instead of the 40-byte internal format.  This is because those
   messages must contain a destination port and a originating port.

   The following message types are valid for Connection Manager:

   User: 8 (CONN_MANAGER).

   Message Types:

      ID Value   Meaning
      --------   ----------
      0          Probe to test existence of peer      (CONN_PROBE)
      1          Reply to probe, confirming existence (CONN_PROBE_REPLY)
      2          Acknowledge N Messages               (MSG_ACK)

   MSG_ACK messages are used for transport-level congestion control, and
   carry one network byte order 32-byte integer as data.  This indicates
   the number of messages acknowledged, i.e.  actually read by the port
   sending the  acknowledge.  This information makes it possible for the
   other port to keep track of the number of sent, but not yet received
   and handled messages, and to take action if this value surpasses a
   certain threshold.

   The details about why and when these messages are sent are described
   in Section 3.5.4.



Maloy, et al.            Expires April 27, 2005                [Page 78]


Internet-Draft                    TIPC                      October 2004


4.3.5  Routing Table Update Protocol

   User: 9 (ROUTE_DISTRIBUTOR).

   Message Types:

      ID Value  Meaning
      --------  ----------
      0         Sender's routes to cluster    (EXT_ROUTING_TABLE)
                external nodes
      1         Sender's routes to cluster    (LOCAL_ROUTING_TABLE)
                internal nodes
      2         Sender's routes to secondary  (SEC_ROUTING_TABLE)
                nodes
      3         New route to a node           (ROUTE_ADDITION)
      4         Lost contact with a node      (ROUTE_REMOVAL)

   EXT_ROUTING_TABLE messages have the following structure:



       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     TIPC Internal Header                      /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w10|                       Cluster Address                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               |
      /                            bitmap             +-+-+-+-+-+-+-+-+
      \                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


            Figure 36: External Routing Table message format

   The first four bytes of the message payload is a TIPC Network Address
   in network byte order, indicating the remote cluster concerned by the
   table.  The rest of the message is a bitmap, indicating to which
   nodes within that cluster the sending node has direct links.  E.g.
   if node <1.1.7> has a direct link to the nodes <2.3.4> and <2.3.5>,
   Cluster Address will contain <2.3.0>, and bit 4 and 5 of the
   remaining data is set to a non-zero value.  The message need not be
   longer than to contain the last non-zero bit in the map.  Along with
   the Originating Node field this message contains all information
   needed for any node in the cluster to know how to reach the two



Maloy, et al.            Expires April 27, 2005                [Page 79]


Internet-Draft                    TIPC                      October 2004


   nodes.

   LOCAL_ROUTING_TABLE and SEC_ROTING_TABLE messages have the same
   structure as EXT_ROUTING_TABLE, but Cluster Address may be left
   undefined, since the receiving nodes already know to which cluster
   they belong.

   ROUTE_ADDITION and ROUTE_REMOVAL messages have the following format:



       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     TIPC Internal Header                      /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w10|                          Node Address                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


            Figure 37: Route Addition/Removal message format

   Here the only information needed is a Network Address indicating
   which node the route addition/loss concern, i.e.  to which node the
   sender node has established/lost direct contact.

4.3.6  Link Changeover Protocol

   User: 10 (CHANGEOVER_PROTOCOL)

      ID Value    Meaning
      --------    ----------
      0           Tunneled duplicate of packet       (DUPLICATE_MSG)
      1           Tunneled original of packet        (ORIGINAL_MSG)

   DUPLICATE_MSG messages contain no extra information in the header
   apart from the first thee words.  The first ORIGINAL_MSG message sent
   out MUST contain a valid value in the Message Count field, in order
   to inform the recipient about how many such messages, inclusive the
   first one, to expect.  If this field is zero in the first message, it
   means that there are no packets wrapped in that message, and none to
   expect.

4.3.7  Name Table Update Protocol

   User: 11 (NAME_DISTRIBUTOR)



Maloy, et al.            Expires April 27, 2005                [Page 80]


Internet-Draft                    TIPC                      October 2004


      ID Value   Meaning
      --------   ----------
      0          One or more port name publications  (PUBLICATION)
      1          Withdraw port name publication      (WITHDRAWAL)

   These messages have the following structure:



       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     TIPC Internal Header                      /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w10|                           type                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w11|                           lower                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w12|                           upper                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w13|                        port reference                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w14|                            key                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                     More publications                         /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


              Figure 40: Name Table Update message format

   PUBLICATION messages may contain one or more Publications.  A
   Publication consists of the five-word field listed in the figure.
   Each field is stored in network byte order, and have the following
   meaning.
   o  Type:  The 'type' part of a published/withdrawn port name
      sequence.
   o  Lower: The 'lower' part of a published/withdrawn port name
      sequence.
   o  Upper:  The 'upper' part of a published/withdrawn port name
      sequence.
   o  Port Reference: The random number part of the publishing port's
      identity.
   o  Key: A key created by the publishing port.  Must be presented to
      the receiver in the corresponding WITHDRAW message.



Maloy, et al.            Expires April 27, 2005                [Page 81]


Internet-Draft                    TIPC                      October 2004


   The number of publications in the message is calculated by the
   receiver based on the message length.  The Node Identity part of each
   publication is the same as the messages Originating Node.

   WITHDRAWAL messages may only contain one publication item.

4.3.8  Message Fragmentation Protocol

   User: 12 (MSG_FRAGMENTER)

      ID Value    Meaning
      --------    ----------
      0           First fragment of message (FIRST_FRAGMENT)
      1           Body fragment of message  (FRAGMENT)
      2           Last fragment of message  (LAST_FRAGMENT)

   All packets contain a dedicated identifier, Fragmented Message
   Number, to distinguish them from packets belonging to other messages
   from the same node.  All packets also contain a sequence number
   within its respective message, the Fragment Number field, in order
   to, if necessary, reorder the packets when they arrive to the
   detination node.  Both these sequence numbers must be incemented
   modulo 2^16-1.

4.3.9  Neighbour Detection Protocol

   User: 13 (LINK_CONFIGURATION) The protocol for neighbour detection
   uses a special message format, with the following generic structure:























Maloy, et al.            Expires April 27, 2005                [Page 82]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0:|vers |msg usr|hdr sz |n|resrv|            packet size          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1:|m typ|0| requested links       |       broadcast ack no        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w2:|                      destination domain                       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w3:|                       previous node                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4:|                      network identity                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w5:|                                                               |
      +-+-+-+-+-+-+-                                    +-+-+-+-+-+-+-+
   w6:|                                                               |
      +-+-+-+-+-+-+-  bearer level originating address  +-+-+-+-+-+-+-+
   w7:|                                                               |
      +-+-+-+-+-+-+-                                    +-+-+-+-+-+-+-+
   w8:|                                                               |
      +-+-+-+-+-+-+-                                    +-+-+-+-+-+-+-+
   w9:|                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                 vendor specific data  (optional)              /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


              Figure 42: Link Configuration message format

   o  Header Size: 40 bytes.
   o  Non-sequenced: This bit is set to 1 for this user.
   o  Packet Size: Header Size plus size of Optional Data.
   o  Message Type: 0 if the message is a link request (e.g.  a
      broadcast neighbour detection), 1 if it is a response to such a
      request.
   o  Requested Links (12 bits): The number of links the sender node
      wants to establish to the destination domain, over the specific
      bearer used.  For a link request to a specific node this number
      must be 1, as only one link is permitted per bearer and node pair.
      Recommended default value for this field is 2.
   o  Originating Node: The network address of the originating node.
   o  Destination Domain: The domain to which the link request is
      directed.  <Z.C.N> means that the sender requests a link to that
      specific node, and nothing else.  <Z.C.0> or <Z.0.0> means that
      the sender wants the specified number of links to that cluster or
      zone, but not necessarily to the first node where the message



Maloy, et al.            Expires April 27, 2005                [Page 83]


Internet-Draft                    TIPC                      October 2004


      arrives.  <0.0.0> means that the sender does not know anything
      about the receiver's identity, but wants links to anybody within
      the cluster receiving the message.
   o  Network Identity: The sender node's network identity.  Receivers
      having a network identity different from the one in the message
      must ignore the message.
   o  Bearer Level Originating Address: A Bearer Address containing the
      Ethernet or IP(v4 or v6)-address+port number of the sender.  Note
      that with IP-protocols this is only a hint, as the IP-address may
      have been replaced by NAT.  In such cases, it is the reponsibility
      of the corresponding adaptation layer to extract the correct
      sender address from the incoming message.
   o  Vendor Specific Data: The contents of this optional field is
      vendor specific.

   Link Probe Message

      The messages used for finding the optimal location for responding
      to a Link Request are called Link Probes:
































Maloy, et al.            Expires April 27, 2005                [Page 84]


Internet-Draft                    TIPC                      October 2004


        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0: |                   Magic Identifier (0x54495043)               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1: |                   Requesting Node TIPC Address                |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w3: |                                                               |
       |                       Requesting Node                         |
       |                     Bearer Level Address                      |
       |                                                               |
       |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8: |                        Network Plane                          |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w9: |                        Hop Count                              |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w10:|                       Requested Links                         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w11:|                       Found Links                             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w12:|                      Lowest Link Count                        |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w13:|                    Lowest Link Count This Tour                |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w14:|                         Entry Node                            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                  Figure 43: Link Probe message format


      *  Requesting Node: The node in the remote cluster which
         originally sent out the request.
      *  Requesting Node Bearer Level Address: The bearer address (e.g.
         IP or Ethernet address) of the node in the remote cluster which
         originally sent out the request.
      *  Network Plane: The identifier of the network where the Link
         Request originally was received.  E.g.  0 for Network A, 1 for
         Network B.
      *  Hop Count: The number of node hops this probe has performed.
         If this number exceeds 10 * size of cluster, the probe must be
         discarded.
      *  Requested Links: The number of links the requesting node wants
         to this cluster.
      *  Found Links: The number of found, established links to the
         originating node so far.  When this number equals 'Requested
         Links', the establishing procedure is finished,and the probe



Maloy, et al.            Expires April 27, 2005                [Page 85]


Internet-Draft                    TIPC                      October 2004


         can be discarded.
      *  Lowest Link Count: The lowest number of links to the requesting
         cluster found in the cluster during the previous tour.  If, at
         the next tour, a node is found with this number of links, a
         link setup attempt is initiated.  For each fulfilled tour, this
         field is updated with the contents of Lowest Link Count This
         Tour.
      *  Lowest Link Count This Tour: The lowest number of links to the
         requesting cluster found in the cluster during the current
         tour.  This is useful if the probe fulfils a tour without
         finding the Lowest Link Count number of links, which may happen
         when the number of links is changing very rapidly.
      *  Entry Node: The node that received the original link request.
         This helps identifying when the probe has fulfilled a tour.
         Hop Count can not be trusted alone for this, since the number
         of nodes may change during the tour.
      Link Probe Messages are sent to port name <1,302> using the
      identified next hop node as lookup domain.

   Get Node Info Message


        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0: |                 Magic Identifier (0x54495043)                 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1: |                   Remote Node TIPC Address                    |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure 44: Get Node Info message format

      Remote Node Address: The cluster remote node for which the
      information is requested.
      Get Node Info Messages are sent to port name <1,300> using the
      identified router node as lookup domain.

   Get Node Info Result Message













Maloy, et al.            Expires April 27, 2005                [Page 86]


Internet-Draft                    TIPC                      October 2004


        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0: |                 Magic Identifier (0x54495043)                 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1: |                   Remote Node TIPC Address                    |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w2: |                                                               |
       |                       Remote Node                             |
       |                   Bearer Level Address                        |
       |                                                               |
       |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7: \                                                               \
       /                        Bearer Name                            /
       \                                                               \
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

             Figure 45: Get Node Info Result message format


      *  Remote Node TIPC Address: The cluster remote node for which the
         information is valid.
      *  Remote Node Bearer Level Address: The bearer address (e.g.  IP
         or Ethernet address) of the node in the remote cluster.
      *  Bearer Name: The string identifying the bearer where the Bearer
         Level Address is valid, i.e.  the bearer to be used for the
         Link Requests to be sent.
      Get Node Info Result Messages are sent to port name <1,301> using
      the node that sent the corresponding Get Node Info message as
      lookup domain.

   Link Request Accepted Message

      This message only has the four-byte Magic Identifier as data.  It
      is sent to port name <1,304>, using the node that sent the
      corresponding Link Request message as lookup domain.

   Link Request Rejected Message

      This message only has the four-byte Magic Identifier as data.  It
      is sent to port name <1,303>, using the node that sent the
      corresponding Link Request message as lookup domain.

   Drop Link Request Message

      This message only has the four-byte Magic Identifier as data.  It
      is sent to port name <1,305>, using the node that sent the



Maloy, et al.            Expires April 27, 2005                [Page 87]


Internet-Draft                    TIPC                      October 2004


      corresponding Link Request message as lookup domain.

   Check Link Count Message

      This message only has the four-byte Magic Identifier as data.  It
      is sent to port name <1,306>, using the node node to check as
      lookup domain.

4.4  Media Adapter Protocols

   The protocol for adapting to various underlying media is described in
   the following sections.  For the time being only one such mapping is
   publicly available, the one for Ethernet

4.4.1  Ethernet Adaptation

   Ethernet Number, formally assigned from IEEE: 0x88ca

   Ethernet Adaptation Protocol Header

      No header is added at this level.






























Maloy, et al.            Expires April 27, 2005                [Page 88]


Internet-Draft                    TIPC                      October 2004


5.  Management

   The management interface towards TIPC is a message interface.  A TIPC
   node is managed by sending a correctly formatted message to that
   node.  using a port name destination address with Type set to 0, and
   Instance set to the network address of the target node.

5.1  Command Types

   There are three groups of management commands:
   o  Group 1: Read-only commands, or other non-intrusive commands.
      E.g.  commands reading statistics, table contents etc.  or
      commands resetting statitics.  These commands may be called from
      anywhere, by sending connectionless messages.
   o  Group 2: Intrusive commands.  Commands changing settings in TIPC
      must be handled very strictly.  Before issuing such commands, a
      manager must establish an exclusive connection towards TIPC on the
      concerned node.  Exclusive means than no more than one such link
      may exist at any time towards a node.  Typically, a manager
      process will establish such a management link to all nodes in the
      cluster or zone at system start, and then hold on to that link as
      long as he is up.
   o  Group 3: Remote Subscriptions.  Port name sequence subcriptions
      can be ordered not only from local users via the ordinary API, but
      also remotely, by issuing a correctly formatted management message
      towards a specific node.  This way, it is possible to e.g.
      subscribe for a certain port name in a remote zone, even if the
      subscriber is located outside the publishing scope of the
      requested name sequence.  Since such remote subscriptions are
      potentially intrusive, there can only exist a limited number,
      which should be configurable, at any time.  Just like with write
      commands, they require that a connection is set up towards the
      target node.  One such connection is established per-subscription.

5.2  Command Message Formats

   All fields described as integers in the following sections are
   transferred in network byte order.

5.2.1  Command Messages

   The first 16 bytes of all command messages have the same structure:









Maloy, et al.            Expires April 27, 2005                [Page 89]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0:|                       Magic (0x54495043)                      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1:|                         Command                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w2:|                                                               |
      +                       User Handle                             +
   w3:|                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


                Figure 46: Command Message header format

   Magic:  32 bit integer

      This is an identifier with the value 0x54495043 ('T','I','P','C'),
      meant to protect against accidental access by corrupted or wrongly
      addressed command messages.

   Command: 32 bit integer

      This is the command issued by this message.

   User Handle: 64 bits

      A handle at the user's disposal, for storing e.g.  a pointer.

5.2.2  Command Response Messages

   The first 24 bytes of all command response messages have the same
   structure:


















Maloy, et al.            Expires April 27, 2005                [Page 90]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0:|                         Command                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w1:|                                                               |
      +                       User Handle                             +
   w2:|                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w3:|                      Return Value                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4:|                         Remains                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w5:|                        Result lenght                          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


           Figure 47: Command Response Message header format

   Command: 32 bit integer

      The original command issued in the corresponding command message.

   User Handle: 64 bits

      The original handle passed by the corresponding command message.

   Return Value: 32 bit integer

      If the command was successful, this field is 0, otherwise
      0xffffffff.

   Remains: 32 bit integer

      If the command resulted in more return data than can be stored in
      one message, this field indicates the number of bytes to be
      expected in subsequent messages.

   Result Length: 32 bit integer

      The length of the remainder of the message, i.e.  the command
      specific result.

5.3  Commands

5.3.1  Group 1: Query Commands

   Get Port Statistics            Group 1 command: 211



Maloy, et al.            Expires April 27, 2005                [Page 91]


Internet-Draft                    TIPC                      October 2004


      Get statistics for a certain port.
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4 |                        Port Reference                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

          Figure 48: Get Port Statistics Query message format

      Port Reference contains the random number part of the identity of
      the port from which statistics is wanted.
      Result message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               |
      /              Zero-terminated string           +-+-+-+-+-+-+-+-+
      \                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       Figure 49: Get Port Statistics Query Result message format

   Reset Port Statistics          Group 1 command: 212

      Command message:













Maloy, et al.            Expires April 27, 2005                [Page 92]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4 |                        Port Reference                         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

        Figure 50: Reset Port Statistics Command message format

      Port Reference contains the random number part of the identity of
      the port for which the statistics should be reset.
      Result message:
         The result of this command is a Common Command Result Header,
         indicating success or failure.

   Get Nodes                    Group 1 command: 201

      Get information about all nodes within a given domain known by the
      target node.
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4 |                           Domain                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 51: Get Nodes Query Command message format

      Domain is a Network Address in the form <0.0.0>, <Z.0.0> etc.
      Result message:












Maloy, et al.            Expires April 27, 2005                [Page 93]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                           Up                                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                         Node Address                          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                          More Nodes                           /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 52: Get Nodes Query Result message format

      Result data is a sequence of structures, each consisting of two
      integers.
      *  Up: Integer indicating whether the indicated node is reachable
         or not from the target  node.  Non-zero means it is reachable.
      *  Node Address: The address of the known node.

   Get Links                    Group 1 command: 182

      Get information about all links from the target node to other
      nodes within the given domain.
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4 |                           Domain                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 53: Get Links Query Command message format

      Domain is a Network Address in the form <0.0.0>, <Z.0.0> etc.
      Result message:






Maloy, et al.            Expires April 27, 2005                [Page 94]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                             Up                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                         Node Address                          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8 \                                                               \
      /                          Link Name                            /
   w24\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w25\                                                               \
      /                          More Links                           /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 54: Get Links Query Result message format

      Result data is a sequence of structures, each consisting of three
      elements.
      *  Up: Integer indicating whether the indicated link is working or
         not.
      *  Node Address: The node at the other end of the link.
      *  Link Name: A 72 byte field, contianing a zero-terminated
         string, uniquely identifying the link.

   Get Routes                         Group 1 command: 230

      Get all routes to a given domain known by the target node.
      Command message:

















Maloy, et al.            Expires April 27, 2005                [Page 95]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w4 |                           Domain                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 55: Get Routes Query Command message format

      Domain is a Network Address in the form <0.0.0>, <Z.0.0> etc.
      Result message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                       Destination Node Address                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                       Router Node Address                     |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                          More Routes                          /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 56: Get Routes Query Result message format

      Result data is a sequence of structures, each consisting of two
      elements:
      *  Node Address: The network address of the node to which the
         route leads.
      *  Router Address: The network address of the cluster local node
         through which the indicated detination node can be reached.

   Get Links Statistics                 Group 1 command: 183

      Get statistics from the link indicated by Link Name.
      Command message:






Maloy, et al.            Expires April 27, 2005                [Page 96]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               \
      /                    Zero-terminated Link Name                  /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 57: Get Links Statistics Query Command message format

      Link Name is a 72-byte field, containing the name of the requested
      link.
      Result message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               |
      /              Zero-terminated string           +-+-+-+-+-+-+-+-+
      \                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 58: Get Links Statistics Query Result message format

   Reset Links Statistics                 Group 1 command: 184

      Reset statistics for the link indicated by Link Name.
      Command message:
         Same as for Get Link Statistics.
      Result message:
         The result of this command is a Common Command Result Header,
         indicating success or failure.

   Get Peer Address                 Group 1 command: 193

      Get the bearer level address.  e.g.  ethernet or
      IP-address:portno, for the other endpoint of the indicated link.
      Command message:




Maloy, et al.            Expires April 27, 2005                [Page 97]


Internet-Draft                    TIPC                      October 2004


         Same as for Get Link Statistics.
      Result message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                      Address Type                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 \                                                               \
      /                     Bearer Level Address                      /
      \                                                               \
   w11/                                                               /
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

        Figure 59: Get Peer Address Query Result  message format

      The integer Address Type has the following defined values for now:

      Value     Type
      -----     ----
      0         6-byte ethernet address
      1         Socket address IPv4.
      2         Socket address IPv6.

      An Ipv4 socket address has the following structure:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |           Undefined           |         Port Number           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8 |                          IP Address                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 61: Socket Address format for peer address

      Port Number and IP address are integers transferred in network
      byte order.
      An Ipv6 socket address has the following structure:







Maloy, et al.            Expires April 27, 2005                [Page 98]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |           Undefined           |         Port Number           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8 |                                                               |
      |                         IPv6 Address                          |
      |                                                               |
   w11|                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 62: Socket Address format for peer address

      Port Number and IP address are integers transferred in network
      byte order.

   Get Name Table                 Group 1 command: 220

      Get selected contents of the target node's naming table:
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                         Name Type                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                          Depth                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


         Figure 63: Get Name Table Query Command message format

      Name Type is an integer indicating which name table entry is to be
      investigated.  If this field is zero, all types in the table will
      be investigated and returned.
      Depth is an integer indicating how much information is wanted
      about the requested entry or entries.  It may have the following
      values:

      Value     Description
      -----     -----------
      0         All information about the entry, i.e. all known publications.
      1         All known sequences for the requested name type.



Maloy, et al.            Expires April 27, 2005                [Page 99]


Internet-Draft                    TIPC                      October 2004


      2         Only the Name Type of the entry.

      If both Name Type and Depth are 0 the whole table contents is
      returned.  A depth value of 2 only makes sense with The Name Type
      value 0.
      Result message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
     \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               |
      /              Zero-terminated string           +-+-+-+-+-+-+-+-+
      \                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         Figure 65: Get Name Table Query Result message format

   Get Bearers                    Group 1 command: 190

      Get information about all bearer instances found on the target
      node.
      Command message:
         Only a common command message header, with no arguments.
      Result message:






















Maloy, et al.            Expires April 27, 2005               [Page 100]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               \
      /                          Bearer Name                          /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w18\                                                               \
      /                          More Bearer Names                    /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


          Figure 66: Get Bearers Query Result  message format

      Result data is a sequence of 48-byte sized structures, each
      consisting of a zero-terminated string, uniquely identifying each
      bearer.

   Get Media                    Group 1 command: 194

      Get information about all bearer types registered on the target
      node.
      Command message:
         Only a common command message header, with no arguments.
      Result message:





















Maloy, et al.            Expires April 27, 2005               [Page 101]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               \
      /                            Media Name                         /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w18\                                                               \
      /                          More Media Names                     /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 67: Get Media Query Result message format

      Result data is a sequence of 48-byte sized structures, each
      consisting of a zero-terminated string, uniquely identifying each
      bearer.

   Get Ports                    Group 1 command: 210

      Get reference (random number part of identity) of all existing
      ports on the node.
      Command message:
         Only a common command message header, with no arguments.
      Result message:






















Maloy, et al.            Expires April 27, 2005               [Page 102]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                         Port Reference                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      \                                                               \
      /                       More Port References                    /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 68: Get Ports Query Result message format


5.3.2  Group 2: Manipulating Commands

   Establish Management Connection                Group 1 command: 111

      Command message:
         Only a common command message header, with no arguments.
      Result message:
         A common command result message header, indicating success or
         failure.

   Create Link                   Group 2 command: 180

      Establish a link to the indicated domain.
      Command message:




















Maloy, et al.            Expires April 27, 2005               [Page 103]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                            Domain                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 \                                                               \
      /                     Bearer Level Address                      /
      \                                                               \
      /                                                               /
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w12\                                                               \
      /                        Bearer Name                            /
   w22\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

             Figure 69: Create Link Command message format

      Domain is a network address in one of the formats <0.0.0>, <Z.C.0>
      etc.  Bearer Level Address is the address to be used for the
      establishment.  Bearer Name indicates the bearer instance through
      which the establishment should be attempted.
      Result message:
         A common command result message header, indicating success or
         failure.

   Remove Link                        Group 2 command: 181

      Remove the link indicated by Link Name.
      Command message:
         Same as for Get Link Statistics.
      Result message:
         A Common Command Result Header, indicating success or failure.

   Block Link                        Group 2 command: 185

      Set the link indicated by Link Name in BLOCKED state.
      Command message:
         Same as for Get Link Statistics.
      Result message:
         A Common Command Result Header, indicating success or failure.

   Unlock Link                        Group 2 command: 186





Maloy, et al.            Expires April 27, 2005               [Page 104]


Internet-Draft                    TIPC                      October 2004


      Set the  blocked link indicated by Link Name in RESET_UNKNOWN
      state.
      Command message:
         Same as for Get Link Statistics.
      Result message:
         A Common Command Result Header, indicating success or failure.

   Set Link Tolerance                       Group 2 command: 187

      Set the link tolerance of the link indicated by Link Name.
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                          Value                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 \                                                               \
      /                          Link Name                            /
   w24\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

          Figure 70: Set Link Tolerance Command message format

      Result message:
         A Common Command Result Header, indicating success or failure.

   Set Link Priority                       Group 2 command: 188

      Set the link priority of the link indicated by Link Name.
      Command message:
         Same as for Set Link Tolerance.
      Result message:
         A Common Command Result Header, indicating success or failure.

   Set Link Window                       Group 2 command: 189

      Set Send Window Limit for the link indicated by Link Name.
      Command message:
         Same as for Set Link Tolerance.
      Result message:
         A Common Command Result Header, indicating success or failure.




Maloy, et al.            Expires April 27, 2005               [Page 105]


Internet-Draft                    TIPC                      October 2004


   Enable Bearer                        Group 2 command: 191

      Enable the bearer instance indicated by Bearer Name.
      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                        Bearer Priority                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 \                                                               \
      /                        Bearer Name                            /
   w18\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 71: Enable Bearer Command message format


         Bearer Priority is the default priority given to all links
         established over this bearer.
      Result message:
         A common command result message header, indicating success or
         failure.

   Disable Bearer                        Group 2 command: 192

      Disable the bearer instance indicated by Bearer Name.
      Command message:


















Maloy, et al.            Expires April 27, 2005               [Page 106]


Internet-Draft                    TIPC                      October 2004


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 \                                                               \
      /                        Bearer Name                            /
   w17\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            Figure 72: Disable Bearer Command message format

      Result message:
         A common command result message header, indicating success or
         failure.

5.3.3  Group 3: Subscriptions

   Subscribe For Port Name Sequence            Group 3 command: 174

      Command message:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                     Common Command Msg Header                 /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                         Name Type                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                         Lower                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8 |                         Upper                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w9 |                         Timeout                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 73: Subscribe for Port Name Sequence Command message format


         All the four words in the command argument are integers.  Their
         values and interpretation are the same as described in Section
         3.2.5.




Maloy, et al.            Expires April 27, 2005               [Page 107]


Internet-Draft                    TIPC                      October 2004


      Result messages:
      *  A common command result message header, indicating success or
         failure.  If successful, TIPC has established a connection
         towards the port which sent out the original command message.
      *  For each change in the naming table pertaing to the subcribed
         name sequence, a message with the following structure will be
         received:



       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                         Event                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 |                         Found Lower                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w8 |                         Found Upper                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w9 |                         Name Type                             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w10|                         Lower                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w11|                         Upper                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w12|                         Timeout                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


   Figure 74: Subscribe for Port Name Sequence Result message format


         Event may have the following values:

         Hex Value  Description
         ---------  -----------
         0x800      A sequence overlapping with the requested range was
                    published.
         0x1000     No sequences overlapping with the requested range
                    remain.
         0x2000     The subscription exceeded the limit set in Timeout.
                 The connection was shut down.





Maloy, et al.            Expires April 27, 2005               [Page 108]


Internet-Draft                    TIPC                      October 2004


         Found Lower and Found Upper indicate the overlapping part
         between the subscribed values and the actually published
         values.  Name Type, Lower etc.  are the same values as
         originally passed in the command message.

   Subscribe For Link                Group 3 command: 67

      Subscribe for working state of the link indicated in Link Name
      Command message:
         Same as for Get Link Statistics
      Result messages:
      *  A common command result message header, indicating success or
         failure.  If successful, TIPC has established a connection
         towards the port which sent out the original command message.
      *  For each change in status of the concerned link, a message with
         the following structure will be received:


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w0 \                                                               \
      /                  Common Command Result Msg Header             /
      \                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w6 |                         Event                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   w7 \                                                               \
      /                          Link Name                            /
   w24\                                                               \
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

          Figure 76: Subscribe for Link Result message format


         Event may have the following values:

         Hex Value  Description
         ---------  -----------
         0x800      The link went up to WORKING_WORKING state.
         0x1000     The link went down to RESET_UNKNOWN state.
         0x2000     The subscription exceeded the limit set in Timeout.
                    The connection was shut down.

         Link Name is the original link name, sent with the command
         message.





Maloy, et al.            Expires April 27, 2005               [Page 109]


Internet-Draft                    TIPC                      October 2004


6.  Security

   TIPC is a special-purpose transport protocol designed for operation
   within a secure, closed network interconnecting nodes within a
   cluster.  TIPC does not possess any native security features, and
   hence rely on the properites of the selected bearer protocol, e.g.
   IP-Sec, when such features are needed

7  References

   [ForCES]   Doria et al., A., "ForCES Protocol Specification",
              September 2004,
              <http://www.ietf.org/internet-drafts/draft-ietf-forces-pro
              tocol-00.txt>.

   [RFC2026]  Bradner, S., "The Internet Standards Process -- Revision
              3", RFC 2026, BCP 9, October 1996,
              <http://www.rfc-editor.org/rfc/rfc2026.txt>.

   [RFC2104]  Krawczyk, H., Bellare, M. and R. Canetti, "HMAC:
              Keyed-Hashing for Message Authentication", RFC 2104,
              February 1997,
              <http://www.rfc-editor.org/rfc/rfc2104.txt>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", RFC 2119, March 1997.

   [RFC2406]  Kent, S. and R. Atkinson, "IP Encapsulating Security
              Payload (ESP)", RFC 2406, November 1998,
              <http://www.rfc-editor.org/rfc/rfc2406.txt>.

   [RFC2408]  Maughan, D., Schertler, M., Schneider, M. and J. Turner,
              "Internet Security Association and Key Management
              Protocol", RFC 2408, November 1998,
              <http://www.rfc-editor.org/rfc/rfc2408.txt>.

   [RFC2434]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", RFC 2434, BCP 26,
              October 1998, <http://www.rfc-editor.org/rfc/rfc2434.txt>.

   [RFC2460]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
              (IPv6) Specification", RFC 2460, December 1998,
              <http://www.rfc-editor.org/rfc/rfc2460.txt>.

   [RFC2581]  Allman, M., Paxson, V. and W. Stevens, "TCP Congestion
              Control", RFC 2581, April 1999,
              <http://www.rfc-editor.org/rfc/rfc2581.txt>.




Maloy, et al.            Expires April 27, 2005               [Page 110]


Internet-Draft                    TIPC                      October 2004


   [RFC2960]  Stewart, R., Xie, Q., Morneault, K., Sharp, C.,
              Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M.,
              Zhang, L. and V. Paxson, "Stream Control Transmission
              Protocol", RFC 2960, October 2000,
              <http://www.rfc-editor.org/rfc/rfc2960.txt>.

   [RFC768]   Postel, J., "User Datagram Protocol", RFC 768, STD 6,
              August 1980, <http://www.rfc-editor.org/rfc/rfc768.txt>.

   [RFC793]   Postel, J., "Transmission Control Protocol", RFC 793, STD
              7, September 1981,
              <http://www.rfc-editor.org/rfc/rfc793.txt>.

   [TIPC]     Maloy, J., "Telecom Inter Process Communication", January
              2003, <http://tipc.sourceforge.net>.


Authors' Addresses

   Jon Paul Maloy
   Ericsson
   Research Canada
   8400, boul. Decarie
   Ville Mont-Royal, Quebec  H4P 2N2
   Canada

   Phone: +1 514 576-2150
   EMail: jon.maloy@ericsson.com


   Steven Blake
   Modularnet
   Raleigh, NC  27606
   USA

   Phone:
   EMail: steven.blake@modularnet.com


   Maarten Koning
   WindRiver
   15983 Loyalist Campway
   RR2
   Bloomfield, ON  KOK 1G0
   Canada

   Phone: +1 613 399-5669
   EMail: maarten.koning@windriver.com



Maloy, et al.            Expires April 27, 2005               [Page 111]


Internet-Draft                    TIPC                      October 2004


   Jamal Hadi Salim
   Znyx
   195 Staford Road West,
   Suite 104
   Nepean, ON  K2H 9C1
   Canada

   Phone: +1 613 596-1138
   EMail: hadi@znyx.com


   Hormuzd M. Khosravi
   Intel
   2111 NE 25th Avenue,
   Hillsboro, OR  97124
   USA

   Phone: +1 503 264-0334
   EMail: hormuzd.m.khosravi@intel.com
































Maloy, et al.            Expires April 27, 2005               [Page 112]


Internet-Draft                    TIPC                      October 2004


Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.


Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Copyright Statement

   Copyright (C) The Internet Society (2004).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.


Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.




Maloy, et al.            Expires April 27, 2005               [Page 113]