INTERNET DRAFT EXPIRES MARCH 1998 INTERNET DRAFT Rex Buddenberg Editor Automated Digital Network System Description <draft-rfced-info-buddenberg-00.txt> Status of This Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet- Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Contributing Authors The following are the authors of the description. This work compiled and digested internal notes, in-house training materials and several conversations with the NRaD engineers. The material of this memo was used as a chapter in each of these students' theses: Brian Rehard, Lt, US Navy Jim Sullivan, Lt, US Navy Eric Andalis, Lt, US Navy Mark Witzel, Maj, US Marine Corps Editors' Note This memo is a first attempt at publicly documenting internal Navy laboratory work aimed at encapsulating a variety of connectivity options inside a radio-WAN shell that fits into the internet structure. I. Introduction A. What is ADNS? B. What is ADNS good for? 1. Mobile Platforms 2. Alternative to Wire/Fiber Transmission C. Why invest in ADNS? 1. Quality of Service 2. Cost Effective Bandwidth 3. Leverages the existing Internet 4. Flexibility D. How does ADNS work? E. ADNS Advantages 1. Removing Humans From the Loop 2. Load Sharing 3. Optimal Use of Bandwidth 4. Communications Agility 5. Transparency of Installation and Use 6. Ease of Upgrade 7. Single Point for Communications Management 8. Ability to Transmit All Types of Data F. ADNS Disadvantages 1. Cost of Installation II. ADNS Operational Description A. Overview B. Network Features 1. Routing Protocols a. OSPF b. BGP4 2. Logical Organization C. Key Features/Functions 1. Priority a. Priority Tables b. Determining Message Priority c. Message Transmission 2. Load Balancing 3. Congestion Control a. Load Sharing 1) Restrictions 2) Implementation b. Source Quench 4. TCP Duplicate Packet Transmission Problems a. TCP Duplicate Packet Rejection D. Integrated Network Management 1. Overview a. Local Control Center 1) Network Manager 2) Distributed Manager 3) Communication Automation Manager i) Communication Manager ii) Site Manager iii) Equipment Manager b. Autonomous System Control Center c. Network Operations Center 2. Network Management Tools a. Network Management System Software b. Third Party Applications III. Hardware A. LAN B. Router C. CRIU D. CAP E. Cryptographic Device F. Modem G. Connectivity Media IV. Acknowledgements and addresses I. Introduction A. What is ADNS? The Navy's Automated Digital Network System (ADNS) provides a means for ship's to centralize and automate the operation of multiple independent radio communications systems into an efficient communications network. ADNS provides connectivity for transmitting bits (which may represent voice, video or data) creating a seamless ship to ship and ship to shore communications network. By managing all of the radio assets within one system, ADNS creates a reliable multiple path communications network. This network is essentially a radio-based Wide Area Network (Radio-WAN). What constitutes the internals of the Radio-WAN are those radio systems configured to support ADNS. Media Dependent Layer |---------------------------------------------------------| | Media Independent Layer | | |-------------------------| | | | | | ---| | ________ | (MilSat) | ________ | |--- ---| | | | | (Commercial) | | | | |--- ---|----| Router |----| (Satellites) |----| Router |----|--- ---| | |________| | | |________| | |--- ---| | | (HF) | | |--- | |-------------------------| | LAN | Radio-WAN | LAN |---------------------------------------------------------| Although currently a Navy specific installation, ADNS is like any other LAN/WAN Internet connection utilizing commercial products. Applications need only adhere to the established Internet protocols that ADNS has adopted. This allows a sense of transparency of applications to ADNS. It is also an open-ended system that allows for future expansion. ADNS allows a plug and play like addition of radio links in a process completely transparent to the user. B. What is ADNS good for? A group of platforms, linked by ADNS, create a radio-based packet-switched WAN. By using existing Internet technology and open standards users of ADNS have seamless transparent access to the Internet. Using a load balancing concept ADNS spreads traffic equally across the appropriate radio links such that the available capacity is the sum of all the links. ADNS does not provide additional bandwidth instead it multiplexes the bandwidth that is already available. There has recently been an insatiable demand for Internet access in areas never previously deemed necessary and although Internet technologies are relatively new, limitations are being experienced on traditional wire/fiber transmission paths. The primary purpose of wireless data transfer is for communications with mobile platforms. This capability already exists in various forms. However, ADNS provides a robust means of choosing the most efficient set of paths to transfer data in a way that is transparent to the user. It allows existing stovepipe systems to be integrated into one common data transmission network. When linked with a fixed shore site, to provide wire/fiber connectivity, this network becomes, in essence, a mobile extension of the Internet. 1. Mobile Platforms Although ADNS was specifically designed for the Navy, it's commercial potential is great. The easiest technology transfer can be applied to maritime platforms. Commercial and research ships have similar needs as the Navy for transferring data to and from shore sites. Imagery (such as weather) transfer, e-mail, Internet access, and file transfer capability are becoming essential tools necessary to accomplish everyday tasks. Commercial aircraft crew and passengers can also benefit from these same capabilities. Cellular phones in automobiles are commonplace. Some cars already receive one way satellite position information using the Global Positioning System (GPS). Currently there is even auto industry research into providing cars with Internet access. The field of mobile communications has become increasingly complex and will continue to grow. However, what should be avoided is a spaghetti-like architecture of different transmission paths linked to different applications. The traditional way to adopt new data transfer technologies is to implement a stovepipe system with its own dedicated transmission path. Mobile platforms, especially large ones like ships, typically have more than one transmission path for data transfer. However, if data is to be transferred, a dedicated radio link has to be assigned to a specific application. An application can not share different links or be distributed. The same is true for aircraft. Although more limited in space, aircraft too have different radio links which transmit data in a stovepipe fashion. The requirement for data transfer capability in autos is a relatively new concept. However, the reality of cellular phones and GPS combined with the possibility of Internet access already points to multiple transmission paths. Wireless communications do not have to be limited to just mobile platforms. It can also be a viable alternative to traditional shore links especially if they are saturated or can not be established, for example, in remote areas where the infrastructure just doesn't exist. 2. Alternative to Wire/Fiber Transmission Traditional shore transmission paths have been saturated with the increasing number of Internet users. Although much research has been done in alternative technologies to alleviate this congestion, such as installing optical fiber, these solutions often require investing in a whole new and different infrastructure. However, ADNS does not provide the same high capacity data transfer capability as shore backbones but instead allows an alternative to traditional mediums for transmitting data without worrying about infrastructure changes. Wireless data transfer could also be an attractive short term solution for areas where the infrastructure doesn't exist or is temporary such as in remote regions. A parallel to this can be seen in many lesser-developed countries where cellular telephones have proliferated because of inadequate landline telephone networks. C. What Does ADNS Do? A mobile platform can be thought of as a roaming Local Area Network (LAN). What existed onboard U.S. Navy ships prior to ADNS was a potpourri of different LANs and radio systems. If data was to be transferred to and from a ship, a different radio system was used for each application. ADNS allows platforms with more than one transmission path to integrate these different systems via one black box (ADNS), which then distributes data throughout the different paths in the most efficient manner. This method is desirable for several reasons. 1. Load Sharing If one or more transmission paths fail or are congested, ADNS can redirect data flow to open channels, which leads to an increased quality of service (QoS). ADNS can distribute data flow much more efficiently than the present stovepipe system. For example, a video teleconference (VTC) often inundates bandwidth, leaving other applications looking for an open transmission path. Other applications such as e-mail can be redirected to less congested channels instead of being stacked in a queue, waiting for transmission. 2. Cost Effective Bandwidth ADNS can direct data from different applications through desired transmission paths. This can be done to preferentially use the most cost-effective means for data transfer. 3. Leverages the existing Internet Another big appeal for ADNS, and one of the main reasons why the Navy has developed it, is that ADNS ties together the existing stovepipe communications architecture. There is no need to create a brand new infrastructure. Existing organizational LANs can be connected to ADNS and have access to the full range of communications assets available to that unit. 4. Flexibility The use of open protocols and Commercial off the Shelf (COTS) hardware creates a very flexible system. Modifications or additions to the shipboard LAN have no effect on ADNS. By using IP routers as the interface between ADNS and the shipboard LAN modifications on one side of the router are transparent to the other. Adding a new radio system is not much more complicated than adding a new circuit card. D. How does ADNS work? The easiest way to visualize how the system works is through an example. The following is a high level block diagram of ADNS and general description of operation. |---------------------------------| |---------------------------------| | | | | | _____ ________ ______ | Radio- | ______ ________ _____ | | | | | | | | | WAN | | | | | | | | | | LAN |---| Router |---| ADNS | |----------| | ADNS |---| Router |---| LAN | | | |_____| |________| |______| | | |______| |________| |_____| | | | | | | -------------------- -----------| |---------------------------------| Mobile Platform Mobile Platform Suppose that a user on a ship at sea wished to transfer a file to another user on a different ship. Let us also assume that both users' computers are connected to their respective shipboard LANs. When the originating user is ready to send the message he simply clicks on the appropriate button to send the message on its way via the ship's LAN. The size of most data files will necessitate their being broken into multiple IP datagrams. The router, processing each datagram independently, uses the Open Shortest Path First (OSPF) protocol to determine the best path(s) to reach the destination. If there are multiple equal cost paths the router will balance the load amongst them. Similar to a packet switched network a single message may be routed via multiple paths. The router then forwards the datagrams to ADNS. ADNS prioritizes, queues and transmits the datagrams on the selected radio system. The transmitted datagrams transit the Radio-WAN much the same way as in a packet switched network. At the destination there is a mirror image of the transmitting site. Arriving IP datagrams pass back through ADNS to the router and onto the LAN where they are received and reassembled at the destination host. This example described the transmission of one message to one destination via a single RF path. To understand the system's true potential, envision multiple ADNS capable platforms communicating simultaneously from multiple applications via multiple RF paths. E. ADNS Advantages 1. Removing humans from the loop In current naval communication systems, messages are generated on personal computers or workstations. These messages are transmitted via LAN, (or by use of magnetic media such as floppy disks where no LAN exists), to the communication center. The messages are then processed by technicians and transmitted. This process introduces time delays ranging from minutes to hours. ADNS eliminates the need for human processing of messages by establishing a direct connection from any node on the LAN, through the transmitter, to the receiver at the intended destination. The result is complete automation of the transmission process, with total elimination of any handling delays caused by human interaction. 2. Load Sharing Most naval vessels maintain at least two operational communication channels at all times. The reason for multiple channels is a legacy one - systems were developed such that only certain types of information could be transmitted and received over each channel. This frequently results in one or more channels being completely silent, while another is backlogged with traffic. The Load Sharing Feature of ADNS was specifically designed to alleviate these backlogs by making more efficient use of all operational communication channels. This is accomplished assigning a "cost" value to each network. Message queues in each CAP are monitored and messages are routed evenly across equal cost circuits. 3. Optimal use of bandwidth Network costs are assigned such that higher capacity circuits are assigned lower cost values. ADNS maximizes throughput by finding the lowest cost path for a message to reach its destination. The combination of removing humans from the loop, load sharing and using the lowest cost paths discussed above results in a four-fold increase in throughput during peak traffic times. This is a direct increase in the bottom line throughput of the communications system without purchasing additional transmitters. 4. Communications Agility ADNS provides the capability for two units that do not share a common communication channel to maintain communications. As long as each unit is operating at least one communication channel and at least one node on the network is operating both channels simultaneously, communications can occur. This process is completely transparent to the users, and occurs with no human intervention. This is analogous to Internet packet delivery. Few end systems share a common communications channel (that is, they are on the same network segment). 5. Transparency of installation and use The installation of ADNS is totally transparent to the end users. It merely appears that a new router has been added to the LAN with links to many other LANs. There is no major LAN or transmitter reconfiguration that is required. Additionally, there are no major infrastructure modifications (cooling, ventilation, etc.) required and power requirements are modest. 6. Logistics The entire installation is small and lightweight, allowing it to be installed in any unused space without impacting shipboard weight and balance. 7. Ease of upgrade Following initial installation, upgrading of ADNS is quite simple. Addition of new communication channels can be accomplished through the installation of the appropriate CAP cards. Adding capabilities to ADNS itself, such as installing successive builds as they become available, is as simple as downloading the new software. Router reconfiguration is a relatively simple matter as well. 8. Single point for Communications Management ADNS provides a single point for monitoring all communications, both incoming and outgoing. Prior to ADNS, monitoring all communications was much more difficult due to the lack of interconnection between stovepipe systems. Each of these systems had to be monitored separately. This monitoring capability is available locally via the local net manager's workstation, or remotely from the Network Operations Center. 9. Ability to transmit all types of data Essentially, ADNS transmits Internet Protocol (IP) datagrams from one router to another. It is the applications on these LANs that decode the datagrams and put the information contained in them to use. Therefore, ADNS can transmit text, graphics, voice, or video applications over existing channels, without the need for developing expensive new stovepipe systems to support each new application. F. ADNS Disadvantages 1. Cost of installation The high initial cost of an ADNS installation is a large obstacle to its widespread use. However, new technology, innovation, and mass production of ADNS should continue to drive costs down. The hardware used in an ADNS installation is COTS equipment but it is very implementation specific. It is unlikely that a unit will already possess equipment that can be modified for ADNS in order to save money on an initial installation. However, future builds of ADNS are planned that will incorporate more readily available hardware. II. ADNS Operational Description A. Overview The behavior of the Radio-WAN created by ADNS is the same as a terrestrial WAN. The router on one platform still "talks" to routers on other platforms, but at a slower rate than if they were connected by wire or fiber. Some of the circuits used in the Navy's ADNS program, such as HF and UHF have transmission rates in the 2.4Kbps range. The insertion of the ADNS hardware and the RF transmission path is simply a conduit for creating a router based network. ADNS deals strictly with IP datagrams. Although some encapsulation occurs as a result of the handling process the underlying packets are not altered and thus the path between destinations is in essence transparent to the routers. The diagram below shows the relative position of each component in a typical ADNS setup. The minimum component mix needed for a complete ADNS installation consists of: LAN-Router-CRIU-CAP-Cryptographic Device-Modem-RF System. From the Channel Access Protocol (CAP) to Router Interface Unit (CRIU) back (to the left) there will be only one of each for a given installation. From the CAP forward (to the right) there will be one chain for each radio system that is part of the system (i.e. there may be a UHF SATCOM chain, a UHF LOS chain, an SHF chain, an HF chain, etc.). In this particular configuration there are three RF paths connected to ADNS. _____ _______ _______ ______ | | | | | | | | |-| CAP |---|CRYPTO |---|MODEM |---|RADIO | | |_____| |_______| |_______| |______| _____ ________ ______ | _____ ______ _______ ______ | | | | | | | | | | | | | | | | LAN |---| Router |---| CRIU |---| CAP |---|CRYPTO |---|MODEM |---|RADIO | |_____| |________| |______| | |_____| |_______| |_______| |______| | _____ _______ _______ ______ | | | | | | | | | |-| CAP |---|CRYPTO |---|MODEM |---|RADIO | |_____| |_______| |_______| |______| As discussed earlier, the router accepts outbound datagrams from the LAN and selects the best path for reaching the destination. The CRIU, which interfaces between the router and CAP, assigns a priority to outbound IP datagrams. Priority is inferred based on both the source application (logical port number) and the host (IP address) from which the message originated. At the CAP the message is placed in a queue to await transmission. Messages in the CAP queue are sorted by the priority assigned by the CRIU. When the message leaves the CAP it passes through a cryptographic device. The standard Navy ADNS configuration operates at the secret high level of classification, thus all information entering the RF network is link encrypted. This: 1. Conforms to existing practice. 2. Provides resistance to AS spoofing. 3. Provides limited content confidentiality/authenticity protection (because this layer of encryption is stripped off at each routing point). Although this provides protection during transmission it does not provide content security once the information passes through the cryptographic device at the receiving end. 4. Provides opportunities for secure tunnels such as Unix Secure Shell (SSH) or Network Encryption System (NES), which deal with IP datagram encapsulation (IP datagrams inside other datagrams). These encapsulated IP datagrams are transmitted by ADNS in the same manner as any other IP datagrams. 5. Does not affect applications that offer end-to-end security (e.g. secure e-mail). Similar to secure tunnels, end system encrypted datagrams are unaffected by the presence of ADNS in the system. After leaving the Cryptographic device the datagram passes through a modem and then enters the transmitter. Once it leaves the ship the message begins traveling via the predetermined path to its destination. Upon arrival at its destination the datagram, traveling through a mirror image of the originating system, terminates at the host specified in the IP header. B. Network Features 1. Routing Protocols ADNS uses three different routing protocols. The primary reason for using these algorithms was that the specifications for all three are in the public domain. a. Open Shortest Path First (OSPF)/Multicast OSPF (MOSPF) OSPF is used as the Internal Gateway Protocol (IGP) for routing within an AS. The specification for OSPF Version 2 is contained in Request For Comments (RFC) 2178. It is a dynamic protocol in that each router maintains a continuously updated database containing the status of all other routers in the same system. OSPF uses a lowest cost algorithm to determine the best path to send a message to its destination. Costs are determined based on metrics values assigned to the various transmission paths. Multicast OSPF (MOSPF) is used for multicast within an AS. The specification for MOSPF is contained in RFC 1584. MOSPF uses the same lowest cost concept as OSPF except the lowest cost is determined with respect to the group. b. Border Gateway Protocol Version 4 (BGP4) BGP4 is used as the External Gateway Protocol (EGP) for routing between ASs. Specifics for BGP4 can be found in RFC 1771. BGP4 is not as dynamic as OSPF and makes its routing decisions based on predetermined routes. In ADNS, BGP4 will typically reside at the shore station in a system. Since BGP4 requires a more stable environment than OSPF the shore station is the logical choice. 2. Logical Organization The naming and logical grouping of the elements in an ADNS network are based on the concepts established by the routing protocols used by ADNS. The basic unit of an OSPF network is an area. For ADNS a ship is typically considered an area. Certain shore installations will also be areas since the ships need an interface point with other shore based establishments. A number of ships grouped together using OSPF create an Autonomous System (AS). A typical AS consists of a group of Navy ships with some logical connection, such as a common mission. A Battle Group is a typical AS. The emphasis in AS establishment is on mission and not location. The units do not have to be in the same geographic region to be in the same AS. At least one and possibly two or more shore communications establishments will also be a part of an AS to act as the gateway to other navy networks such as SIPRNET (Secret IP Router Network) or the Internet. The combined network of RF systems creates the subnet backbone of the AS. Each subnet is a different RF system such as UHF Satcom, SHF Satcom or INMARSAT B. The router on each ship that interfaces with ADNS is established as an Area Border Router (ABR). Each ABR operates OSPF. Part of the data that is maintained in the OSPF routing tables are metrics for each subnet in the AS. In current ADNS installations, metrics values are assigned based on subnet capacity or bandwidth. Higher capacity subnets are assigned lower metric values. The values chosen for these metrics determine how the system performs load balancing and load sharing, as discussed below. Obviously since each router must maintain a dynamically updated table of every other router in the AS there is a limit to the number of routers which can be managed effectively. This is what drives the upper limit to the size of an AS. The router that acts as the gateway between an AS and other ASs, WANs, or the Internet uses BGP4. The shore establishment usually performs this function since BGP4 needs a stable environment. The OSPF to BGP4 transition acts to hide the internals of the AS from the outside. Routers outside the AS don't need to know the specifics of all the routers inside the AS. They only need to know where the BGP4 gateway into the AS is. Changing missions will prompt changes to an AS. Ships may need to transfer from one AS to another to support operational or training objectives. This dynamic reorganization requirement reinforces the need to shield the internal routing issues of each AS from the outside. This illustration shows the relationship between routers within a simple Autonomous System. To other Autonomous Systems or Wide Area Networks | ___ / \ / \ / BGP4 \ (-(Shore)-) \ OSPF /| |\ / | | \___/ | | | | | | | ___ +-)---)----)--+ ___ / \ ------------|-)--EHF---)--|-------------/ \ / \ | | | | / \ / OSPF \-----------|-HF-------)--|-----------/ OSPF \ ( router ) | | | ( router ) \(Ship) /-----------|---------UHF-|-----------\(Ship) / \ / +-------------+ \ / \___/ Backbone Networks \___/ The third party routing feature of this type of network can be seen in the illustration below. If the originating and destination ships are not operating a common circuit ADNS will route traffic through a third platform which has connectivity on both source and destination circuits. By maintaining the status of other ships in the AS, ADNS can determine the best path to ensure delivery of each message. The diagram shows how the sending ship's router (R1) will send via either EHF or UHF (or both, depending on the metric values assigned to each RF path) to R3. R3 will then forward via HF to the destination ship, R2. ___ / \ / \ / \ ( R3 ) \ /| |\ / | | \___/ | v | ^ | ^ | ___ +-)---)----)--+ ___ / \------->-----|-)--EHF ) | / \ / \ | | | | / \ / R1 \ | HF-------)--|--->-------/ R2 \ ( Origin ) | | | ( Dest. ) \ /------>----|---------UHF | \ / \ / +-------------+ \ / \___/ \___/ C. Key Features/Functions of ADNS 1. Priority Several different methods for assigning priority to outgoing messages were evaluated during the ADNS implementation process. One obvious method, using the built-in precedence field in the IP header, was briefly considered. This idea was quickly discarded since no relevant applications currently use this feature of the IP header. Eventually, a priority scheme was implemented which assigned priorities of 0 (lowest) to 15 (highest). The two methods which proved most useful for assigning priority were based on source IP address (Host), or port number (Application). This approach has the same advantages and drawbacks of a firewall that uses the same data to make its filtering decisions. The advantage is its practicality. The disadvantage is that it's rather crude and, at the moment requires manual configuration of the router's routing table. a. Priority Tables The CRIU maintains two priority tables. The Source IP table contains the IP addresses of hosts on the associated LAN and the priority which they have been assigned. There are no default settings for this table. If a host is to have an associated priority it must be entered into the table. This table is filled in manually by the local ADNS Manager during initial system configuration and can be updated at any time. The Source IP table contains space for up to 40 entries. The second table maintained by the CRIU is the Port priority table. It contains the dedicated port numbers used by certain applications and the priority that has been assigned to that particular application. Just as with the Source IP table above, there are no default values, priorities must be entered manually, and it contains space for up to 40 entries. b. Determining Message Priority The CRIU receives datagrams from the router. The CRIU determines the port number and originating IP address for each datagram and assigns priority based on entries in the Source IP and Port priority tables. Here, a conflict may arise. If the Source IP priority table assigns a certain priority to a particular datagram and the Port priority table indicates a different priority for the same datagram, priority assignment will be made on the basis of Source IP address. This allows priority based primarily on host, and secondarily on application should the host have no assigned priority. If neither the host nor the application have an assigned priority, the CRIU assigns a default value of priority 4. Once assigned, the priority is placed in the IP datagram header and the entire IP datagram is passed to the CAP. c. Message Transmission Following Assignment of priority, the IP datagram is forwarded to the appropriate CAP, where it is entered into one of 16 queues based on priority. Datagrams are assembled into transmission units, each of which can contain up to 64 IP datagrams. The size of the transmission unit depends on the capacity of the link. Lower capacity links will have to utilize lower transmission unit sizes. The CAP builds a transmission unit by removing datagrams from the queues in order of priority. Datagrams are removed from the highest priority queue first, until it is empty. Datagrams are removed in sequence, continuing down the priority queues until the transmission unit is complete or all queues are empty. The transmission unit is sent from the CAP to the corresponding RF transmitter and the process is repeated. 2. Load Balancing Load balancing is the sharing of transmission load equally among different subnets. When the router selects a transmission path it does so based on the metrics assigned to that RF system. OSPF metrics are based on link capacity, with links having similar capacity being assigned identical metric values. If multiple CAPs have the same metrics values then the router will balance the load evenly by alternating between those CAPs. For load balancing to work effectively the sharing must be done among systems of equivalent capacity. Consequently, when assigning metric values to RF systems it is important that only networks of like capacity be assigned the same values. For example, if a ship is operating two active subnets, HF (which operates at about 2.4Kbps) and SHF (which operates at about 64Kbps) assigning the same metric values to each would overload the HF circuit. The router would divide the load equally between the two, not proportionally. During periods of high traffic density the SHF link could handle the load more effectively than the HF link, which would become backlogged with data. 3. Congestion Control As described above, each CAP maintains separate queues for each priority (0-15). Should one of these queues become full, the CAP does not provide any overflow queue so additional datagrams with this same priority will be dropped. In order to prevent this situation from occurring, the CRIU monitors the CAP queues and either starts load sharing or issues a Source Quench command. Each queue in a CAP is allocated a certain queue size to store IP datagrams prior to transmission. The CAP manages this queue space. The CRIU sets a queue threshold, slightly smaller than the queue size, to use as a benchmark to determine if congestion of the queue exists. The gap between the queue threshold and the maximum queue size provides a buffer to allow action to be taken before the queue becomes full and datagrams start being discarded. These queue thresholds are pre-determined and entered into the CRIU by the local ADNS Manager. The congestion identification function operates in the following sequence. The CAP generates a queue report, at intervals specified by the queue report threshold. This report captures the actual queue levels and sends them to the CRIU. These levels are compared to the queue threshold for each queue. If any queue level is greater than the queue threshold, then a congestion condition exists in that queue. The macro behavior of this arrangement is very similar to congested routers in a conventional Internet so TCP, including the Karn and Nagel algorithms, will work without change. a. Load Sharing One of the key features of ADNS is its ability to share the traffic load over available subnets. In current Navy circuits a situation frequently occurs in which one communication channel is overloaded while another is completely idle. The load sharing feature of ADNS alleviates this problem by shifting some of the congestion to the idle channel, thereby increasing throughput and shortening communication system delays. This differs from load balancing in that balancing distributes traffic over channels with similar metric values before congestion occurs. Sharing distributes traffic over similar cost channels because a congestion condition exists. 1) Restrictions There are two restrictions on the use of load sharing. First, the traffic being shifted to an alternate channel must be unicast traffic only. Multicast applications introduce a level of complexity that causes diminished returns, making it not worth the effort to attempt to load share using multicast applications. Second, load sharing is only feasible between subnets whose bandwidths are in the same range, meaning they share a similar time delay. Thus, possible opportunities for a load sharing situation are between UHF and EHF, or between SHF and Challenge Athena. 2) Implementation The load sharing process begins when the CRIU determines that a congestion condition exists on a subnet in one of its associated CAPs. The CRIU then scans all other compatible (those with similar delay times) subnets to determine if a path from origin to destination exists. If another subnet does exist with a path from origin to destination and no congestion condition exists on this subnet, load sharing commences. b. Source Quench When congestion is determined to exist in the CAP queue for priority n, the CRIU issues a Source Quench ICMP command. This command stops the generation of message packets for all applications and hosts with priority n or less. Assuming compliant TCPs this Source Quench command has been pre-set to remain in force for five seconds. At the end of five seconds, transmission from the affected hosts and applications resumes automatically unless or until another Source Quench command is issued. It should be noted that all applications and hosts require some sort of flow control to ensure that during Source Quench conditions, packets are not discarded but rather stored for transmission when the Source Quench has timed-out. 4. Transmission Control Protocol (TCP) Duplicate Packet Transmission Problems One of the major early setbacks to implementing the ADNS architecture was solving the problem of TCP duplicate transmissions when initially establishing a TCP connection. ADNS causes the LAN gateway router to act as if it is hard-wired to other routers on other LANs. Thus the router expects to encounter minimal delays (less than 0.5 seconds) in receiving acknowledgments to its TCP packets being sent. In reality, these TCP packets are being transmitted over RF links to distant LANs. The minimum acknowledgment time for a 1500 byte packet over a 2400 BPS connection is in the neighborhood of 5 seconds. When TCP hasn't received packet acknowledgment after 0.5 seconds, it re-transmits the packet. If acknowledgement is still not received after an additional 1 second, TCP retransmits the packet again, and again after 2 seconds, 4 seconds, 8 seconds, and so on. Under optimal conditions, a 1500 byte packet will be sent 4 times over a 2400 BPS connection. The end result is the use of 6000 bytes to transmit 1500, an efficiency of 25%. a. TCP Duplicate Packet Rejection A practical solution, and the one implemented in ADNS, is to design the CRIU to discard duplicate TCP packets before they are transmitted over the RF link. This is accomplished by the use of a table for each subnet that contains the TCP sequence number and time-stamp indicating when the packet was received by the CRIU for transmitting. A TCP original packet and each duplicate packet sent will have the same TCP sequence number. When a TCP packet is received by the CRIU for transmitting, its TCP sequence number is examined. If this number already exists in the table, the packet is rejected. If this number does not exist in the table, it is added to the table along with its time-stamp, and the packet is passed along for transmission. Each subnet is assigned a TCP duplicate rejection time. If a TCP sequence number has been in the table for longer than the TCP duplicate rejection time, it is deleted from the table. The TCP duplicate rejection time has a default value of 10 seconds. This provides for transmission of the original TCP packet followed by a 10 second delay for acknowledgment. If none is received, the packet is allowed to be retransmitted followed by another 10 second delay. This time delay can be modified by the Local ADNS Manager, based on the latency of the link, for optimum performance. D. ADNS INTEGRATED NETWORK MANAGEMENT 1. Overview Network management of ADNS is based on SNMPv1 standards. There are no proprietary Navy protocols to confront, thus allowing the use of standard network management tools and practices. Most of the objects to be managed (hosts, routers, etc) will have agents attached and MIBs will be written for any unique objects. The Navy will adopt a standard, commercial Network Management System (NMS) to provide the foundation for network management. However, there are Navy-specific concerns, such as command and control relationships, which impact network management. For these special requirements, the Navy will create special applications and concepts to the NMS. This section gives a broad description of how the Navy intends to manage ADNS. Network management of naval nodes is similar to managing shore-based nodes. The fundamental concepts are the same. However, the mobile nature of the nodes makes managing shipboard nodes more difficult. The fact that they are warships makes management more important. Just as there is a military hierarchy there is one for network management in ADNS, where each level has different responsibilities. Network management is a vital portion of ADNS because the consequences of system errors or failures can directly affect combat effectiveness. Integrated network management describes how the Navy will manage networks on a distributed basis all the way down to individual objects. They include, but are not limited to: general monitoring, statistic collection, status monitoring, traffic monitoring, trend analysis, network loading, network optimization, configuration control, system configuration, maintenance, problem identification, problem reporting, trouble documentation, system administration, and emissions control [INM Technical Approach]. Network management of ADNS contains three different levels: the Local Control Center (LCC), Autonomous System Control Center (ASCC), and the Navy Operations Center (NOC). The LCC will be responsible for networks at the local level, e.g. within an area (usually a ship). The ASCC will be in charge of networks on a regional level, having several subordinate Autonomous Systems. The NOC will be responsible for all ASCCs in a certain geographic area. This arrangement is consistent with the Navy's organization and its doctrine regarding distribution of authority. a. Local Control Center (LCC) The LCC is the network management center at every unit level. There is a local responsibility to monitor and maintain the status of all subnets at that unit. There are three components of an LCC: a Network Manager, Distributed Manager and a Communication Automation Manager. i) Network Manager The Network Manager is network management system software that is obtained commercially. The purpose of the network manager is basically to give the status of the network and individual objects. An example is the popular HP Open View Network Node Manager product (OV-NNM) which has been in the Navy Tactical Advanced Computer (TAC) contracts since 1991. It provides a topological map representation of a unit's network and shows the status of each object with the use of colors and shapes. However, human interaction is required to interface with the ASCC and the NOC for troubleshooting or maintenance. The specific functions of a Network Manager will be: * Human machine interface * Performance management * Fault management * Accounting management * Security management * Configuration management The Network Manager will be used as the foundation for the Navy's Integrated Network Management System, where specific applications can then be added on to provide other management functions. ii) Distributed Manager Distributed Management is an application that determines what is to be reported locally and what is to be reported to the ASCC and NOC. The Distributed Manager has two mechanisms for discovering if any conditions exist that meet the criteria of its policy rules: * Notification from the Network manager * Query from Distributed Manager to Network Manager The specific functions of the Distributed Manager will be: * Interpretation and implementation of policy * Filtering of management information Although commercial products can provide these functions, the distributed manager in the Navy context specifically describes the policy rules for the communication relationships between the LCC and ASCC. iii) Communication Automation Manager (CAM) The Communication Automation Manager is in charge of the physical communication hardware and their related requirements. On a ship, they are functions typically related to the radio room. Duties include a communication plan implementation, circuit building, and circuit management. Three areas make up the Communication Automation Manager: the Communication Manager, Site Manager, and Equipment Manager. The specific functions of the Communication Automation Manager will be: * Security management * Log Control * Alarm reporting * Summarization * Attributes for representing relationships * Objects and attributes for access control * Usage Metering * Test Management * Event Report Management * State Management * Security alarm reporting * Object management * Bandwidth management * Communication plan management * Equipment control * Site configuration management The Navy specific application for these functions is the use of a remote management tool called the Communications Plan (COMMPLAN). The COMMPLAN will used to direct certain network management functions as described above. This is still mainly accomplished manually by a technician after receiving the COMMPLAN via hardcopy message. However ADNS will allow many of these requirements to be accomplished remotely and automatically via the COMMPLAN transmitted to the Communication Automation Manager. This concept can be applied to commercial industries where it is not cost effective to have the necessary network management expertise at every local site but can instead be centralized at one remote center. b. Autonomous System Control Center (ASCC) An ASCC monitors the operation of several LCCs. The Navy's configuration will use its regional shore communications master stations (NCTAMS) as ASCCs. The ASCC will receive summary reports from subordinate LCCs. The exact nature of reporting from an LCC to an ASCC is still to be determined but will contain mission relevant information. Such reporting requirements can include: * Readiness of communication to support the mission. * Status of communication services. * Status of hardware and software. * Information about usage and reliability. ASCCs can also give direction to LCCs regarding communications posture. This could include such items as prioritization of resources or equipment configuration changes. c. Network Operations Center (NOC) The NOC is the next level above an ASCC for reporting network management information. The NOC would basically monitor all nodes in a certain geographic location. For example the Navy has established a NOC in the Pacific and Atlantic regions. Although capable of monitoring detailed network management information, a NOC would be more interested on the overall status of ASCCs and LCCs. 2. NETWORK MANAGEMENT TOOLS To achieve the above network management requirements, a vast array of tools are available to all levels of management and maintenance personnel. However, each tool comes with their own training requirement. Therefore the total cost of ownership must be taken into consideration against their utility. The basic tool for monitoring the network is commercially available Network Management System software. Another tool available for the goal of transparent and affordable network management is software that is capable of remote monitoring and maintenance. These can also be available commercially or can be developed to be mission specific. There are always emerging tools on the horizon for new technologies. However, one of the primary reasons why network management techniques lag behind new network technologies is that time is needed to see which technologies will become established as industry standards. ADNS will manage objects primarily through SNMPv1 standards. That is not to say that ADNS can not adapt any emerging technologies that become industry standards, such as SNMPv2. However, SNMP has proved that it will be around for a long time. a. Network Management System Software (NMS) A commercial Network Management System software has been adopted for the foundation of the INM. Network Management System software allows for the basic functions of monitoring nodes and network status. As described earlier, many different types of enterprise management software are available commercially, such as the popular HP Open View Network Node Manager (OV-NNM). Although commercial software provides excellent monitoring tools, proprietary software is often required to achieve other network management requirements. Commercial Network Management System software offers a fairly inexpensive solution that provides a solid foundation of network management tools. Additionally, to provide the flexibility desired throughout ADNS a COTS product is appropriate. b. Third Party Applications An attractive feature of a Network Management System such as OV-NNM is that third party applications can be integrated into it. Especially for organizations like the Navy, solutions to mission specific requirements can not be obtained off the shelf. These mission specific add-ons must be developed independently and then integrated into the existing NMS. Proprietary equipment also requires some kind of integration with the NMS. Such things as configuration management software for specific objects must be obtained from the vendor. For example, companies offer software that can be integrated with an NMS to allow managers to remotely configure their hardware. Third party applications offer remote management capability. This is the whole purpose of enterprise management. It is very cost effective to centrally manage nodes rather than paying for the necessary expertise at every local level. Although there needs to be some human interaction at every level, full management capabilities are not required down to the local level. ADNS is a good example of the need for remote management. Implementation of remote management over ADNS will allow managers to configure and manage mobile platforms from a central management location. This, in turn, allows the assignment of minimal personnel at the local level, thus saving on personnel costs. With such standards as RMON and SNMPv2, remote managers can access remote networks in a secure manner and troubleshoot or reconfigure the network. For example, if one transmission path fails, a remote manager can gain access to the system via a second transmission path and troubleshoot the system. The use of more than one transmission path allows the ability to continually manage LCCs and even ASCCs remotely through just one open path. Although ADNS has not adopted such standards as RMON or SNMPv2 yet, the technologies currently exist and can be readily integrated into ADNS. III. Hardware A. LAN The LAN will typically be the existing shipboard Ethernet or FDDI network. Hosts on the network will run a wide variety of applications. B. Router The router is an IP router that acts as a gateway to the ADNS network. The router can be any commercial router capable of running OSPF. Currently the ADNS program uses the CNX 600 Proteon router. C. CRIU (Channel Access Protocol to Router Interface Unit) The CRIU is implemented on a single board computer installed in a VME chassis. D. CAP (Channel Access Protocol) A CAP is also implemented on a single board computer mounted in the same VME chassis as the CRIU. E. Cryptographic Device Navy ADNS installations use the KG-84 for link encryption. F. Modem For each CAP there is a corresponding Modem that performs the analog to digital (inbound) or digital to analog (outbound) conversion of data passing through ADNS. G. Connectivity Media Each RF system (e.g. UHF Satcom, EHF Satcom or INMARSAT B) constitutes one network when considering all assets in one ADNS Autonomous System. IV. Acknowledgements. The engineers at NRaD, Code 82 working on ADNS. Authors addresses. Rex A Buddenberg Naval Postgraduate School Code SM/Bu Monterey, Ca 93943 budden@nps.navy.mil The masters students have graduated and moved to new assignments in the military. With new e-mail addresses; reach them through the above. Complete copies of theses available at http://web.nps.navy.mil/~seanet. INTERNET DRAFT EXPIRES MARCH 1997 INTERNET DRAFT