Internet Engineering Task Force Inter-Domain Multicast Routing Working Group INTERNET-DRAFT W. Fenner draft-ietf-idmr-traceroute-ipm-00.txt Xerox PARC S. Casner Precept Software November 13, 1995 Expires: 3/31/95 A "traceroute" facility for IP Multicast. Status of this Memo This document is an Internet Draft. Internet Drafts are working docu- ments of the Internet Engineering Task Force (IETF), its Areas, and its Working Groups. Note that other groups may also distribute working documents as Internet Drafts. Internet Drafts are draft documents valid for a maximum of six months. Internet Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet Drafts as reference material or to cite them other than as a "working draft" or "work in progress." To learn the current status of any Internet-Draft, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Abstract This draft describes the IGMP multicast traceroute facility. As the deployment of IP multicast has spread, it has become clear that a method for tracing the route that a multicast IP packet takes from a source to a particular receiver is absolutely required. Unlike unicast traceroute, multicast traceroute requires a special packet type and implementation on the part of routers. This specification describes the required functionality. This document is a product of the Inter-Domain Multicast Routing working group within the Internet Engineering Task Force. Comments are soli- cited and should be addressed to the working group's mailing list at idmr@cs.ucl.ac.uk and/or the author(s). Casner, Fenner Expires March 1996 [Page 1]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 1. Introduction The unicast "traceroute" program allows the tracing of a path from one machine to another, using mechanisms that already existed in IP. Unfor- tunately, no such existing mechanisms can be applied to IP multicast paths. The key mechanism for unicast traceroute is the ICMP TTL exceeded message, which is specifically precluded as a response to multicast packets. Thus, we specify the multicast "traceroute" facility to be implemented in multicast routers and accessed by diagnostic programs. While it is a disadvantage that a new mechanism is required, the multi- cast traceroute facility can provide additional information about packet rates and losses that the unicast traceroute cannot, and generally requires fewer packets to be sent. Goals: + To be able to trace the path that a packet would take from some source to some destination. + To be able to isolate packet loss problems (e.g., congestion). + To be able to isolate configuration problems (e.g., TTL threshold). + To minimize packets sent (e.g. no flooding, no implosion). 2. Overview Tracing from a source to a multicast destination is hard, since you don't know down which branch of the multicast tree the destination lies. This means that you have to flood the whole tree to find the path from one source to one destination. However, walking up the tree from desti- nation to source is easy, as all existing multicast routing protocols know the previous hop for each source. Tracing from destination to source can involve only routers on the direct path. The party requesting the traceroute (which need be neither the source nor the destination) sends a traceroute request packet to the last-hop multicast router for the given destination. The last-hop router adds a response data block to the request packet containing its interface addresses and packet statistics, and then forwards the request packet via unicast to the router that it believes is the proper previous hop for the given source. Each hop adds its response data to the end of the request packet, then unicast forwards it to the previous hop. The first hop router (the router that believes that packets from the source ori- ginate on one of its directly connected networks) changes the packet type to indicate a response packet and sends the completed response to the response destination address. The response may be returned before reaching the first hop router if an error condition such as "no route" Casner, Fenner Expires March 1996 [Page 2]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 is encountered along the path. 3. Request / Response header The header for both requests and responses is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IGMP Type | # hops | checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multicast Group Address | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Response Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | resp ttl | Query ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 3.1. IGMP Type: 8 bits The IGMP type field is defined to be 0x1F for traceroute requests sent to the last hop router and forwarded hop by hop towards the source. The IGMP type field is changed to 0x1E when the packet is completed and sent as a response from the first hop router to the querier. Two codes are required so that multicast routers won't attempt to process a completed response in those cases where the initial query was issued from a router or the response is sent via multicast. 3.2. # hops: 8 bits This field specifies the maximum number of hops that the requester wants to trace. If there is some error condition in the middle of the path that keeps the traceroute request from reaching the first-hop router, this field can be used to perform an expanding- length search to trace the path to just before the problem. 3.3. Checksum: 16 bits This is the standard IGMP checksum. Casner, Fenner Expires March 1996 [Page 3]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 3.4. Group address This field specifies the group address to be traced. 3.5. Source address This field specifies the IP address of the multicast source for the path being traced. The traceroute request proceeds hop-by-hop from the intended multicast receiver towards this source. 3.6. Destination address This field specifies the IP address of the multicast receiver for the path being traced. The trace starts at this destination and proceeds toward the source. 3.7. Response Address This field specifies where the completed traceroute response packet gets sent. It can be a unicast address or a multicast address, as explained in section 5.2. 3.8. resp ttl: 8 bits This field specifies the TTL at which to multicast the response, if the response address is a multicast address. 3.9. Query ID: 24 bits This field is used as a unique identifier for this traceroute request so that duplicate or delayed responses may be detected and to minimize collisions when a multicast response address is used. Casner, Fenner Expires March 1996 [Page 4]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 4. Response data Each router adds a "response data" segment to the traceroute packet be- fore it forwards it on. The response data looks like this: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Query Arrival Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Incoming Interface Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Outgoing Interface Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Previous-Hop Router Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Input packet count on incoming interface | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Output packet count on outgoing interface | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Total number of packets for this source-group pair | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Rtg Protocol | FwdTTL |MBZ| Src Mask | ForwardingErr | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 4.1. Query Arrival Time The Query Arrival Time is a 32-bit NTP timestamp specifying the arrival time of the traceroute request packet at this router. The 32-bit form of an NTP timestamp consists of the middle 32 bits of the full 64-bit form; that is, the low 16 bits of the integer part and the high 16 bits of the fractional part. 4.2. Incoming Interface Address This field specifies the address of the interface on which packets from this source are expected to arrive, or 0 if unknown. 4.3. Outgoing Interface Address This field specifies the address of the interface on which packets from this source flow to the specified destination, or 0 if unk- nown. Casner, Fenner Expires March 1996 [Page 5]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 4.4. Previous-Hop Router Address This field specifies the router from which this router expects packets from this source, or 0 if unknown. 4.5. Input packet count on incoming interface This field contains the number of multicast packets received for all groups and sources on the incoming interface, or 0xffffffff if no count can be reported. 4.6. Output packet count on outgoing interface This field contains the number of multicast packets that have been transmitted for all groups and sources on the outgoing interface, or 0xffffffff if no count can be reported. 4.7. Total number of packets for this source-group pair This field counts the number of packets from the specified source forwarded by this router to the specified group, or 0xffffffff if no count can be reported. 4.8. Rtg Protocol: 8 bits This field describes the routing protocol in use between this router and the previous-hop router. Specified values include: 1 - DVMRP 2 - MOSPF 3 - PIM 4 - CBT 4.9. FwdTTL: 8 bits This field contains the TTL that a packet is required to have before it will be forwarded over the outgoing interface. 4.10. Src Mask: 6 bits This field contains the number of 1's in the netmask this router has for the source (i.e. a value of 24 means the netmask is 0xffffff00) 4.11. ForwardingErr: 8 bits This field contains a forwarding error code. Specified values include: Casner, Fenner Expires March 1996 [Page 6]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 0x00 No error 0x01 Traceroute request arrived on an interface that is not the proper outgoing interface for this source,group,destination. 0x02 This router has sent a prune upstream for the group. 0x03 The next hop router has pruned the group. 0x04 The group is subject to administrative scoping at this hop. 0x05 This router has no route for the source. 0x07 This router is not forwarding this source,group for an unspecified reason. 0x81 There was not enough room to insert another response data block in the packet. 0x82 The next hop router does not understand traceroute requests. Note that if a router discovers there is not enough room in a packet to insert its response, it puts the 0x81 error code in the previous router's ForwardingErr field, overwriting any error the previous router placed there. It is expected that a multicast tra- ceroute client, upon receiving this error, will restart the trace at the last hop listed in the packet. The 0x80 bit of the ForwardingErr code is used to indicate a fatal error. A fatal error is one that causes a router to be unable to forward this traceroute request on to the next hop. <<< Note that 0x01 and 0x05 should be fatal, but renumbering would be painful as backwards-compatibility is required. It's not clear that an explicit fatal bit is required because a response issued when the number of hops has not reached the maximum indicates that the trace cannot go further. Feedback from implementors is requested. >>> 5. Using multicast traceroute Several problems may arise when attempting to use multicast traceroute. 5.1. Last hop router The traceroute querier may not know which is the last hop router, or that router may be behind a firewall that blocks unicast packets but passes multicast packets. In these cases, the traceroute request should be multicasted to the group being traced (since the last hop router listens to that group). All routers except the correct last hop router should ignore any multicast traceroute request received via multicast. Casner, Fenner Expires March 1996 [Page 7]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 5.2. First hop router The traceroute querier may not be unicast reachable from the first hop router. In this case, the querier should set the traceroute response address to a multicast address, and should set the response TTL to a value sufficient for the response from the first hop router to reach the querier. It may be appropriate to start with a small TTL and increase in subsequent attempts until a suffi- cient TTL is reached, up to an appropriate maximum (such as 192). The IANA has assigned 224.0.1.32, MTRACE.MCAST.NET, as the standard multicast group for multicast traceroute responses. 5.3. Broken intermediate router A broken intermediate router might simply not understand traceroute packets, and drop them. The querier would then get no response at all from its traceroute requests. It should then perform a search by setting the number of responses field until it gets a response (both linear and binary search are options, but binary is likely to be slower because a failure requires waiting for a timeout). 6. Problem Diagnosis 6.1. Forwarding Inconsistencies The forwarding error code can tell if a group is unexpectedly pruned or administratively scoped. 6.2. TTL problems By taking the maximum of (hops from source + forwarding TTL thres- hold) over all hops, you can discover the TTL required for the source to reach the destination. 6.3. Congestion By taking two traces, you can find packet loss information by com- paring the difference in input packet counts to the difference in output packet counts at the previous hop. On a point-to-point link, any difference in these numbers implies packet loss. Since the packet counts may be changing as the trace query is propagat- ing, there may be small errors (off by 1 or 2) in these statistics. However, these errors will not accumulate if multiple traces are taken to expand the measurement period. On a shared link, the count of input packets can be larger than the number of output packets at the previous hop, due to other routers or hosts on the link injecting packets. This appears as "negative loss" which may Casner, Fenner Expires March 1996 [Page 8]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 mask real packet loss. In addition to the counts of input and output packets for all mul- ticast traffic on the interfaces, the response data includes a count of the packets forwarded by a node for the specified source- group pair. Taking the difference in this count between two traces and then comparing those differences between two hops gives a meas- ure of packet loss just for traffic from the specified source to the specified receiver via the specified group. This measure is not affected by shared links. On a point-to-point link that is a multicast tunnel, packet loss is usually due to congestion in unicast routers along the path of that tunnel. On native multicast links, loss is more likely in the out- put queue of one hop, perhaps due to priority dropping, or in the input queue at the next hop. The counters in the response data do not allow these cases to be distinguished. Differences in packet counts between the incoming and outgoing interfaces on one node cannot generally be used to measure queue overflow in the node because some packets may be routed only to or from other interfaces on that node. In the multicast extensions for SunOS 4.1.x from Xerox PARC, both the output packet count and the packet forwarding count for the source-group pair are incremented before priority dropping for rate limiting occurs and before the packets are put onto the interface output queue which may overflow. These drops will appear as (posi- tive) loss on the link even though they occur within the router. In release 3.3/3.4 of the multicast extensions, a multicast packet generated on a router will be counted as having come in an inter- face even though it did not. This can create the appearance of negative loss even on a point-to-point link. In releases up through 3.5/3.6, packets were not counted as input on an interface if the reverse-path forwarding check decided that the packets should be dropped. That causes the packets to appear as lost on the link if they were output by the upstream hop. This situation can arise when two routers on the path for the group being traced are connected by a shared link, and the path for some other group does not flow between those two routers because the downstream router receives packets for the other group on another interface, but the upstream router is the elected forwarder to other routers or hosts on the shared link. 6.4. Link Utilization Again, with two traces, you can divide the difference in the input Casner, Fenner Expires March 1996 [Page 9]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 or output packet counts at some hop by the difference in time stamps from the same hop to obtain the packet rate over the link. If the average packet size is known, then the link utilization can also be estimated to see whether packet loss may be due to the rate limit or the physical capacity on a particular link being exceeded. 6.5. Time delay If the routers have synchronized clocks, you can estimate propaga- tion and queueing delay from the differences between the timestamps at successive hops. 7. Acknowledgments This specification started largely as a transcription of Van Jacobson's slides from the 30th IETF, and the implementation in mrouted 3.3 by Ajit Thyagarajan. Van's original slides credit Steve Casner, Steve Deering, Dino Farinacci and Deb Agrawal. A multicast traceroute client, mtrace, has been implemented by Ajit Thyagarajan and Steve Casner. 8. Security Considerations Security issues are not discussed in this memo. Casner, Fenner Expires March 1996 [Page 10]
Internet Draft draft-ietf-idmr-traceroute-ipm-00.txt November 1995 9. Authors' Addresses William C. Fenner Xerox PARC 3333 Coyote Hill Road Palo Alto, CA 94304 Phone: +1 415 812 4816 Email: fenner@parc.xerox.com Stephen L. Casner Precept Software, Inc. 21580 Stevens Creek Blvd, Suite 207 Cupertino, CA 95014 Email: casner@precept.com Casner, Fenner Expires March 1996 [Page 11]
