Network Working Group V. Paxson, Editor
Internet Draft
Expiration Date: September 1997 March 1997
Known TCP Implementation Problems
<draft-ietf-tcpimpl-prob-00.txt>
1. Status of this Memo
This document is an Internet Draft. Internet Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of six
months, and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet Drafts as reference
material or to cite them other than as ``work in progress''.
To learn the current status of any Internet Draft, please check the
``1id-abstracts.txt'' listing contained in the Internet Drafts shadow
directories on ftp.is.co.za (Africa), nic.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast).
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
2. Introduction
This memo catalogs a number of known TCP implementation problems.
The goal in doing so is to improve conditions in the existing
Internet by enhancing the quality of current TCP/IP implementations.
It is hoped that both performance and correctness issues can be
resolved by making implementors aware of the problems and their
solutions. In the long term, it is hoped that this will provide a
reduction in unnecessary traffic on the network, the rate of
connection failures due to protocol errors, and load on network
servers due to time spent processing both unsuccessful connections
and retransmitted data. This will help to ensure the stability of
the global Internet.
Each problem is defined as follows:
Paxson, Editor [Page 1]
ID Known TCP Implementation Problems March 1997
Name The name associated with the problem. In this memo, the name is
given as a subsection heading.
Category
One or more problem categories for which the problem is classi-
fied. Categories used so far: "congestion control", "perfor-
mance", "reliability". Others anticipated: "security", "intero-
perability", "configuration".
Description
A definition of the problem, succinct but including necessary
background material.
Significance
A quanitification as to how serious the problem is considered.
Categories are "Non-critical", "Serious", and "Critical".
Implications
Why the problem is viewed as a problem.
Relevant RFCs
Brief discussion of the RFCs with respect to which the problem
is viewed as an implementation error.
Trace file demonstrating the problem
One or more ASCII trace files demonstrating the problem, if
applicable. These may in the future be replaced with URLs to
on-line traces.
Trace file demonstrating correct behavior
One or more examples of how correct behavior appears in a trace,
if applicable. These may in the future be replaced with URLs to
on-line traces.
References
References that further discuss the problem.
How to detect
How to test an implementation to see if it exhibits the problem.
This discussion may include difficulties and subtleties associ-
ated with causing the problem to manifest itself, and with
interpreting traces to detect the presence of the problem (if
applicable). In the future, this may include URLs for diagnos-
tic tools.
How to fix
For known causes of the problem, how to correct the implementa-
tion.
Paxson, Editor [Page 2]
ID Known TCP Implementation Problems March 1997
Implementation specifics
If it is viewed as beneficial to document particular implementa-
tions exhibiting the problem, and if the corresponding implemen-
tors approve, then this section gives the specifics of those
implementations, along with a contact address for the implemen-
tors.
3. Known implementation problems
3.1. No initial slow start
Category
Congestion control
Description
When a TCP begins transmitting data, it is required by RFC 1122,
4.2.2.15, to engage in a "slow start" by initializing its conges-
tion window, cwnd, to one packet (one segment of the maximum size).
It subsequently increases cwnd by one packet for each ack it
receives for new data. The minimum of cwnd and the receiver's
advertised window bounds the highest sequence number the TCP can
transmit. A TCP that fails to initialize and increment cwnd in
this fashion exhibits "No initial slow start".
Significance
Serious.
Implications
A TCP failing to slow start when beginning a connection results in
traffic bursts that can stress the network, leading to excessive
queueing delays and packet loss.
Implementations exhibiting this problem might do so because they
suffer from the general problem of not including the required
congestion window. These implementations will also suffer from "No
slow start after retransmission timeout".
There are different shades of "No initial slow start". From the
perspective of stressing the network, the worst is a connection
that simply always sends based on the receiver's advertised window,
with no notion of a separate congestion window. Some other forms
are described in "Uninitialized CWND" and "Initial CWND of 2 pack-
ets".
Relevant RFCs' 5
Paxson, Editor [Page 3]
ID Known TCP Implementation Problems March 1997
RFC 1122 requires use of slow start. RFC 2001 gives the specifics
of slow start.
Trace file demonstrating it
Made using tcpdump/BPF recording at the connection responder. No
losses reported.
10:40:42.244503 B > A: S 1168512000:1168512000(0) win 32768
<mss 1460,nop,wscale 0> (DF) [tos 0x8]
10:40:42.259908 A > B: S 3688169472:3688169472(0)
ack 1168512001 win 32768 <mss 1460>
10:40:42.389992 B > A: . ack 1 win 33580 (DF) [tos 0x8]
10:40:42.664975 A > B: P 1:513(512) ack 1 win 32768
10:40:42.700185 A > B: . 513:1973(1460) ack 1 win 32768
10:40:42.718017 A > B: . 1973:3433(1460) ack 1 win 32768
10:40:42.762945 A > B: . 3433:4893(1460) ack 1 win 32768
10:40:42.811273 A > B: . 4893:6353(1460) ack 1 win 32768
10:40:42.829149 A > B: . 6353:7813(1460) ack 1 win 32768
10:40:42.853687 B > A: . ack 1973 win 33580 (DF) [tos 0x8]
10:40:42.864031 B > A: . ack 3433 win 33580 (DF) [tos 0x8]
After the third packet, the connection is established. A, the con-
nection responder, begins transmitting to B, the connection initia-
tor. Host A quickly sends 6 packets comprising 7812 bytes, even
though the SYN exchange agreed upon an MSS of 1460 bytes (implying
an initial congestion window of 1 segment corresponds to 1460
bytes), and so A should have sent at most 1460 bytes.
The acks sent by B to A in the last two lines indicate that this
trace is not a measurement error (slow start really occurring but
the corresponding acks having been dropped by the packet filter).
A second trace confirmed that the problem is repeatable.
Trace file demonstrating correct behavior
Made using tcpdump/BPF recording at the connection originator. No
losses reported.
12:35:31.914050 C > D: S 1448571845:1448571845(0) win 4380 <mss 1460>
12:35:32.068819 D > C: S 1755712000:1755712000(0) ack 1448571846 win 4096
12:35:32.069341 C > D: . ack 1 win 4608
12:35:32.075213 C > D: P 1:513(512) ack 1 win 4608
12:35:32.286073 D > C: . ack 513 win 4096
12:35:32.287032 C > D: . 513:1025(512) ack 1 win 4608
12:35:32.287506 C > D: . 1025:1537(512) ack 1 win 4608
12:35:32.432712 D > C: . ack 1537 win 4096
Paxson, Editor [Page 4]
ID Known TCP Implementation Problems March 1997
12:35:32.433690 C > D: . 1537:2049(512) ack 1 win 4608
12:35:32.434481 C > D: . 2049:2561(512) ack 1 win 4608
12:35:32.435032 C > D: . 2561:3073(512) ack 1 win 4608
12:35:32.594526 D > C: . ack 3073 win 4096
12:35:32.595465 C > D: . 3073:3585(512) ack 1 win 4608
12:35:32.595947 C > D: . 3585:4097(512) ack 1 win 4608
12:35:32.596414 C > D: . 4097:4609(512) ack 1 win 4608
12:35:32.596888 C > D: . 4609:5121(512) ack 1 win 4608
12:35:32.733453 D > C: . ack 4097 win 4096
References
This problem is documented in [Paxson97].
How to detect
For implementations always manifesting this problem, it shows up
immediately in a packet trace or a sequence plot, as illustrated
above.
How to fix
If the root problem is that the implementation lacks a notion of a
congestion window, then unfortunately this requires significant
work to fix. However, doing so is critical, as such implementa-
tions exhibit "No slow start after retransmission timeout", which
has a significance of "Critical".
3.2. No slow start after retransmission timeout
Category
Congestion control
Description
When a TCP experiences a retransmission timeout, it is required by
RFC 1122, 4.2.2.15, to engage in "slow start" by initializing its
congestion window, cwnd, to one packet (one segment of the maximum
size). It subsequently increases cwnd by one packet for each ack
it receives for new data until it reaches the "congestion
avoidance" threshold, ssthresh, at which point the congestion
avoidance algorithm for updating the window takes over. A TCP that
fails to enter slow start upon a timeout exhibits "No slow start
after retransmission timeout".
Significance
Critical.
Implications
Entering slow start upon timeout forms one of the cornerstones of
Paxson, Editor [Page 5]
ID Known TCP Implementation Problems March 1997
Internet congestion stability, as outlined in [Jacobson88]. If
TCPs fail to do so, the network becomes at risk of suffering
"congestion collapse" [RFC896].
Relevant RFCs
RFC 1122 requires use of slow start after loss. RFC 2001 gives the
specifics of how to implement slow start. RFC 896 describes
congestion collapse.
The retransmission timeout discussed here should not be confused
with the separate "fast recovery" retransmission mechanism dis-
cussed in RFC 2001.
Trace file demonstrating it
Made using tcpdump/BPF recording at the sending TCP (A). No losses
reported.
10:40:59.090612 B > A: . ack 357125 win 33580 (DF) [tos 0x8]
10:40:59.222025 A > B: . 357125:358585(1460) ack 1 win 32768
10:40:59.868871 A > B: . 357125:358585(1460) ack 1 win 32768
10:41:00.016641 B > A: . ack 364425 win 33580 (DF) [tos 0x8]
10:41:00.036709 A > B: . 364425:365885(1460) ack 1 win 32768
10:41:00.045231 A > B: . 365885:367345(1460) ack 1 win 32768
10:41:00.053785 A > B: . 367345:368805(1460) ack 1 win 32768
10:41:00.062426 A > B: . 368805:370265(1460) ack 1 win 32768
10:41:00.071074 A > B: . 370265:371725(1460) ack 1 win 32768
10:41:00.079794 A > B: . 371725:373185(1460) ack 1 win 32768
10:41:00.089304 A > B: . 373185:374645(1460) ack 1 win 32768
10:41:00.097738 A > B: . 374645:376105(1460) ack 1 win 32768
10:41:00.106409 A > B: . 376105:377565(1460) ack 1 win 32768
10:41:00.115024 A > B: . 377565:379025(1460) ack 1 win 32768
10:41:00.123576 A > B: . 379025:380485(1460) ack 1 win 32768
10:41:00.132016 A > B: . 380485:381945(1460) ack 1 win 32768
10:41:00.141635 A > B: . 381945:383405(1460) ack 1 win 32768
10:41:00.150094 A > B: . 383405:384865(1460) ack 1 win 32768
10:41:00.158552 A > B: . 384865:386325(1460) ack 1 win 32768
10:41:00.167053 A > B: . 386325:387785(1460) ack 1 win 32768
10:41:00.175518 A > B: . 387785:389245(1460) ack 1 win 32768
10:41:00.210835 A > B: . 389245:390705(1460) ack 1 win 32768
10:41:00.226108 A > B: . 390705:392165(1460) ack 1 win 32768
10:41:00.241524 B > A: . ack 389245 win 8760 (DF) [tos 0x8]
The first packet indicates the ack point is 357125. 130 msec after
receiving the ack, A transmits the packet after the ack point,
357125:358585. 640 msec after this transmission, it retransmits
357125:358585, in an apparent retransmission timeout. At this
point, A's cwnd should be one MSS, or 1460 bytes, as A enters
Paxson, Editor [Page 6]
ID Known TCP Implementation Problems March 1997
slow-start. The trace is consistent with this possibility.
B replies with an ack of 364425, indicating that A has filled a
sequence hole. At this point, A's cwnd should be 1460*2 = 2920
bytes, since in slow start receiving an ack advances cwnd by MSS.
However, A then launches 19 consecutive packets, which is incon-
sistent with slow start.
A second trace confirmed that the problem is repeatable.
Trace file demonstrating correct behavior
Made using tcpdump/BPF recording at the sending TCP (C). No losses
reported.
12:35:48.442538 C > D: P 465409:465921(512) ack 1 win 4608
12:35:48.544483 D > C: . ack 461825 win 4096
12:35:48.703496 D > C: . ack 461825 win 4096
12:35:49.044613 C > D: . 461825:462337(512) ack 1 win 4608
12:35:49.192282 D > C: . ack 465921 win 2048
12:35:49.192538 D > C: . ack 465921 win 4096
12:35:49.193392 C > D: P 465921:466433(512) ack 1 win 4608
12:35:49.194726 C > D: P 466433:466945(512) ack 1 win 4608
12:35:49.350665 D > C: . ack 466945 win 4096
12:35:49.351694 C > D: . 466945:467457(512) ack 1 win 4608
12:35:49.352168 C > D: . 467457:467969(512) ack 1 win 4608
12:35:49.352643 C > D: . 467969:468481(512) ack 1 win 4608
12:35:49.506000 D > C: . ack 467969 win 3584
After C transmits the first packet shown to D, it takes no action
in response to D's acks for 461825, because the first packet
already reached the advertised window limit of 4096 bytes above
461825. 600 msec after transmitting the first packet, C
retransmits 461825:462337, presumably due to a timeout. Its
congestion window is now MSS (512 bytes).
D acks 465921, indicating that C's retransmission filled a sequence
hole. This ack advances C's cwnd from 512 to 1024. Very shortly
after, D acks 465921 again in order to update the offered window
from 2048 to 4096. This ack does not advance cwnd since it is not
for new data. Very shortly after, C responds to the newly enlarged
window by transmitting two packets. D acks both, advancing cwnd
from 1024 to 1536. C in turn transmits three packets.
References
This problem is documented in [Paxson97].
Paxson, Editor [Page 7]
ID Known TCP Implementation Problems March 1997
How to detect
Packet loss is common enough in the Internet that generally it is
not difficult to find an Internet path that will force retransmis-
sion due to packet loss.
If the effective window prior to loss is large enough, however,
then the TCP may retransmit using the "fast recovery" mechanism
described in RFC 2001. In a packet trace, the signature of fast
recovery is that the packet retransmission occurs in response to
the receipt of three duplicate acks, and subsequent duplicate acks
may lead to the transmission of new data, above both the ack point
and the highest sequence transmitted so far. An absence of three
duplicate acks prior to retransmission suffices to distinguish
between timeout and fast recovery retransmissions. In the face of
only observing fast recovery retransmissions, generally it is not
difficult to repeat the data transfer until observing a timeout
retransmission.
Once armed with a trace exhibiting a timeout retransmission, deter-
mining whether the TCP follows slow start is done by computing the
correct progression of cwnd and comparing it to the amount of data
transmited by the TCP subsequent to the timeout rtransmission.
How to fix
If the root problem is that the implementation lacks a notion of a
congestion window, then unfortunately this requires significant
work to fix. However, doing so is critical, for reasons outlined
above.
3.3. Inconsistent retransmission
Category
Reliability
Description
If, for a given sequence number, a sending TCP retransmits dif-
ferent data than previously sent for that sequence number, then a
strong possibility arises that the receiving TCP will reconstruct a
different byte stream than that sent by the sending application,
depending on which instance of the sequence number it accepts.
Such a sending TCP exhibits "Inconsistent retransmission".
Significance
Critical.
Implications
Paxson, Editor [Page 8]
ID Known TCP Implementation Problems March 1997
Reliable delivery of data is a fundamental property of TCP.
Relevant RFCs
RFC 793, section 1.5, discusses the central role of reliability in
TCP operation.
Trace file demonstrating it
Made using tcpdump/BPF recording at the receiving TCP (B). No
losses reported.
12:35:53.145503 A > B: FP 90048435:90048461(26) ack 393464682 win 4096
4500 0042 9644 0000
3006 e4c2 86b1 0401 83f3 010a b2a4 0015
055e 07b3 1773 cb6a 5019 1000 68a9 0000
data starts here>504f 5254 2031 3334 2c31 3737*2c34 2c31
2c31 3738 2c31 3635 0d0a
12:35:53.146479 B > A: R 393464682:393464682(0) win 8192
12:35:53.851714 A > B: FP 90048429:90048463(34) ack 393464682 win 4096
4500 004a 965b 0000
3006 e4a3 86b1 0401 83f3 010a b2a4 0015
055e 07ad 1773 cb6a 5019 1000 8bd3 0000
data starts here>5041 5356 0d0a 504f 5254 2031 3334 2c31
3737*2c31 3035 2c31 3431 2c34 2c31 3539
0d0a
The sequence numbers shown in this trace are absolute and not
adjusted to reflect the ISN. The 4-digit hex values show a dump of
the packet's IP and TCP headers, as well as payload. A first sends
to B data for 90048435:90048461. The corresponding data begins
with hex words 504f, 5254, etc.
B responds with a RST. Since the recording location was local to
B, it is unknown whether A received the RST.
A then sends 90048429:90048463, which includes six sequence posi-
tions below the earlier transmission, all 26 positions of the ear-
lier transmission, and two additional sequence positions.
The retransmission disagrees starting just after sequence 90048447,
annotated above with a leading '*'. These two bytes were origi-
nally transmitted as hex 2c34 but retransmitted as hex 2c31. Sub-
sequent positions disagree as well.
This behavior has been observed in other traces involving different
hosts. It is unknown how to repeat it.
In this instance, no corruption would occur, since B has already
indicated it will not accept further packets from A.
Paxson, Editor [Page 9]
ID Known TCP Implementation Problems March 1997
A second example illustrates a slightly different instance of the
problem. The tracing again was made with tcpdump/BPF at the
receiving TCP (D).
22:23:58.645829 C > D: P 185:212(27) ack 565 win 4096
4500 0043 90a3 0000
3306 0734 cbf1 9eef 83f3 010a 0525 0015
a3a2 faba 578c 70a4 5018 1000 9a53 0000
data starts here>504f 5254 2032 3033 2c32 3431 2c31 3538
2c32 3339 2c35 2c34 330d 0a
22:23:58.646805 D > C: . ack 184 win 8192
4500 0028 beeb 0000
3e06 ce06 83f3 010a cbf1 9eef 0015 0525
578c 70a4 a3a2 fab9 5010 2000 342f 0000
22:31:36.532244 C > D: FP 186:213(27) ack 565 win 4096
4500 0043 9435 0000
3306 03a2 cbf1 9eef 83f3 010a 0525 0015
a3a2 fabb 578c 70a4 5019 1000 9a51 0000
data starts here>504f 5254 2032 3033 2c32 3431 2c31 3538
2c32 3339 2c35 2c34 330d 0a
In this trace, sequence numbers are relative. C sends 185:212, but
D only sends an ack for 184 (so sequence number 184 is missing). C
then sends 186:213. The packet payload is identical to the previ-
ous payload, but the base sequence number is one higher, resulting
in an inconsistent retransmission.
Neither trace exhibits checksum errors.
Trace file demonstrating correct behavior
(Omitted, as presumably correct behavior is obvious.)
References
None known.
How to detect
This problem unfortunately can be very difficult to detect, since
available experience indicates it is quite rare that it is mani-
fested. No "trigger" has been identified that can be used to
reproduce the problem.
How to fix
In the absence of a known "trigger", we cannot always assess how to
fix the problem.
In one implementation (not the one illustrated above), the problem
manifested itself when (1) the sender received a zero window and
Paxson, Editor [Page 10]
ID Known TCP Implementation Problems March 1997
stalled; (2) eventually an ack arrived that offered a window larger
than that in effect at the time of the stall; (3) the sender
transmitted out of the buffer of data it held at the time of the
stall, but (4) failed to limit this transfer to the buffer length,
instead using the newly advertised (and larger) offered window.
Consequently, in addition to the valid buffer contents, it sent
whatever garbage values followed the end of the buffer. If it then
retransmitted the corresponding sequence numbers, at that point it
sent the correct data, resulting in an inconsistent retransmission.
Note that this instance of the problem reflects a more general
problem, that of initially transmitting incorrect data.
3.4. Failure to retain above-sequence data
Category
Congestion control, performance
Description
When a TCP receives an "above sequence" segment, meaning one with a
sequence number exceeding RCV.NXT but below RCV.NXT+RCV.WND, it
SHOULD queue the segment for later delivery (RFC 1122, 4.2.2.20).
A TCP that fails to do so is said to exhibit "Failure to retain
above-sequence data".
It may sometimes be appropriate for a TCP to discard above-sequence
data to reclaim memory. If they do so only rarely, then we would
not consider them to exhibit this problem. Instead, the particular
concern is with TCPs that always discard above-sequence data.
Significance
Serious.
Implications
In times of congestion, a failure to retain above-sequence data
will lead to numerous otherwise-unnecessary retransmissions, aggra-
vating the congestion and potentially reducing performance by a
large factor.
Relevant RFCs
RFC 1122 revises RFC 793 by upgrading the latter's MAY to a SHOULD
on this issue.
Trace file demonstrating it
Made using tcpdump/BPF recording at the receiving TCP. No losses
reported.
Paxson, Editor [Page 11]
ID Known TCP Implementation Problems March 1997
B is the TCP sender, A the receiver. A exhibits failure to retain
above sequence data:
10:38:10.164860 B > A: . 221078:221614(536) ack 1 win 33232 [tos 0x8]
10:38:10.170809 B > A: . 221614:222150(536) ack 1 win 33232 [tos 0x8]
10:38:10.177183 B > A: . 222150:222686(536) ack 1 win 33232 [tos 0x8]
10:38:10.225039 A > B: . ack 222686 win 25800
Here B has sent up to (relative) sequence 222676 in-sequence, and A
accordingly acknowledges.
10:38:10.268131 B > A: . 223222:223758(536) ack 1 win 33232 [tos 0x8]
10:38:10.337995 B > A: . 223758:224294(536) ack 1 win 33232 [tos 0x8]
10:38:10.344065 B > A: . 224294:224830(536) ack 1 win 33232 [tos 0x8]
10:38:10.350169 B > A: . 224830:225366(536) ack 1 win 33232 [tos 0x8]
10:38:10.356362 B > A: . 225366:225902(536) ack 1 win 33232 [tos 0x8]
10:38:10.362445 B > A: . 225902:226438(536) ack 1 win 33232 [tos 0x8]
10:38:10.368579 B > A: . 226438:226974(536) ack 1 win 33232 [tos 0x8]
10:38:10.374732 B > A: . 226974:227510(536) ack 1 win 33232 [tos 0x8]
10:38:10.380825 B > A: . 227510:228046(536) ack 1 win 33232 [tos 0x8]
10:38:10.387027 B > A: . 228046:228582(536) ack 1 win 33232 [tos 0x8]
10:38:10.393053 B > A: . 228582:229118(536) ack 1 win 33232 [tos 0x8]
10:38:10.399193 B > A: . 229118:229654(536) ack 1 win 33232 [tos 0x8]
10:38:10.405356 B > A: . 229654:230190(536) ack 1 win 33232 [tos 0x8]
A now receives 13 additional packets from B. These are above-
sequence because 222686:223222 was dropped. The packets do however
fit within the offered window of 25800. A does not generate any
duplicate acks for them.
The trace contributor (V. Paxson) verified that these 13 packets
had valid IP and TCP checksums.
10:38:11.917728 B > A: . 222686:223222(536) ack 1 win 33232 [tos
0x8] 10:38:11.930925 A > B: . ack 223222 win 32232
B times out for 222686:223222 and retransmits it. Upon receiving
it, A only acknowledges 223222. Had it retained the valid above-
sequence packets, it would instead have ack'd 230190.
10:38:12.048438 B > A: . 223222:223758(536) ack 1 win 33232 [tos
0x8] 10:38:12.054397 B > A: . 223758:224294(536) ack 1 win 33232
[tos 0x8] 10:38:12.068029 A > B: . ack 224294 win 31696
B retransmits two more packets, and A only acknowledges them. This
pattern continues as B retransmits the entire set of previously-
received packets.
Paxson, Editor [Page 12]
ID Known TCP Implementation Problems March 1997
A second trace confirmed that the problem is repeatable.
Trace file demonstrating correct behavior
Made using tcpdump/BPF recording at the receiving TCP (C). No
losses reported.
09:11:25.790417 D > C: . 33793:34305(512) ack 1 win 61440
09:11:25.791393 D > C: . 34305:34817(512) ack 1 win 61440
09:11:25.792369 D > C: . 34817:35329(512) ack 1 win 61440
09:11:25.792369 D > C: . 35329:35841(512) ack 1 win 61440
09:11:25.793345 D > C: . 36353:36865(512) ack 1 win 61440
09:11:25.794321 C > D: . ack 35841 win 59904
A sequence hole occurs because 35841:36353 has been dropped.
09:11:25.794321 D > C: . 36865:37377(512) ack 1 win 61440
09:11:25.794321 C > D: . ack 35841 win 59904
09:11:25.795297 D > C: . 37377:37889(512) ack 1 win 61440
09:11:25.795297 C > D: . ack 35841 win 59904
09:11:25.796273 C > D: . ack 35841 win 61440
09:11:25.798225 D > C: . 37889:38401(512) ack 1 win 61440
09:11:25.799201 C > D: . ack 35841 win 61440
09:11:25.807009 D > C: . 38401:38913(512) ack 1 win 61440
09:11:25.807009 C > D: . ack 35841 win 61440
09:11:25.884113 D > C: . 52737:53249(512) ack 1 win 61440
09:11:25.884113 C > D: . ack 35841 win 61440
Each additional, above-sequence packet C receives from D elicits a
duplicate ack for 35841.
09:11:25.887041 D > C: . 35841:36353(512) ack 1 win 61440
09:11:25.887041 C > D: . ack 53249 win 44032
D retransmits 35841:36353 and C acknowledges receipt of data all
the way up to 53249.
References
This problem is documented in [Paxson97].
How to detect
Packet loss is common enough in the Internet that generally it is
not difficult to find an Internet path that will result in some
above-sequence packets arriving. A TCP that exhibits "Failure to
retain ..." may not generate duplicate acks for these packets.
However, some TCPs that do retain above-sequence data also do not
Paxson, Editor [Page 13]
ID Known TCP Implementation Problems March 1997
generate duplicate acks, so failure to do so does not definitively
identify the problem. Instead, the key observation is whether upon
retransmission of the dropped packet, data that was previously
above-sequence is acknowledged.
Two considerations in detecting this problem using a packet trace
are that it is easiest to do so with a trace made at the TCP
receiver, in order to unambiguously determine which packets arrived
successfully, and that such packets may still be correctly dis-
carded if they arrive with checksum errors. The latter can be
tested by capturing the entire packet contents and performing the
IP and TCP checksum algorithms to verify their integrity; or by
confirming that the packets arrive with the same checksum and con-
tents as that with which they were sent, with a presumption that
the sending TCP correctly calculates checksums for the packets it
transmits.
It is considerably easier to verify that an implementation does NOT
exhibit this problem. This can be done by recording a trace at the
data sender, and observing that sometimes after a retransmission
the receiver acknowledges a higher sequence number than just that
which was retransmitted.
How to fix
If the root problem is that the implementation lacks buffer, then
then unfortunately this requires significant work to fix. However,
doing so is important, for reasons outlined above.
4. Security Considerations
This version of this memo does not discuss any security-related
implementation problems. Futures versions most likely will, so secu-
rity considerations will require revisiting.
5. Acknowledgements
Thanks to numerous correspondents on the tcp-impl mailing list for
their input: Steve Alexander, Mark Allman, Larry Backman, Jerry Chu,
Alan Cox, Kevin Fall, Richard Fox, Jim Gettys, Rick Jones, Allison
Mankin, Perry Metzger, der Mouse, Thomas Narten, Andras Olah, Steve
Parker, Francesco Potorti`, Luigi Rizzo, Allyn Romanow, Al Smith,
Jerry Toporek, Joe Touch, and Curtis Villamizar.
Paxson, Editor [Page 14]
ID Known TCP Implementation Problems March 1997
6. References
[Jacobson88]
V. Jacobson, "Congestion Avoidance and Control," Proc. SIGCOMM '88.
ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z
[Paxson97]
V. Paxson, "Automated Packet Trace Analysis of TCP Implementa-
tions," available in draft form from vern@ee.lbl.gov, Feb. 1997.
[RFC896]
J. Nagle, "Congestion Control in IP/TCP Internetworks," Jan. 1984.
[RFC1122]
R. Braden, Editor, "Requirements for Internet Hosts -- Communica-
tion Layers," Oct. 1989.
[RFC2001]
W. Stevens, "TCP Slow Start, Congestion Avoidance, Fast Retransmit,
and Fast Recovery Algorithms," Jan. 1997.
7. Author's Address
Vern Paxson <vern@ee.lbl.gov>
Network Research Group
Lawrence Berkeley National Laboratory
Berkeley, CA 94720
USA
Phone: +1 510/486-7504
Paxson, Editor [Page 15]
