Bug #304

I was poking through the ledbat kernel module, and I realized how important having tcp timestamps actually on, was…

and discovered it was turned off in sysctl.conf

Assuming that netperf doesn’t enable it itself (I have to look at some old packet captures) this invalidates all testing we’ve done to date, against westwood+. While there is a performance impact to timestamping, it’s kind of required to give tcp an actual clue as to real delays - in, for example, a tcp proxy case, or… as I do all the time - testing how well wireless is working from the host to the router.

Sigh. Enabled by default in rc8.

Yep, it’s been turned off all this time in cerowrt/openwrt.

http://www.ietf.org/rfc/rfc1323.txt

-snip-

It is vitally important to use the RTTM mechanism with big
windows; otherwise, the door is opened to some dangerous
instabilities due to aliasing. Furthermore, the option is
probably useful for all TCP’s, since it simplifies the sender.

3.2 TCP Timestamps Option

TCP is a symmetric protocol, allowing data to be sent at any time
in either direction, and therefore timestamp echoing may occur in
either direction. For simplicity and symmetry, we specify that
timestamps always be sent and echoed in both directions. For
efficiency, we combine the timestamp and timestamp reply fields
into a single TCP Timestamps Option.

The other thing on my list is checking when things are timestamped, particularly in the wireless driver. Given the impact of queue length on wireless it would make more sense to timestamp when it enters the queue, rather than when it exits the device. My thought is we timestamp when it hits the device, and that would explain some things.

Unless the router is the end point of the TCP connection, it won’t be dealing with timestamps at all, so this isn’t an issue for TCP sessions that are being routed through it; local connections won’t end up with big windows.

Some ethernet/wireless hardware makes doing the timestamps cheap/free. So it isn’t clear this change is a big deal (and may help performance on hardware where timestamps are expensive). I suspect the WNDR3700v2’s hardware is recent enough that doing timestamps won’t cost much.

Unless I’m missing something…

To be more clear, the majority of my tests have been using a router as an endpoint, to eliminate the variables introduced by using other devices as endpoints.

Secondly, when used as a web proxy, it was my hope that westwood+ would help, and it wasn’t.

Thirdly timestamping would help vpn over tcp when the router is the endpoint there.

Fourthly, certain network monitoring tools on the router continually update the web page,
and would benefit from better congestion control.

As you also note, having it on when rarely used anyway, means it isn’t going to hurt…

Thus, timestamping being off by default is a bad idea.

In some preliminary tests I saw no real difference in cpu usage with timestamping on, and a slight reduction in throughput (from about 94Mbit to 92.X Mbit) due to the increase in ack size.

But I now have several hundred gb of captures to throw out and some more thorough tests to re-run. I really hope to see westwood looking like westwood, in particular, now.

Dave Täht wrote:
> The other thing on my list is checking when things are timestamped, particularly in the wireless driver. Given the impact of queue length on wireless it would make more sense to timestamp when it enters the queue, rather than when it exits the device. My thought is we timestamp when it hits the device, and that would explain some things.

And the context of the above comment is not actually as applicable to APs but to stations…

It seems this one is ready to be closed?
cerowrt indeed now has tcp timestamps enabled by default.

Relevant commit seems to be -

Commit: 61886bddee89adcd955c4de7dc940f071786d062
https://github.com/dtaht/cerofiles/commit/61886bddee89adcd955c4de7dc940f071786d062 Author: Dave Taht
Date: 2012-01-21 (Sat, 21 Jan 2012)

Changed paths:
M files/etc/sysctl.conf

Log Message:
———–
Make sure TCP timestamps are on by default

I just need to ship the patch upstream to openwrt. thx for the reminder.

And I’d turned it on, only to have months of weirdness to deal with. See #560 and the journal of irreproducible results.