Bloat » Cerowrt

Yep, it's been turned off all this time in cerowrt/openwrt.

http://www.ietf.org/rfc/rfc1323.txt

--snip--

It is vitally important to use the RTTM mechanism with big
      windows; otherwise, the door is opened to some dangerous
      instabilities due to aliasing.  Furthermore, the option is
      probably useful for all TCP's, since it simplifies the sender.

3.2  TCP Timestamps Option

TCP is a symmetric protocol, allowing data to be sent at any time
      in either direction, and therefore timestamp echoing may occur in
      either direction.  For simplicity and symmetry, we specify that
      timestamps always be sent and echoed in both directions.  For
      efficiency, we combine the timestamp and timestamp reply fields
      into a single TCP Timestamps Option.

The other thing on my list is checking when things are timestamped, particularly in the wireless driver. Given the impact of queue length on wireless it would make more sense to timestamp when it enters the queue, rather than when it exits the device. My thought is we timestamp when it hits the device, and that would explain some things.

Unless the router is the end point of the TCP connection, it won't be dealing with timestamps at all, so this isn't an issue for TCP sessions that are being routed through it; local connections won't end up with big windows.

Some ethernet/wireless hardware makes doing the timestamps cheap/free. So it isn't clear this change is a big deal (and may help performance on hardware where timestamps are expensive). I suspect the WNDR3700v2's hardware is recent enough that doing timestamps won't cost much.

Unless I'm missing something...

To be more clear, the majority of my tests have been using a router as an endpoint, to eliminate the variables introduced by using other devices as endpoints.

Secondly, when used as a web proxy, it was my hope that westwood+ would help, and it wasn't.

Thirdly timestamping would help vpn over tcp when the router is the endpoint there.

Fourthly, certain network monitoring tools on the router continually update the web page,
and would benefit from better congestion control.

As you also note, having it on when rarely used anyway, means it isn't going to hurt...

Thus, timestamping being off by default is a bad idea.

In some preliminary tests I saw no real difference in cpu usage with timestamping on, and a slight reduction in throughput (from about 94Mbit to 92.X Mbit) due to the increase in ack size.

But I now have several hundred gb of captures to throw out and some more thorough tests to re-run. I really hope to see westwood looking like westwood, in particular, now.

Dave Täht wrote:

The other thing on my list is checking when things are timestamped, particularly in the wireless driver. Given the impact of queue length on wireless it would make more sense to timestamp when it enters the queue, rather than when it exits the device. My thought is we timestamp when it hits the device, and that would explain some things.

And the context of the above comment is not actually as applicable to APs but to stations...

It seems this one is ready to be closed?
cerowrt indeed now has tcp timestamps enabled by default.

Relevant commit seems to be -

Commit: 61886bddee89adcd955c4de7dc940f071786d062
https://github.com/dtaht/cerofiles/commit/61886bddee89adcd955c4de7dc940f071786d062
Author: Dave Taht <dave.taht at bufferbloat.net>
Date: 2012-01-21 (Sat, 21 Jan 2012)

Changed paths:
    M files/etc/sysctl.conf

Log Message:
  -----------
  Make sure TCP timestamps are on by default

I just need to ship the patch upstream to openwrt. thx for the reminder.

And I'd turned it on, only to have months of weirdness to deal with. See #560 and the journal of irreproducible results.