Bloat » Cerowrt

And it turned out RED was busted. Folding the patch back into cerowrt will not be a problem, but as usual with bugs of this kind, I have to throw out a month's worth of results.

I am building a version of both net-next for x86_64 and folding in this patch into cerowrt-rc8

commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc
Author: Eric Dumazet <eric.dumazet@gmail.com>
Date: Thu Dec 1 11:06:34 2011 +0000

sch_red: fix red_change

Le mercredi 30 novembre 2011 à 14:36 -0800, Stephen Hemminger a écrit :

> (Almost) nobody uses RED because they can't figure it out.
    > According to Wikipedia, VJ says that:
    >  "there are not one, but two bugs in classic RED."

RED is useful for high throughput routers, I doubt many linux machines
    act as such devices.

I was considering adding Adaptative RED (Sally Floyd, Ramakrishna
    Gummadi, Scott Shender), August 2001

In this version, maxp is dynamic (from 1% to 50%), and user only have to
    setup min_th (target average queue size)
    (max_th and wq (burst in linux RED) are automatically setup)

By the way it seems we have a small bug in red_change()

if (skb_queue_empty(&sch->q))
        red_end_of_idle_period(&q->parms);

First, if queue is empty, we should call
    red_start_of_idle_period(&q->parms);

Second, since we dont use anymore sch->q, but q->qdisc, the test is
    meaningless.

Oh well...

[PATCH] sch_red: fix red_change()

Now RED is classful, we must check q->qdisc->q.qlen, and if queue is empty,
    we start an idle period, not end it.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
    Signed-off-by: David S. Miller <davem@davemloft.net>

Also a major problem is that the current linux packet schedulers are scheduling 'superpackets' - full GSO and TSO offloads - up to 64k- through the scheduler.

The effect this has on any attempt to classify or schedule packets has to be seen to be believed.
A 64*k* 'packet', followed by a 64 byte packet, is a 1000x1 ratio. Mere packet oriented schedulers, like PFIFO_fast, survive this abuse, but the effect on packets marked background
or priority, over best effort, is correspondingly, um, 'enhanced'.

Spitting this sort of abuse through a grouping scheduler such as SFB or SFQ is really bad, as the above effect is magnified. All packet schedulers that try to do something with bytes rather
than packets, are similarly problematic.

We never figured out how to handle jumbo frames outside the datacenter in the first place.

People doing AQM analysis work work MUST turn off TSO and GSO in order to get a sane result -
and if you merely turn off TSO, GSO will rise in it's stead on some devices, and bite you.

(yes, this happened to me)

Even if the device says it's turning TSO off (because it's running at sub 100Mbit speeds)

(this was on the e1000e driver, at least, while I was manually setting it below 100Mbit for testing), gso stays on.

The way to turn off both is:

ethtool -K tso off
ethtool -K gso off

commit 212b573f5552c60265da721ff9ce32e3462a2cdd
Author: Michał Mirosław <mirq-linux@rere.qmqm.pl>
Date: Tue Feb 15 16:59:16 2011 +0000

ethtool: enable GSO and GRO by default

Signed-off-by: Michał Mirosław <mirq-linux@rere.qmqm.pl>
    Signed-off-by: David S. Miller <davem@davemloft.net>

If it were up to me, I'd revert half this commit, with a message like:

"Disable GSO by default"

Disable GSO until the effects of unleashing superpackets on the global internet
as a whole is better understood by theorists and network designers, and
the Linux scheduler is revised to handle them properly.