Bug #301

Need to arrange for AQM testing and bakeoff....

Added by Jim Gettys on Nov 18, 2011. Updated on Dec 5, 2011.
New High Dave Täht


We have SFB; it may be buggy right now.

We hope to see RED light; it’s vaporware.

We think we’ll need some sort of fair queuing as well as AQM; SFQ looks good there…

And QFQ looks even better.


Updated by Dave Täht on Dec 5, 2011.
And it turned out RED was busted. Folding the patch back into cerowrt will not be a problem, but as usual with bugs of this kind, I have to throw out a month’s worth of results.

I am building a version of both net-next for x86_64 and folding in this patch into cerowrt-rc8

commit 1ee5fa1e9970a16036e37c7b9d5ce81c778252fc
Author: Eric Dumazet eric.dumazet@gmail.com
Date: Thu Dec 1 11:06:34 2011 +0000

sch_red: fix red_change

Le mercredi 30 novembre 2011 à 14:36 -0800, Stephen Hemminger a écrit :

(Almost) nobody uses RED because they can’t figure it out.
According to Wikipedia, VJ says that:
“there are not one, but two bugs in classic RED.”

RED is useful for high throughput routers, I doubt many linux machines
act as such devices.

I was considering adding Adaptative RED (Sally Floyd, Ramakrishna
Gummadi, Scott Shender), August 2001

In this version, maxp is dynamic (from 1% to 50%), and user only have to
setup min_th (target average queue size)
(max_th and wq (burst in linux RED) are automatically setup)

By the way it seems we have a small bug in red_change()

if (skb_queue_empty(&sch->q))

First, if queue is empty, we should call

Second, since we dont use anymore sch->q, but q->qdisc, the test is

Oh well…

[PATCH] sch_red: fix red_change()

Now RED is classful, we must check q->qdisc->q.qlen, and if queue is empty,
we start an idle period, not end it.

Signed-off-by: Eric Dumazet eric.dumazet@gmail.com
Signed-off-by: David S. Miller davem@davemloft.net

Updated by Dave Täht on Dec 5, 2011.
Also a major problem is that the current linux packet schedulers are scheduling ‘superpackets’ - full GSO and TSO offloads - up to 64k- through the scheduler.

The effect this has on any attempt to classify or schedule packets has to be seen to be believed.
A 64*k* ‘packet’, followed by a 64 byte packet, is a 1000x1 ratio. Mere packet oriented schedulers, like PFIFO_fast, survive this abuse, but the effect on packets marked background
or priority, over best effort, is correspondingly, um, ‘enhanced’.

Spitting this sort of abuse through a grouping scheduler such as SFB or SFQ is really bad, as the above effect is magnified. All packet schedulers that try to do something with bytes rather
than packets, are similarly problematic.

We never figured out how to handle jumbo frames outside the datacenter in the first place.

People doing AQM analysis work work MUST turn off TSO and GSO in order to get a sane result -
and if you merely turn off TSO, GSO will rise in it’s stead on some devices, and bite you.

(yes, this happened to me)

Even if the device says it’s turning TSO off (because it’s running at sub 100Mbit speeds)

(this was on the e1000e driver, at least, while I was manually setting it below 100Mbit for testing), gso stays on.

The way to turn off both is:

ethtool -K tso off
ethtool -K gso off

commit 212b573f5552c60265da721ff9ce32e3462a2cdd
Author: Michał Mirosław mirq-linux@rere.qmqm.pl
Date: Tue Feb 15 16:59:16 2011 +0000

ethtool: enable GSO and GRO by default

Signed-off-by: Michał Mirosław mirq-linux@rere.qmqm.pl
Signed-off-by: David S. Miller davem@davemloft.net

If it were up to me, I’d revert half this commit, with a message like:

“Disable GSO by default”

Disable GSO until the effects of unleashing superpackets on the global internet
as a whole is better understood by theorists and network designers, and
the Linux scheduler is revised to handle them properly.

This is a static export of the original bufferbloat.net issue database. As such, no further commenting is possible; the information is solely here for archival purposes.
RSS feed

Recent Updates

Apr 12, 2024 Wiki page
What Can I Do About Bufferbloat?
Dec 3, 2022 Wiki page
Codel Wiki
Jun 11, 2022 Wiki page
More about Bufferbloat
Jun 11, 2022 Wiki page
Tests for Bufferbloat
Dec 7, 2021 Wiki page
Getting SQM Running Right

Find us elsewhere

Bufferbloat Mailing Lists
#bufferbloat on Twitter
Google+ group
Archived Bufferbloat pages from the Wayback Machine


Comcast Research Innovation Fund
Nlnet Foundation
Shuttleworth Foundation

Bufferbloat Related Projects

OpenWrt Project
Congestion Control Blog
Flent Network Test Suite
The Cake shaper
CeroWrt (where it all started)

Network Performance Related Resources

Jim Gettys' Blog - The chairman of the Fjord
Toke's Blog - Karlstad University's work on bloat
Voip Users Conference - Weekly Videoconference mostly about voip
Candelatech - A wifi testing company that "gets it".