The most recent beta-test CeroWrt (version 3.10.48-2) is working very well. The incidence of the wifi-related bug #442 has dropped, and it otherwise has been stable for almost two weeks. Get it from http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.10.48-2/
There is a newer version - 3.10.50-1 - that has not been well tested, but it incorporates another wifi bug-fix. If it checks out, this may be close to a release candidate.
In response to the heartbleed (CVE-2014-0160) vulnerability, on April
9th 2014 we updated the under-development CeroWrt release to include
the fixed version of openssl. The fix is in CeroWrt 3.10.36-3 and
We have no means of fixing the "stable" (3.7.5) release of CeroWrt,
nor any of the innumerable development releases since then.
Please do a clean, fresh upgrade to CeroWrt 3.10.36-6 or later. 
Images are available in: http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/
Reflashing instructions are here:
In the base image, the administration gui of recent CeroWrt versions
depended on openssl (however it is protected by firewall rules to only
be accessible from within your own network), and several optional packages
did also - stunnel - used for "secure" tunneling, and openvpn in particular.
Heartbleed is one of the most serious bugs that has ever hit the
internet, and in addition to web services, critical network daemons
such as those that manage network printing, logging, monitoring, voip,
chat, tunnels, vpns and email, all can potentially be exploited.
We strongly advise resyncing your source trees with us and distributing
new firmware images containing the updated libraries. All
network facing TLS-using daemons are potentially a risk, as are
any TLS using services exposed behind the firewall.
Once your system is secured again, you should re-issue certs and passwords,
as per: https://www.eff.org/deeplinks/2014/04/bleeding-hearts-club-heartbleed-recovery-system-administrators
and check for unverified commits.
Packages maintained in the openwrt core repositories that can be
affected when compiled for openssl2 may include: libevent2,
ustream-ssl, hostapd, openvpn, authsae, luci-ssl, and uhttpd.
Optional network daemons in other repositories such as radsecproxy,
vsftpd, squid, mini_httpd, pure-ftpd, cups, ndyndns, elinks,
libtorrent, monit, nagios, syslog-ng3, boxbackup, rsyncrypto, curl,
cyrus-sasl, openldap, icecast, fetchmail, dovecot, transmission,
stunnel, httptunnel, apache, lighttpd, znc, net-snmp, bitlbee,
asterisk, postfix and openvpn all use TLS level security, are
often linked against openssl, and are thus potentially vulnerable.
Please see the relevant website for each of the products above
for news on their vulnerabilities. Much of the furor over heartbleed
has focused on websites, where notably smtp and imaps and im traffic
has also been shown vulnerable.
Other infrastructure, router and CPE distributions are also affected.
Two examples among many:
Network facing Applications built on top of php4, php5, python, luasec, erlang, ruby
are also potentially affected.
Packages maintained in the ceropackages repository that were potentially
vulnerable are xorp, python-lafs, ccnx, and resiprocate.
Please take this seriously and check your firmware and your products for
usage of the vulnerable openssl versions.
We note also that multiple other serious vulnerabilities have been
fixed in other CeroWrt and OpenWrt packages and in the Linux kernel over
the past years; you should consider fixing those vulnerabilities in
your downstream products and routers while you are at it.
We have long been supportive of adding new features for openwrt to
make it more easily updated in the field, the work could use more
eyeballs and developers, and we need to find resources and funding for
a code audit in the coming months.
 Regrettably in the present development branch (3.10.36-4) we are
trying to isolate a wifi bug that crops up after much traffic, we will
announce a fix for that when it arrives. See Bug #442 .
 The base as-provided-by OpenWrt base binary installations are not
vulnerable to HeartBleed, as neither the builtin SSH server nor the
optional LuCI SSL support rely on OpenSSL for cryptographic TLS
support. Their Attitude Adjustment release used cyassl as a base,
and the underway Barrier Breaker development series uses PolarSSL
for as many packages as support exists and the GPLv2 license allows.
In other words the OpenSSL library is not installed within the stock
base images available on their download servers, however they too
contain many optional packages that do depend on openssl to function,
and many downstream products may have chosen openssl over those
Check your trees! And if you are having a bad week, perhaps this
will help: http://www.taht.net/~mtaht/uncle_bills_helicopter.html
Stay calm and keep on patching!
Jim Gettys will be giving a talk at MIT about insecurity
in home devices and what can be done about it.
For more details, see:
The newest build of CeroWrt - 3.10.24-8 - is working quite well for us. There's a Release Note page that gives the current status. Note that, although it has been very stable, this is still an experimental release. It's available at http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.10.24-8/
The CeroWrt development team has been working to nail down a no-brainer set of instructions for eliminating bufferbloat - the lag/latency that kills voice & video chat, gaming, and overall network responsiveness. The hard part is that optimal configuration of the Smart Queue Management (SQM) link is difficult - there are tons of options an ISP can set. Although CeroWrt can adapt to any of them, it's difficult to find out the exact characteristics of the link you have. Check out the latest version of our instructions at http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_AQM_for_CeroWrt_310
Although the CeroWrt site (http://www.bufferbloat.net/projects/cerowrt/) has been quiet, that doesn't mean that we haven't been working.
The CeroWrt-Devel mailing list (https://lists.bufferbloat.net/listinfo/cerowrt-devel) has been bubbling with lots of energy all summer and fall, and we're getting close to a new release that we can recommend to everyone. Here's what's we've been working on:
- Significant refinement to the CoDel code, to further knock down bufferbloat
- Update to Linux 3.10 kernel that incorporates much of our earlier work on bufferbloat
- All the good effort from the OpenWrt Barrier Breaker development firmware
- And too many other improvements to mention here
You can check out any number of Bufferbloat videos at http://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos to learn more about the problem and our solutions.
If you just want good router firmware, purchase a Netgear WNDR3800 and burn the current 3.7.5-2 firmware. You can also follow our activity on the CeroWrt-devel list to see what's happening and learn when experimental builds or solid new releases are available.
What can I say? 3.7.5-2 has been deployed and thoroughly tested at multiple sites, and under high load, and on places like comcast's cable network, and it just. keeps. working.
In this release:
Vastly improved ipv6 naming and interopability, 3 new forms of codel and fq_codel, much better QoS...
And all the chocolately goodness from OpenWrt Barrier Breaker.
More news to come. Thank you all for your support and donations! We couldn't have done it without you.
I've been working fixing bufferbloat now for a really long time, and of late it's been really difficult to keep the lights on and the servers fed. Last month I'd hit bottom, this month is worse. I put up a donations page, and got a totally wonderful level of response - enough to make rent! Then I tripped over an obscure portion of the Amazon EULA, and ended up refunding everyone's money and cancelling your subscriptions.
So, I've put up a new subscribe/donations page for the cerowrt portion of the project, using paypal at:
I won't jimmy wales you'all on further solicitations like this. (I know, that's what I said last month)
But a little more help in getting over this hump would be very nice to have.
Cerowrt-3.3.8-10 is stable but forward-looking. It has an outline
towards what a more wifi-bloat-free future would look like. Maybe.
For more details, see the announcement on the mailing list
I just put out http://huchra.bufferbloat.net/~cero1/3.3/3.3.8-6/ and deployed it as my default gw and ran a bunch of tests that it survived.
This is a version after 5 development releases and I'm hoping it proves out stable enough for more deploy.
I'd prefer to test 24 hours but I'm about to start a trip and can't do that. Hopefully after some more testers leap on it we can declare it stable later this week and move on...
Also the source tree is mostly pushed out but a bit of a mess, I don't know if I'll be able to get Cero independently buildable before late next week.
+ update to linux 3.3.8
+ Fix for bind9 CVE
+ switch to netifd
+ a complete resync with openwrt - this includes much new stuff,
including wireless-testing - way too many updates to talk about
without pulling in the commit log
+ memory problem with ath9k appears gone
+ ECN dropping instituted under load
+ fq_codel packet limits
+ There is now 6rd support, totally untested and unconfigured;
+ transmission bittorrent is in there, too
+ fq_codel on all interfaces by default, on wireless using all 4 subqueues
-s on this release: I went for "stable" rather than new features after it cost me too much time.
- I had to rip out opkg signing support, and some ipv6/diffserv classification support in transmission that wasn't fully baked.
- re-running simple_qos.sh with new values appears to require a reboot first
-The default gui for AQM doesn't work, the one for "qos" uses hfsc + fq_codel (but lacks ipv6 and diffserv support), and the command line simple_qos.sh has ipv6 and diffserv, but has to be edited and run manually. And perhaps it's use of htb etc can be improved. I get pretty good results on comcast with simple_qos, see speedtest results here:
but not quite as good as I hoped for. However, under heavier loads the fq_codel stuff is working great under netperf with various numbers of threads and classifications and users.
I would hope some folk here run some benchmarks against various things but some cautions - for example - chrome's benchmark tends to hit dns
hard, and cero by default is not using your most local forwarder so it can bottleneck on dns - ways to fix that if you have dnssec is to edit
forwarders.conf to point to your local forwarder, and uncomment the forwarders line in named.conf. If your ISP doesn't
do dnssec yet, disable dnssec and point forwarders.conf to their nameservers - but I otherwise am getting good results.
Also: I would really prefer people clearly identify when they are testing over wireless vs ethernet and until you have a fq_codel and
debloat-script enabled kernel on your laptop, too, I am finding most of the time the bloat is coming from the testing box rather than cerowrt
There are now fq_codel enabled kernels for ubuntu 12.4 and fedora 16 available here:
I look forward to analyzing htb vs hfsc and further tuning of qos-scripts and the simple_qos script. I'm too stupid apparently to
come up with a way to run simple_qos out of the aqm gui... (help wanted)
The new version of quagga-babeld is available in the opkg repository and it has been confirmed to work right with ipv4 mesh interfaces. I
am really looking forward to people trying this and the authentication code now in quagga so we can migrate off of the existing babeld.
Have fun. I am traveling the rest of this week. Patches, benchmarks and data gladly accepted (preferably on the cerowrt-devel list)
A test release of CeroWrt is now available that has support for Kathie Nichols' and Van Jacobson's new AQM, Codel , and Eric Dumazet's new fair queuing implementation on top of that, fq_codel.
fq_codel is enabled on all interfaces by default. It is vastly simpler than what we were using before (sfqred) and draws upon and improves on the same body of ideas (head drop, fq, timestamping) but now tied to Kathie and Van's blinding insights as to a good drop strategy, and Eric's successor ideas as towards head of queue behavior and cache line optimizations.
There is a simple_qos.sh script that can be set to your uplink and downlink speeds, but no uci interface for it as yet, nor gui. (help on finishing aqm-scripts and the luci interface gladly accepted)
To see all the chocolately goodness of what fq_codel can do to wired and wireless latency, it would be good for more to play with it.
Benchmarks have been very good thus far, and more benchmarks and analysis are highly desired.
This release suffers from an unrelated bug ( #379 ) and should NOT be installed as your main router. I would love to beat this bug because it's the only prio 1 remaining but thus far, no luck. Under lighter loads CeroWrt appears to work just fine, but that's for me. YMMV.
Get it here: http://huchra.bufferbloat.net/~cero1/3.3/3.3.6-2/
Also available in: Atom