Bug #352

Major differences between ipv6 and ipv4 performance

Added by David Taht on Apr 5, 2012. Updated on Apr 21, 2012.
Closed Normal Felix Fietkau


This is probably related to the unaligned accesses patches for tcp
recv and checksum
that are implemented currently for ipv4 only, but only profiling will tell.

root@remote:/etc/config# iptables -I INPUT -p tcp -m multiport –ports
5000:5010 -j ACCEPT
root@remote:/etc/config# iperf -s
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
[ 4] local port 5001 connected with port 59727
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-60.0 sec 1.86 GBytes 266 Mbits/sec
root@remote:/etc/config# iperf -w256k -V -s
Server listening on TCP port 5001
TCP window size: 512 KByte (WARNING: requested 256 KByte)
[ 4] local 2001:4f8:fff8:500::1 port 5001 connected with
2001:4f8:3:203::14 port 60094
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-60.1 sec 732 MBytes 102 Mbits/sec


Updated by Dave Täht on Apr 6, 2012.
It would be nice if someone that cared more about ipv6 than I do at the moment, and was willing to pay for the work, to get it done, with the person most qualified to do it, who isn’tme.

A factor of 4 disimprovement, and mere 60Mbit performance on ipv6 is not terrible but fixing it will require oprofiling and deeper investigation.

I’ve seen a similar problem on ipv6 tunnels.

Updated by David Taht on Apr 6, 2012.
documenting this publicly

ipv6 runs at less than half the speed as ipv4.

CPU: MIPS 24K, speed 678 MHz (estimated)
Counted INSTRUCTIONS events (1-0 Instructions completed) with a unit
mask of 0x00 (No unit mask) count 100000
samples  %        app name                 symbol name
6703     22.7367  vmlinux                  do_ade
 6703     100.000  vmlinux                  do_ade [self]
3754     12.7336  ip6_tables               /ip6_tables
 3754     100.000  ip6_tables               /ip6_tables [self]
2834      9.6130  ipv6                     /ipv6
 2834     100.000  ipv6                     /ipv6 [self]
2240      7.5981  vmlinux                  __copy_user
 2240     100.000  vmlinux                  __copy_user [self]
2164      7.3403  vmlinux                  csum_partial
 2164     100.000  vmlinux                  csum_partial [self]

m@cruithne:~\$ iperf -V -c fd25:b7ff:b098:8001::1
Client connecting to fd25:b7ff:b098:8001::1, TCP port 5001
TCP window size: 64.6 KByte (default)
[  3] local fd43:e2b3:341c:5071::1 port 35233 connected with
fd25:b7ff:b098:8001::1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   185 MBytes   155 Mbits/sec

Updated by David Taht on Apr 6, 2012.
While this is a better method to track down traps, when you are having
lots of them, the serial port gets overrun completely.

And on ipv6, we get a lot of them

———- Forwarded message ———-
From: Felix Fietkau nbd@openwrt.org
Date: Sat, Feb 18, 2012 at 8:41 AM
Subject: Re: oprofile magic? (ipv6 terrible)
To: Dave Taht dave.taht@gmail.com

On 2012-02-18 5:35 PM, Dave Taht wrote:
> I was curious as to what oprofiling magic tricks you used
> to track down the unaligned exception traps. I haven’t
> touched oprofile since august…
> ipv6 runs at about half the speed as ipv4.
I’m not using oprofile to track down such traps, there’s a much easier
way: echo 2 > /sys/kernel/debug/mips/unaligned_action
After that, the kernel will throw a stack trace for every single
unaligned exception. You can then run gdb to track down the line of code
that triggered the exception, assuming you’ve enabled debug symbols for
the kernel.
Just enter l *+
When it’s trapping based on access to a struct, just mark the struct as
__packed in a patch.

  • Felix
Updated by Dave Täht on Apr 13, 2012.
vs a vs bug #360 we are up from 110Mb/sec to 185Mb/sec

I’m going to dismiss the slight lowering of ipv4 as statistical noise.

d@io:~\$ iperf -t 60 -c
Client connecting to, TCP port 5001
TCP window size: 64.0 KByte (default)
[ 3] local port 35441 connected with port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 1.83 GBytes 262 Mbits/sec

d@io:~\$ iperf -V -t 60 -c 2001:4f8:3:203::217
Client connecting to 2001:4f8:3:203::217, TCP port 5001
TCP window size: 64.0 KByte (default)
[ 3] local 2001:4f8:3:203::14 port 44866 connected with 2001:4f8:3:203::217 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 1.29 GBytes 185 Mbits/sec

Updated by Robert Bradley on Apr 16, 2012.
Based on Dave’s latest comments on bug #360 (http://www.bufferbloat.net/issues/360#note-65), I am tentatively marking this as resolved. Performance of IPv6 is now over double the original measurements, and similar to that of IPv4.
Updated by Dave Täht on Apr 21, 2012.
I’m going to treat the route update bug of #371 as a separate bug.

This bug was too epic in scope. Please do not delete any patches, they are useful to have around as blind alleys.

This is a static export of the original bufferbloat.net issue database. As such, no further commenting is possible; the information is solely here for archival purposes.
RSS feed

Recent News & Articles

Mar 21, 2019 Wiki page
Dave Taht's Take on TCP
Mar 17, 2019 Wiki page
Jake Holland's Stance on ECN
Sep 6, 2018 Wiki page
Pete Heist's Thoughts on ECN
Sep 5, 2018 Wiki page
Dave Taht's Stance on ECN
Sep 4, 2018 Wiki page
Jonathan Morton's Take on ECN

Find us elsewhere

Bufferbloat Mailing Lists
#bufferbloat on Twitter
Google+ group
Archived Bufferbloat pages from the Wayback Machine


Comcast Research Innovation Fund
Nlnet Foundation
Shuttleworth Foundation

Bufferbloat Related Projects

Congestion Control Blog
Lede Project (OpenWrt)
Flent Network Test Suite
The Cake shaper

Network Performance Related Resources

Jim Gettys' Blog - The chairman of the Fjord
Toke's Blog - Karlstad University's work on bloat
Voip Users Conference - Weekly Videoconference mostly about voip
Candelatech - A wifi testing company that "gets it".