Bloat » Cerowrt

I was going to reboot you into a txqueuelen of 256, but I merely left it setup so if you rebooted the ethernet drivers
would have a txqueuelen of 256.

I enabled qos-scripts, setting it to 2000 bidirectionally, as the max as I could get on a single stream was ~8000

the 'overhead' value is for things like ethernet over something. Left it off.

root@europa:~# iperf -t 60 -w220k -c 98.229.130.122 
------------------------------------------------------------
Client connecting to 98.229.130.122, TCP port 5001
TCP window size:  220 KByte
------------------------------------------------------------
[  3] local 149.20.63.19 port 49117 connected with 98.229.130.122 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.7 sec  9.50 MBytes  1.31 Mbits/sec

and as usual red does nothing inside of qos scripts, the outer qdiscs do all the work.

root@gw:/etc/config# tc -s qdisc
qdisc pfifo_fast 0: dev se00 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 2470004177 bytes 1906833 pkt (dropped 39, overlimits 0 requeues 588) 
 backlog 0b 0p requeues 588 
qdisc hfsc 1: dev ge00 root refcnt 2 default 30 
 Sent 671105 bytes 9377 pkt (dropped 0, overlimits 117 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 100: dev ge00 parent 1:10 limit 127p quantum 1514b divisor 1024 perturb 2sec 
 Sent 344884 bytes 4956 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 200: dev ge00 parent 1:20 limit 127p quantum 1514b divisor 1024 perturb 2sec 
 Sent 45722 bytes 512 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc red 300: dev ge00 parent 1:30 limit 150Kb min 12800b max 38400b ecn 
 Sent 89683 bytes 372 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  marked 0 early 0 pdrop 0 other 0
qdisc red 400: dev ge00 parent 1:40 limit 150Kb min 12800b max 38400b ecn 
 Sent 190816 bytes 3537 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  marked 0 early 0 pdrop 0 other 0
qdisc ingress ffff: dev ge00 parent ffff:fff1 ---------------- 
 Sent 22572047 bytes 16460 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30 
 Sent 22351988 bytes 15603 pkt (dropped 291, overlimits 29212 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 100: dev ifb0 parent 1:10 limit 127p quantum 1514b divisor 1024 perturb 2sec 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 200: dev ifb0 parent 1:20 limit 127p quantum 1514b divisor 1024 perturb 2sec 
 Sent 181753 bytes 490 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc red 300: dev ifb0 parent 1:30 limit 150Kb min 12800b max 38400b ecn 
 Sent 22170235 bytes 15113 pkt (dropped 291, overlimits 408 requeues 0) 
 backlog 0b 0p requeues 0 
  marked 117 early 291 pdrop 0 other 0
qdisc red 400: dev ifb0 parent 1:40 limit 150Kb min 12800b max 38400b ecn 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  marked 0 early 0 pdrop 0 other 0
qdisc mq 0: dev sw00 root 
 Sent 1980985 bytes 22216 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev gw01 root 
 Sent 793308 bytes 7866 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev mon.sw00 root 
 Sent 1387796 bytes 7937 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev gw00 root 
 Sent 1433275 bytes 9257 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev sw10 root 
 Sent 11511732 bytes 149188 pkt (dropped 0, overlimits 0 requeues 23) 
 backlog 0b 0p requeues 23 
qdisc mq 0: dev gw11 root 
 Sent 792407 bytes 7852 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev mon.sw10 root 
 Sent 22944 bytes 142 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc mq 0: dev gw10 root 
 Sent 1170022 bytes 7800 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 

d@bob-desktop:~$ iperf t 60 -w220k -c 98.229.130.122
-----------------------------------------------------------
Client connecting to 98.229.130.122, TCP port 5001
TCP window size: 256 KByte (WARNING: requested 220 KByte)
------------------------------------------------------------
[ 3] local 149.20.63.20 port 56628 connected with 98.229.130.122 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 13.9 MBytes 1.94 Mbits/sec

The last result was interesting in context, so I setup the cerowrt router (europa) to have a txqueuelen of 1000 for iperf and repeated the test...

then dropped it back to 8, to minimize the skew inflicted by back to back tests.

root@europa:~# ifconfig ge00 txqueuelen 1000
root@europa:~# iperf -t 60 -w220k -c 98.229.130.122 
------------------------------------------------------------
Client connecting to 98.229.130.122, TCP port 5001
TCP window size:  220 KByte
------------------------------------------------------------
[  3] local 149.20.63.19 port 49118 connected with 98.229.130.122 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  13.6 MBytes  1.90 Mbits/sec
root@europa:~# ifconfig ge00 txqueuelen 8
root@europa:~# iperf -t 60 -w220k -c 98.229.130.122 
------------------------------------------------------------
Client connecting to 98.229.130.122, TCP port 5001
TCP window size:  220 KByte
------------------------------------------------------------
[  3] local 149.20.63.19 port 49119 connected with 98.229.130.122 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.6 sec  13.8 MBytes  1.90 Mbits/sec