#1 What is TCN TCN is a new ECN marking scheme. It can work with any packet scheduling algorithm. Currently, we have implemented TCN on the top of two schedulers: SP/DWRR and SP/WFQ. In both schedulers, we have several priorities. Queues on the same priority are scheduled based on DWRR (Deficit Weighted Round Robin) or WFQ (Weighted Fair Queueing). Queues from different priorities are scheduled based on strict priorities (SP).

#2 How to use ##2.1 Compiling TCN software prototype is implemented as a Linux queuing discipline (qdisc) kernel module, running on multi-NIC servers to emulate switch hardware behaviors. So you need the kernel headers to compile it. You can find available headers on your system in /lib/modules. To install the kernel headers, you just need to use the following command：

$ apt-get install linux-headers-$(uname -r)

To compile SP/DWRR kernel module:

$ cd sch_dwrr
$ make

This will produce a kernel module called sch_dwrr.ko.

To compile SP/WFQ kernel module:

$ cd sch_wfq
$ make

This will produce a kernel module called sch_wfq.ko.

##2.2 Installing TCN replaces token bucket rate limiter module. Hence, you need to remove sch_tbf before installing TCN. To install TCN (e.g., SP/DWRR) on a device (e.g., eth1):

$ rmmod sch_tbf
$ insmod sch_dwrr.ko
$ tc qdisc add dev eth1 root tbf rate 995mbit limit 1000k burst 1000k mtu 66000 peakrate 1000mbit

To remove TCN (SP/DWRR) on a device (e.g., eth1):

$ tc qdisc del dev eth1 root
$ rmmod sch_dwrr

In above example, we install TCN (SP/DWRR) on eth1. The shaping rate is 995Mbps (line rate is 1000Mbps). To accurately reflect switch buffer occupancy, we usually trade a little bandwidth.

##2.3 Note To better emulate real switch hardware behaviors, we should avoid large segments on server-emulated software switches. Hence, we need to disable related offloading techniques on all involved NICs. For example, to disable offloading on eth0:

$ ethtool -K eth0 tso off
$ ethtool -K eth0 gso off
$ ethtool -K eth0 gro off

##2.4 Configuring Except for shaping rate, all the parameters of TCN are configured through sysctl interfaces. Here, I only show several important parameters. For the rest, see params.h and params.c for more details.

• ECN marking scheme for SP/DWRR:

  $ sysctl dwrr.ecn_scheme
  $ sysctl wfq.ecn_scheme
  

  To enable per-queue ECN marking:

  $ sysctl -w dwrr.ecn_scheme=1
  $ sysctl -w wfq.ecn_scheme=1
  

  To enable per-port ECN marking:

  $ sysctl -w dwrr.ecn_scheme=2
  $ sysctl -w wfq.ecn_scheme=2
  

  To enable MQ-ECN (this has no use for SP/WFQ):

  $ sysctl -w dwrr.ecn_scheme=3
  

  To enable TCN:

  $ sysctl -w dwrr.ecn_scheme=4
  $ sysctl -w wfq.ecn_scheme=4
  

  To enable CoDel ECN marking:

  $ sysctl -w dwrr.ecn_scheme=5
  $ sysctl -w wfq.ecn_scheme=5
  

• Per-port ECN marking threshold (byte):

  $ sysctl dwrr.port_thresh
  $ sysctl wfq.port_thresh
  

• Per-queue ECN marking threshold (byte) (i is queue index starting from 0):

  $ sysctl dwrr.queue_thresh_i
  $ sysctl wfq.queue_thresh_i
  

• Per-queue priority (i is queue index starting from 0):

  $ sysctl dwrr.queue_prio_i
  $ sysctl wfq.queue_prio_i
  

  To enable the highest priority (priority 0) for queue 0:

  $ sysctl -w dwrr.queue_prio_0=0
  $ sysctl -w wfq.queue_prio_0=0
  

• Per-port shared buffer size (byte):

  $ sysctl dwrr.shared_buffer
  $ sysctl wfq.shared_buffer
  

• TCN sojourn time threshold (1024 ns):

  $ sysctl dwrr.tcn_thresh
  $ sysctl wfq.tcn_thresh
  

• CoDel target (1024 ns):

  $ sysctl dwrr.codel_target 
  $ sysctl wfq.codel_target 
  

• CoDel interval (1024 ns):

  $ sysctl dwrr.codel_interval
  $ sysctl wfq.codel_interval