Ethernet bonding with Linux and 802.3ad

2 NIC + SwitchNowadays, most desktop mainboards provide more than one gigabit ethernet port. Connecting them both to the same switch causes most Linux distros by default to get a individual IP on each device and route traffic only on the primary device (based on device metric) or round-robin. A single connection always starts at one IP and so all traffic goes through one device, limiting maximum bandwidth to 1 GBit.

Here comes bonding (sometimes called (port) trunking or link aggregation) to play. It connects two ore more ethernet ports to one virtual port with only one MAC and so mostly one IP address. Wheres earlier only two hosts (with the same OS running) or two switches (from the same vendor) could be connected, nowadays there's a standard protocol which makes it easy: LACP which is part of IEEE 802.3ad. Linux supports difference bonding mechanisms including 802.3ad. To enable bonding at all there are some kernel settings needed:

Device Drivers  --->
[*] Network device support  --->
<*>   Bonding driver support

After compiling and rebooting, we need a userspace tool for configuring the virtual interface. It's called ifenslave and provided with the Linux kernel. You can either compile it by hand

/usr/src/linux/Documentation/networking
gcc -Wall -O -I/usr/src/linux/include ifenslave.c -o ifenslave
cp ifenslave /sbin/ifenslave

or install it by emerge if you run Gentoo Linux:

emerge -va ifenslave

Now we can configure the bonding device, called bond0. Firstofall we need to set the 802.3ad mode and the MII link monitoring frequency by

echo "802.3ad" > /sys/class/net/bond0/bonding/mode
echo 100 >/sys/class/net/bond0/bonding/miimon

Now we can up the device and add some ethernet ports:

ifconfig bond0 up
ifenslave bond0 eth0
ifenslave bond0 eth1

Now bond0 is ready to be used. Run a dhcp client or set an IP by

ifconfig bond0 192.168.1.2 netmask 255.255.255.0

These steps are needed on each reboot. If you're running gentoo, you can use baselayout for this. Add

config_eth0=( "none" )
config_eth1=( "none" )
preup() {
	# Adjusting the bonding mode / MII monitor
	# Possible modes are : 0, 1, 2, 3, 4, 5, 6,
	#     OR
	#   balance-rr, active-backup, balance-xor, broadcast,
	#   802.3ad, balance-tlb, balance-alb
	# MII monitor time interval typically: 100 milliseconds
	if [[ ${IFACE} == "bond0" ]] ; then
		BOND_MODE="802.3ad"
		BOND_MIIMON="100"
		echo ${BOND_MODE} >/sys/class/net/bond0/bonding/mode
		echo ${BOND_MIIMON}  >/sys/class/net/bond0/bonding/miimon
		einfo "Bonding mode is set to ${BOND_MODE} on ${IFACE}"
		einfo "MII monitor interval is set to ${BOND_MIIMON} ms on ${IFACE}"
	else
		einfo "Doing nothing on ${IFACE}"
	fi
	return 0
}
slaves_bond0="eth0 eth1"
config_bond0=( "dhcp" )

to your /etc/conf.d/net. I found this nice preup part in the Gentoo Wiki Archive.

Now you have to configure the other side of the link. You can either use a Linux box and configure it the same way or a 802.3ad-capable switch. I used an HP Procurve 1800-24G switch. You have to enable LACP on the ports you're connected:

HP Procurve 1800-24G LACP Configuration

HP Procurve 1800-24G LACP Configuration

Now everything should work and you can enjoy a 2 GBits (or more) link. Further details can be found in the kernel documentation.

8 thoughts on “Ethernet bonding with Linux and 802.3ad

  1. for baselayout-2 is no need the preup. For ajusting the bondig just put in the net file the settings in the var_iface="setting" format:
    ex: mode_bond0="802.3ad"

  2. What do you mean you have to configure the other side of the link on LACP ports? Shouldn't it be that you just bond the interfaces on the Gentoox box and you have 2Gbit virtual ethernet ready to take 2Gbits traffic?

    • The switch needs also to setup the port trunkation. As it is disabled on most switches by default, you have to enable it. Needless to say that you must have a switch which supports 802.3ad!

  3. Hi,

    Can we have odd number (more than 1 like 3,5 etc) of interfaces as part of 802.3ad bonding algorithm or do we need even numbers?

    Appreciate your help.

    Thank you
    Amit

  4. Well, you should also add, that Linux lacp implementation is incomplete. It is not able to do active LACP, which is a real pain ...

  5. Since you are using sysfs to get stuff done (like set mode), you might as well use it for everything and forgo ifenslave entirely. Infact the latest documentation states ifenslave is obsolete. You can keep the setup two lines as is -

    echo "802.3ad" > /sys/class/net/bond0/bonding/mode
    echo 100 >/sys/class/net/bond0/bonding/miimon

    But when adding ethernet ports, it can be done as follows -

    echo +eth0 > /sys/class/net/bond0/bonding/slaves
    echo +eth1 > /sys/class/net/bond0/bonding/slaves

    You can then continue with the rest as per usual, bringing up bond0 as mentioned in your article -

    ifconfig bond0 up

    Hope this helps.

Leave a Reply

Your email address will not be published. Required fields are marked *