XCP-NG LACP bonding - 4Gb/s with Cisco 3750X
As I wrote some time ago here, I've got new server to host all my internal servers and services, especially, home automation, lots of monitoring tools, space for files, vpn tunnels and many more.
I have choosen this server which have quad gigabit NICs (and dedicated IPMI port which leaves all NIC for production - no shared management)
Normally I would make active-backup bonding on 2 NIC because this works out of the box. This can be done even on running already XCP. Simple You go to New -> Network in XenOrchestra, select pool which you want to configure. Then enable "Bonded network", assign two or more NICs, and select active backup from "Bond mode". Click Create Network, and voila. If you have any machine or XCP itself on one of those physical cards, it will be not available for few seconds then will be back online and in Network tab you will see not eth0 or eth1 but bond0 interface.
With LACP bonding which is providing failover if one NIC fails, you have also aggregation of links. So making bonding of 4 NIC with 1 Gb/s each, gives virtual link with 4Gb/s speed and also balances load over all bonded cards. This however requires additional setup on switch. I'm using Cisco 3750X layer-3 switch.
First configure bond of 2 or more NIC you have in XenOrchestra as described above, but choose lacp in "Bond mode", This will create bond0 interface on which you can create another networks with vlans, or use it as main network. All NICs assigned becomes "Disconnected" which is normal. Here is how it should look like:
Now you have to configure Port-channel on switch. I'm using vlans so my portchannel is in mode trunk which passess all vlans, however if you are using only one vlan (no vlan / default vlan no. 1), you can use mode access.
interface Port-channel1
switchport trunk encapsulation dot1q
switchport mode trunk
spanning-tree bpduguard enableIf you cant do switchport mode trunk on freash portchannel because of error that says: Command rejected: An interface whose trunk encapsulation is "Auto" can not be configured to "trunk" mode. You have to tell switch which encapsulation to use. Dot1q usually works great.
So: Type "switchport trunk encapsulation dot1q" to solve this.
Now, on every port on switch which is connected to server you have to make same configuration related to switchport, so if portchannel is trunk, set interfaces to trunk as well. Same additional configuration for encapsulation, and add channel-protocol lacp that will configure interface to use lacp capabilities. Last thing to configure is assigning all physical interfaces on switch to port-channel created earlier, here is my configuration for reference:
interface GigabitEthernet1/0/15
description Supermicro - eth1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 1 mode active
interface GigabitEthernet1/0/17
description Supermicro - eth0
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 1 mode active
interface GigabitEthernet1/0/19
description Supermicro - eth2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 1 mode active
interface GigabitEthernet1/0/21
description Supermicro - eth3
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 1 mode active
Now LACP is configured you can start using network in XCP as normal. If You have some monitroing tools like cacti you can observe traffic on ports on switch, that is well balanced:
