LAG Configuration: which hash mode to use for linux bond.

[robfantini]

Hello   I've got lAG working with a few nodes in our proxmox cluster.

/etc/network/interfaces  use this per proxmox wiki suggestion:

auto bond0
iface bond0 inet manual
       slaves eth0 eth1
       bond_miimon 100
       bond_mode 802.3ad
       bond_xmit_hash_policy layer2+3


auto vmbr0
iface vmbr0 inet static
        address 10.1.10.8
        netmask 255.255.255.0
        gateway 10.1.10.1
        bridge_ports bond0
        bridge_stp off
        bridge_fd 0

My question is which  'hash mode' should I use on the switch.

Currently this is used , i think it is the default:   3 Src/Dest MAC, VLAN, EType, incoming port

[LaurentMa · Accepted solution]

Hi robfantini

You are using a NETGEAR Fully Managed Switch since you see default LAG hashmode 3 Src/Dest MAC, VLAN, EType, incoming port.

This is good news, Fully Managed Switches offer seven different, selectable hashmodes with Layer 2 (MAC), Layer 3 (IP), Layer 4 (TCP/UDP) source and destination parsing combinations, for maximum interop and performance.

Smart Managed Switches instead offer fixed Layer 2 (MAC) destination parsing only for packets entering the LAG.

You show bond_xmit_hash_policy layer2+3 in your proxmox cluster configuration file for bond0 LAG interface. I am almost sure it is LACP based and using source and destination hashing algorithm, so based on MAC and IP sources and destinations.  

First, please make sure your LAG is using LACP, so with Static Mode = Disabled (it is by default Disabled on Fully Managed Switches, so LACP is used by defaut). But please check.

Second, with current proxmox configuration, you should stay with default 3 Src/Dest MAC, VLAN, EType, incoming port algorithm, it will work OK and this is our most "compatible" mode.

If your proxmox can support bond_xmit_hash_policy layer3+4 instead, it would be even better with our switch LAG configured with 6 Src/Dest IP and TCP/UDP Port fields algorithm instead. This would bring to even better performance for hashing, and maximum interoperability this time.

Let us know how it goes!

Regards,

[LaurentMa · Accepted solution]

Hi robfantini

You are using a NETGEAR Fully Managed Switch since you see default LAG hashmode 3 Src/Dest MAC, VLAN, EType, incoming port.

This is good news, Fully Managed Switches offer seven different, selectable hashmodes with Layer 2 (MAC), Layer 3 (IP), Layer 4 (TCP/UDP) source and destination parsing combinations, for maximum interop and performance.

Smart Managed Switches instead offer fixed Layer 2 (MAC) destination parsing only for packets entering the LAG.

You show bond_xmit_hash_policy layer2+3 in your proxmox cluster configuration file for bond0 LAG interface. I am almost sure it is LACP based and using source and destination hashing algorithm, so based on MAC and IP sources and destinations.  

First, please make sure your LAG is using LACP, so with Static Mode = Disabled (it is by default Disabled on Fully Managed Switches, so LACP is used by defaut). But please check.

Second, with current proxmox configuration, you should stay with default 3 Src/Dest MAC, VLAN, EType, incoming port algorithm, it will work OK and this is our most "compatible" mode.

If your proxmox can support bond_xmit_hash_policy layer3+4 instead, it would be even better with our switch LAG configured with 6 Src/Dest IP and TCP/UDP Port fields algorithm instead. This would bring to even better performance for hashing, and maximum interoperability this time.

Let us know how it goes!

Regards,

[robfantini]

thank you for the fast response.  It is good to know that the default is OK to use.

'Static Mode'  is disabled on all lags.  that must be the default since I did not adjust.

thanks for your suggestion on layer3+4 .   I'll try using on a node with the suggested settings.

[robfantini]

re: "You show bond_xmit_hash_policy layer2+3 in your proxmox cluster configuration file for bond0 LAG interface. I am almost sure it is LACP based and using source and destination hashing algorithm, so based on MAC and IP sources and destinations. "

you are of course correct about LACP   per  https://www.kernel.org/doc/Documentation/networking/bonding.txt:

"

layer2+3

		This policy uses a combination of layer2 and layer3
		protocol information to generate the hash 
                ..  

		This algorithm is 802.3ad compliant.

::