× NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Orbi WiFi 7 RBE973
Reply

Re: Linux Link Aggregation using 802.3ad

Ryushin
Aspirant

Linux Link Aggregation using 802.3ad

I've been at this for more then a day now.  I'm trying to set up 802.3ad link aggregation with a Debian Jessie system.

I have two Intel 10-Gigabit X540-AT2 that I've configured for 802.3ad to combine the four ports together.

 

Linux Side:

 

/etc/network/interfaces:

auto bond_storage
iface bond_storage inet manual
bond_slaves eth4 eth5 eth6 eth7
bond_miimon 100
bond_mode 802.3ad
bond-lacp-rate fast
bond_xmit_hash_policy layer2+3
bond_downdelay 200
bond_updelay 200

 

cat /proc/net/bonding/bond_storage:

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: down
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 0c:c4:7a:34:a3:2c
bond bond_storage has no active aggregator

Slave Interface: eth4
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 0c:c4:7a:34:a3:2c
Slave queue ID: 0
Aggregator ID: N/A

Slave Interface: eth5
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 0c:c4:7a:34:a3:2d
Slave queue ID: 0
Aggregator ID: N/A

Slave Interface: eth6
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:12:17:90
Slave queue ID: 0
Aggregator ID: N/A

Slave Interface: eth7
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:12:17:92
Slave queue ID: 0
Aggregator ID: N/A

 

Contents of /var/log/kern.log:

Dec 16 14:33:33 fatman kernel: [4604427.914455] bond_storage: Setting xmit hash policy to layer2+3 (2)
Dec 16 14:33:33 fatman kernel: [4604427.915885] bond_storage: Setting MII monitoring interval to 100
Dec 16 14:33:33 fatman kernel: [4604427.917102] bond_storage: Note: Updating updelay (to 200) since it is a multiple of the miimon value
Dec 16 14:33:33 fatman kernel: [4604427.918273] bond_storage: Note: Updating downdelay (to 200) since it is a multiple of the miimon value
Dec 16 14:33:33 fatman kernel: [4604427.919464] bond_storage: Setting down delay to 200
Dec 16 14:33:33 fatman kernel: [4604427.920641] bond_storage: Setting up delay to 200
Dec 16 14:33:33 fatman kernel: [4604427.924971] bond_storage: Setting LACP rate to fast (1)
Dec 16 14:33:33 fatman kernel: [4604427.929008] bond_storage: Adding slave eth4
Dec 16 14:33:33 fatman kernel: [4604428.034981] pps pps0: new PPS source ptp0
Dec 16 14:33:33 fatman kernel: [4604428.036174] ixgbe 0000:09:00.0: registered PHC device on eth4
Dec 16 14:33:34 fatman kernel: [4604428.158360] bond_storage: Enslaving eth4 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.162256] bond_storage: Adding slave eth5
Dec 16 14:33:34 fatman kernel: [4604428.263528] pps pps1: new PPS source ptp1
Dec 16 14:33:34 fatman kernel: [4604428.263531] ixgbe 0000:09:00.1: registered PHC device on eth5
Dec 16 14:33:34 fatman kernel: [4604428.385639] bond_storage: Enslaving eth5 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.389529] bond_storage: Adding slave eth6
Dec 16 14:33:34 fatman kernel: [4604428.489314] pps pps2: new PPS source ptp2
Dec 16 14:33:34 fatman kernel: [4604428.489318] ixgbe 0000:85:00.0: registered PHC device on eth6
Dec 16 14:33:34 fatman kernel: [4604428.609600] bond_storage: Enslaving eth6 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.613185] bond_storage: Adding slave eth7
Dec 16 14:33:34 fatman kernel: [4604428.715415] pps pps3: new PPS source ptp3
Dec 16 14:33:34 fatman kernel: [4604428.715418] ixgbe 0000:85:00.1: registered PHC device on eth7
Dec 16 14:33:34 fatman kernel: [4604428.835904] bond_storage: Enslaving eth7 as a backup interface with a down link
Dec 16 14:33:38 fatman kernel: [4604433.075261] ixgbe 0000:85:00.0 eth6: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:38 fatman kernel: [4604433.139097] ixgbe 0000:85:00.0 eth6: NIC Link is Down
Dec 16 14:33:39 fatman kernel: [4604433.254553] ixgbe 0000:85:00.1 eth7: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:39 fatman kernel: [4604434.083028] ixgbe 0000:09:00.0 eth4: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:40 fatman kernel: [4604434.815192] ixgbe 0000:85:00.0 eth6: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:46 fatman kernel: [4604441.034799] ixgbe 0000:09:00.1 eth5: NIC Link is Up 10 Gbps, Flow Control: None

 

 

The switch shows the LAG state as down. 

I've attached pictures of the Switch Config.

LAG_Configuration.pngLAG_Membership.pngPort_Configuration.pngVLAN1_Membership.pngVLAN1000_Membership.pngVLAN_Status.png

Model: M7100-24X (XSM7224)|ProSAFE 24-port 10G Fully Managed L2 Switch
Message 1 of 5

Accepted Solutions
Ryushin
Aspirant

Re: Linux Link Aggregation using 802.3ad

I rebooted the server and now MII Status show as UP.  So good news there.  Not used to having to reboot a linux server to fix something.

 

I rebooted without a separate VLAN for the LAG ports.  The LAG and the LAG ports are in the same VLAN.  I can ping the server so I think it is working.  Do the LAG ports need to be in their own separate VLAN or should the ports and the LAG be in the same VLAN?

 

View solution in original post

Message 2 of 5

All Replies
Ryushin
Aspirant

Re: Linux Link Aggregation using 802.3ad

I rebooted the server and now MII Status show as UP.  So good news there.  Not used to having to reboot a linux server to fix something.

 

I rebooted without a separate VLAN for the LAG ports.  The LAG and the LAG ports are in the same VLAN.  I can ping the server so I think it is working.  Do the LAG ports need to be in their own separate VLAN or should the ports and the LAG be in the same VLAN?

 

Message 2 of 5
DaneA
NETGEAR Employee Retired

Re: Linux Link Aggregation using 802.3ad

Hi @Ryushin,

 

Welcome to the community! 🙂 

 

It seemed that the rebooting the server helped applied the LAG settings configured.  About having the LAG ports (where the server is connected) be in the same or separate VLAN, it would be up for you to decided if whether you need the server be accessible by other users within the same VLAN or not.  If ever you need the server be within the same VLAN having limited access to a number of users, you will need to configure ACLs (Access Control Lists).  Let me share the article below as reference:

 

How do I set up an IP Access Control List (ACL) with two rules using the web interface on my managed...

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 3 of 5
Ryushin
Aspirant

Re: Linux Link Aggregation using 802.3ad

Our end goal is to bond the 4 ports together for access for rest of the LAN.  I did not know if the four individual ports in the LAG need to be in their own VLAN-1000 and the LAG itself needs to be in the VLAN-1.  Right now the individual ports are in VLAN-1 along with the LAG and everything appears to be working properly.

Message 4 of 5
DaneA
NETGEAR Employee Retired

Re: Linux Link Aggregation using 802.3ad

@Ryushin,

 

Thanks for the feedback.  I'm glad to know that everything is working fine. 🙂 

 

Since everything is working properly, I encourage you to mark the appropriate reply as the “Accepted Solution” so others can be confident in benefiting from the solution. The NETGEAR Community looks forward to hearing from you and being a helpful resource in the future!

 


Regards,

 

DaneA

NETGEAR Community Team

Message 5 of 5
Top Contributors
Discussion stats
  • 4 replies
  • 6243 views
  • 0 kudos
  • 2 in conversation
Announcements