- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Linux Link Aggregation using 802.3ad
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've been at this for more then a day now. I'm trying to set up 802.3ad link aggregation with a Debian Jessie system.
I have two Intel 10-Gigabit X540-AT2 that I've configured for 802.3ad to combine the four ports together.
Linux Side:
/etc/network/interfaces:
auto bond_storage
iface bond_storage inet manual
bond_slaves eth4 eth5 eth6 eth7
bond_miimon 100
bond_mode 802.3ad
bond-lacp-rate fast
bond_xmit_hash_policy layer2+3
bond_downdelay 200
bond_updelay 200
cat /proc/net/bonding/bond_storage:
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: down
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 0c:c4:7a:34:a3:2c
bond bond_storage has no active aggregator
Slave Interface: eth4
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 0c:c4:7a:34:a3:2c
Slave queue ID: 0
Aggregator ID: N/A
Slave Interface: eth5
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 0c:c4:7a:34:a3:2d
Slave queue ID: 0
Aggregator ID: N/A
Slave Interface: eth6
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:12:17:90
Slave queue ID: 0
Aggregator ID: N/A
Slave Interface: eth7
MII Status: down
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:12:17:92
Slave queue ID: 0
Aggregator ID: N/A
Contents of /var/log/kern.log:
Dec 16 14:33:33 fatman kernel: [4604427.914455] bond_storage: Setting xmit hash policy to layer2+3 (2)
Dec 16 14:33:33 fatman kernel: [4604427.915885] bond_storage: Setting MII monitoring interval to 100
Dec 16 14:33:33 fatman kernel: [4604427.917102] bond_storage: Note: Updating updelay (to 200) since it is a multiple of the miimon value
Dec 16 14:33:33 fatman kernel: [4604427.918273] bond_storage: Note: Updating downdelay (to 200) since it is a multiple of the miimon value
Dec 16 14:33:33 fatman kernel: [4604427.919464] bond_storage: Setting down delay to 200
Dec 16 14:33:33 fatman kernel: [4604427.920641] bond_storage: Setting up delay to 200
Dec 16 14:33:33 fatman kernel: [4604427.924971] bond_storage: Setting LACP rate to fast (1)
Dec 16 14:33:33 fatman kernel: [4604427.929008] bond_storage: Adding slave eth4
Dec 16 14:33:33 fatman kernel: [4604428.034981] pps pps0: new PPS source ptp0
Dec 16 14:33:33 fatman kernel: [4604428.036174] ixgbe 0000:09:00.0: registered PHC device on eth4
Dec 16 14:33:34 fatman kernel: [4604428.158360] bond_storage: Enslaving eth4 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.162256] bond_storage: Adding slave eth5
Dec 16 14:33:34 fatman kernel: [4604428.263528] pps pps1: new PPS source ptp1
Dec 16 14:33:34 fatman kernel: [4604428.263531] ixgbe 0000:09:00.1: registered PHC device on eth5
Dec 16 14:33:34 fatman kernel: [4604428.385639] bond_storage: Enslaving eth5 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.389529] bond_storage: Adding slave eth6
Dec 16 14:33:34 fatman kernel: [4604428.489314] pps pps2: new PPS source ptp2
Dec 16 14:33:34 fatman kernel: [4604428.489318] ixgbe 0000:85:00.0: registered PHC device on eth6
Dec 16 14:33:34 fatman kernel: [4604428.609600] bond_storage: Enslaving eth6 as a backup interface with a down link
Dec 16 14:33:34 fatman kernel: [4604428.613185] bond_storage: Adding slave eth7
Dec 16 14:33:34 fatman kernel: [4604428.715415] pps pps3: new PPS source ptp3
Dec 16 14:33:34 fatman kernel: [4604428.715418] ixgbe 0000:85:00.1: registered PHC device on eth7
Dec 16 14:33:34 fatman kernel: [4604428.835904] bond_storage: Enslaving eth7 as a backup interface with a down link
Dec 16 14:33:38 fatman kernel: [4604433.075261] ixgbe 0000:85:00.0 eth6: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:38 fatman kernel: [4604433.139097] ixgbe 0000:85:00.0 eth6: NIC Link is Down
Dec 16 14:33:39 fatman kernel: [4604433.254553] ixgbe 0000:85:00.1 eth7: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:39 fatman kernel: [4604434.083028] ixgbe 0000:09:00.0 eth4: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:40 fatman kernel: [4604434.815192] ixgbe 0000:85:00.0 eth6: NIC Link is Up 10 Gbps, Flow Control: None
Dec 16 14:33:46 fatman kernel: [4604441.034799] ixgbe 0000:09:00.1 eth5: NIC Link is Up 10 Gbps, Flow Control: None
The switch shows the LAG state as down.
I've attached pictures of the Switch Config.
Solved! Go to Solution.
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I rebooted the server and now MII Status show as UP. So good news there. Not used to having to reboot a linux server to fix something.
I rebooted without a separate VLAN for the LAG ports. The LAG and the LAG ports are in the same VLAN. I can ping the server so I think it is working. Do the LAG ports need to be in their own separate VLAN or should the ports and the LAG be in the same VLAN?
All Replies
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I rebooted the server and now MII Status show as UP. So good news there. Not used to having to reboot a linux server to fix something.
I rebooted without a separate VLAN for the LAG ports. The LAG and the LAG ports are in the same VLAN. I can ping the server so I think it is working. Do the LAG ports need to be in their own separate VLAN or should the ports and the LAG be in the same VLAN?
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Linux Link Aggregation using 802.3ad
Hi @Ryushin,
Welcome to the community! 🙂
It seemed that the rebooting the server helped applied the LAG settings configured. About having the LAG ports (where the server is connected) be in the same or separate VLAN, it would be up for you to decided if whether you need the server be accessible by other users within the same VLAN or not. If ever you need the server be within the same VLAN having limited access to a number of users, you will need to configure ACLs (Access Control Lists). Let me share the article below as reference:
Regards,
DaneA
NETGEAR Community Team
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Linux Link Aggregation using 802.3ad
Our end goal is to bond the 4 ports together for access for rest of the LAN. I did not know if the four individual ports in the LAG need to be in their own VLAN-1000 and the LAG itself needs to be in the VLAN-1. Right now the individual ports are in VLAN-1 along with the LAG and everything appears to be working properly.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Linux Link Aggregation using 802.3ad
Thanks for the feedback. I'm glad to know that everything is working fine. 🙂
Since everything is working properly, I encourage you to mark the appropriate reply as the “Accepted Solution” so others can be confident in benefiting from the solution. The NETGEAR Community looks forward to hearing from you and being a helpful resource in the future!
Regards,
DaneA
NETGEAR Community Team