Re: LACP no longer working on RN516 running 6.7.5

Tonkinite · ‎2017-08-08

LACP bonding no longer seems to work with my RN516 on 6.7.5, the switch shows only one interface up, as if the teaming is Active Backup, not LACP. The MTU keeps resetting to 1500 from 9000, as if that is the only thing it can get to work and all the uplink gear supports jumbo frame so that shouldn't be an issue.

Have tried dissolving the bond, resetting all the MTUs and re-bonding, but it still doesn't work. Works just fine on my RN626X, so possibly a model specific issue. Hash setting is L2 on both models, switch is a XS728T on firmware 6.5.1.30.

Advice?

Marty_M · ‎2017-08-09

Hello Tonkinite,

Was teaming mode working fine before and the problem just recently occured? Did the NAS encounter anything on the network that may have cause this behavior of the NAS. You may check this thread for additional information.

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-09

Teaming was last setup and verified on 6.6.1, with the switch at the previous minor release. Not verified on the upgrade to 6.7.2, 6.7.4 & 6.7.5, so could have faltered somewhere along the way.

Both links are lit up on both ends, so no obvious problem with the hardware, and I swapped out the cables as well, no difference.

l skimmed the system log, and there does appear to be signs of a problem, but it's not clear what. I can try changing the teaming mode to something else, and then back to see if that helps.

Likely a bug, hopefully is/will be addressed in 6.8 releases.

Marty_M · ‎2017-08-10

Hello Tonkinite,

There is no specific information on the release notes of firmware version 6.8.0 about the teaming/bonding on the NAS. The 6.8.0 firmware version had been already officially release you may want to try it and check if it will make any difference.

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-10

Changed the teaming mode to Round Robin, which seemed to allow the Jumbo frame MTU to stick, even after switching back to LACP, but the switch still only shows one line active (but the opposite one now).

Tried to manually install 6.8.0, it failed with "Unrecognized server error", even though it's a local install.

StephenB · ‎2017-08-10

@Tonkinite wrote:

Tried to manually install 6.8.0, it failed with "Unrecognized server error", even though it's a local install.

That one's new to me (and I haven't seen that error with my NAS). What browser were you using? At what point in the process did it fail?

StephenB · ‎2017-08-10

@StephenB wrote:

@Tonkinite wrote:

Tried to manually install 6.8.0, it failed with "Unrecognized server error", even though it's a local install.

That one's new to me (and I haven't seen that error with my NAS). What browser were you using? At what point in the process did it fail?

It's not new to me anymore I just saw it when installing the firmware on my RN526. Though I didn't see it on the RN524 I upgraded yesterday.

fwbroker didn't generate that error though, so I used that.

Tonkinite · ‎2017-08-10

Was using Safari. Switched to Chrome, FrontView (do we still call it that?) said the image was downloaded and asked permission to restart.

Network issue still there, Bonding appears even more finicky than in 6.7.5 (seems to take two tries to change the teaming mode).

Problem still there, sadly.

Marty_M · ‎2017-08-13

Hello Tonkinite,

Upgrading the firmware did not make any difference and the problem persists. Can you please give us a copy of the complete logs of the NAS, you may use any file sharing website like wikisend and just PM the download link.

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-13

Have moved Zip of logs to DropBox, will post a link to you as a private message.

If you find anything of concern, please let me know.

Thanks!

Marty_M · ‎2017-08-15

Hello Tonkinite,

Upon checking the logs I cannot find anything relevant that may cause this behavior of the NAS. Have you tried removing the bond and then re-creating it after the firmware upgrade? You can also try to remove the bond then observe each ethernet port connection if it will be stable or not.

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-15

Yes, made no difference on either firmware level, switch shows only one link active, unlike on the RN626X, where both links show active.

Marty_M · ‎2017-08-16

Hello Tonkinite,

Have you tried removing the bond and connect the NAS just via ethernet and observe if the connection is stable?

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-16

Yes, tried this morning.

Unbonded, unit works with both links, either separately or with both plugged in. Nothing wrong with the hardware.

Tonkinite · ‎2017-08-16

Has anyone tested this issue there? At this point, it appears to be a bug, possibly dating back to the early 6.7.x releases.

Marty_M · ‎2017-08-17

Hello Tonkinite,

What hashing is set on the bonding/teaming? Please try setting different hashing type and check if it will make any difference? Please provide details on how do you test the disconnection problem? For isolation have you tried connecting the NAS to different port on the switch? The LACP setup, is the NAS set on static IP address or it is getting it via DHCP?

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-17

L2 hashing is being used, and the ports on the switch the unit is normally connected to are LAG'd in LACP mode as L2 hashing is the only possibility on the XS series switches at the present time, would appreciate if you would speak to the team in charge of the XS series firmware on behalf of the ReadyNAS community. If I were to change the hashing mode, the LAG'd ports on the switch would simply not see either connection as active, you can look at the thread I opened on Community a while back regarding this.

As to connecting to non LAG'd ports on the switch, the switch then shows both connections as active, but per DaneA, this setup is not optimal, I'm guessing that means failover would work properly, but suspect both connections would not be used simultaneously.

Tonkinite · ‎2017-08-17

Additionally, the interfaces on the RN unit have fixed DHCP IP assignments (I typically setup all my RN units this way).

As to the status of the lines, I am trusting the XS switch UI panel not to lie, it says only one line is active, varying from one port to the other over time when I reboot the switch or whatever. This is not the case with the other, more recent unit, the RN626X, where the lines are LAG'd identically and both lines are always shown as active. A screenshot follows (the RN626X is connected on 1&2, the RN516 is connected on 3&4):

Switch port status, XS728T

Marty_M · ‎2017-08-20

Hello Tonkinite,

It does appear that both links wont go up at the same time. As per the type hashing for the switch it will depend on the type of switch if it is a layer 2 or layer 3. XS728T switch is only a layer 2 by default it only has layer 2 hashing. We can perform an OS re-install on the RN516 which will reset the network setting of the NAS and see if it will help with the address concern, for guide on how to perform OS re-install please go here and make sure to follow the guide carefully. Also for further isolation, if possible to connect the NAS to just a gigabit switch set the LAG and bonding/teaming then observe the connection.

Regards,
Marty_M
NETGEAR Community Team

Tonkinite · ‎2017-08-20

Reinstalling the OS had no effect.

I dissolved the bond and reestablished it. Also nothing.

Setting the link speed to force the ports to 1G on the switch also had no effect.

Dissolving the LAG on the switch and recreating it fixed it.

However, after that I noticed the bond MTU had reset to 1500, I reset it back to 9000 and the link hung, I had to reboot the unit to get it to come back up and it had reset itself to 1500 again.

I suspect this is the problem, not LACP. I think that with the newer firmware bonded interfaces may not allow Jumbo frame MTUs (or some subtypes may work but not others, though it's not clear why this would be). If this is the case, I'd be really surprised if a lot of your enterprise customers hadn't noticed.

If you have any further insights, please let me know.

evan2 · ‎2017-08-20

@Tonkinite

I had to reboot the unit to get it to come back up and it had reset itself to 1500 again.

=> This is known iusse, MTU change back from 9000 to 1500 in bonding mode after reboot NAS,

The issue is not fixed until now.

Tonkinite · ‎2017-08-21

Is hanging of bonded interfaces with Jumbo MTUs a known issue?

Is the LACP link issue (only one interface working at a time) a known issue?

evan2 · ‎2017-08-21

Is hanging of bonded interfaces with Jumbo MTUs a known issue?

=> Don't see the issue,

My RN516 (Test 6.7.5 and 6.8.0) set LACP layer 2, MTU 9000, connect XS728T (6.5.1.25), LAG: LACP

Connection is OK, two ports are link UP, and LAG State is UP,

Is the LACP link issue (only one interface working at a time) a known issue?

=> Don't see the issue,

if NAS bond as LACP, two connected ports on switch need to be added into a LAG, and LAG mode need to set LACP or Dynamic,

for interface working on LACP, it is based on Client's MAC and IP,

PC MAC	PC IP	Hash
even	odd	hash 1
odd	even	hash 1
even	even	hash 0
odd	odd	hash 0

NAS Slect Layer 2
Client1's MAC	Client2's MAC	How many Bonding Port is using
evan	odd	2 ports
evan	evan	1 port
odd	odd	1 port

NAS Slect Layer 2+3
Client1's Hash	Client2's Hash	How many Bonding Port is using
0	1	2 ports
0	0	1 port
1	1	1 port

Tonkinite · ‎2017-08-24

I am setting the individual MTUs as Jumbo frame before I bond the interfaces together and reset the bonded interface as Jumbo frame, could that be the issue?

evan2 · ‎2017-08-30

@Tonkinite

I saw the issue today, test switch is M7100, NAS FW is 6.9.0 T build,

Admin GUI disconnect after MTU set as 9000 on LACP mode,

Switch and PC also set to 9000,

If don't set bonding mode, connection is OK,

I have filed a bug in internal bug tracking system.