Terrible Bug - Adding a RBS20 satellite to RBK 50 system breaks the LAN

[superczar]

Following up from another thread- I just got done with a full troubleshoot and it appears that Interoperability between Orbi systems is rather badly broken.

Many users may not even notice this as this issue affects only LAN-LAN communication between mesh nodes while WAN-LAN works unaffected across the entire network

Background - 

I had a RBK50 (RBR50 and RBS50 ) that had been running fine for 2 years with a couple of firmware upgrades and minor woes in between. 

I had some blind spots in my porch so I decided to add another satellite - Since the RBS50 is expensive where I live and I was getting a great deal on amazon, I brought a RBS20 as I didn't need anything more than 200-300 mbps in the porch anyway.

Netgear claims Interoperability between different Orbi systems so I assumed the RBS20 is a no-brainer for my use case.

Bad call!

Adding the RBS20 was rather easy and I was able to add the satellite within 10 minutes - everything seemed absolutely fine to begin with. Internet was fast everywhere but by evening, I realized that a bunch of my LIFX and Hue lights were not responding .

Initial tests showed even more odd behavior e.g. Inability to ping various devices even though WAN access was fine across the board

Testing and Results

4+ hours of troubleshooting and i realized the problem runs deeper - All devices that were connecting to the RBR50 could connect to the Internet fine, However devices connected to the RBR50 could not reach other LAN devices and vice-versa.

All devices are on 192.168.5.0/24 subnet and there are no IP conflicts. All Orbi devices were tested with bost test cases of DHCP and static IPs assigned addresses

I swapped all switches , pulled out a bunch of old switches to see if the issue was with something else but no-go.

I rolled back the firmware on all Orbis, reupdated to latest, performed a full reset but no-go

8+ wasted hours later, I unplugged the RBR20, removed the satellite from the RBR50 and rebooted - Voila! Everything was back to normal!

I have attached a rough network diagram if it helps understand the topology - it's a moderately complex setup but nothing too crazy.

Apologies for my pathetic drawing skills - 30 years ago, my drawing classes at school used to be my worst nightmare 🙂

Edit: There is a (probably irrelevant) typo in the drawing - The wireless backhaul labels are swapped incorrectly

[CrimpOn]

Just a tiny thing.  The backhaul rates for the RBS50 and RBS20 are switched on the drawing.  The RBS50 backhaul is "up to 1333mb" and the RBS20 backhaul is "up to 866mb".  https://www.netgear.com/images/datasheet/orbi/RBS20.pdf 

[FURRYe38]

What happens if you remove the RBS50 and just connect the RBS20 alone to the RBR? 

Is Daisy Chain enabled or diabled on the RBR? Depending upon how you have placed the RBSs, Diasy Chain should be disable if in a STAR formation with the RBR. If RBSs are in a line out from the RBR, then DC can be enabled. 

What is the model # of the Gb switch? 

[superczar]

Yeah - Guess your post and my edit crossed - Noticed that after posting the OP 🙂

[CrimpOn]

The drawing doesn't indicate what is connected to the Orbi WiFi.

Does "location" make a difference in regard to the RBS20?

i.e. when it is connected in the same room as the router, the problems still occur?

[superczar]

During the testing, I did try both Daisy chain enabled and disabled.

In any case, the locations of the units are such that they do connect in star formation.

VErified the same via telnet commands as well.

I haven't explicitly tried removing the RBS50 but my guess is it would work fine as LAN comm between the RBS20 devices and RBR50 was working fine even with the RBS50 in the mix.

To put it another way, devices connected to the RBS50 and devices connected to the RBS20/RBR50 behave as if they are on separate LAN subnets/segments

[superczar]

That would have made the drawing even messier 🙂

There are approx 40 devices connecting to the Orbis, a few phones , laptops , tablets , A couple of streaming devices and a large bunch of Alexas, google hubs,  wireless bulbs and switches 

Location makes a difference in that if I were to connect my latop in proximity to the RBR50/RBS20, it would see devices on this segment fine but not  the RBS50 segment - and vice versa

[FURRYe38]

Lets turn OFF the RBS50 and see what happens. 

Wondering if there is a problem with the RBS20. 

Is FW up to date on the RBS20? 

[superczar]

For my sanity, I have for now removed the RBS20 and gone back to the old setup.

Logically though, removing the RBS50 alone would keep things working as expected as all devices connecting to other two anyway see each other fine.

For the same reason, I doubt that there is hardware issue on the new RBS20 either

The real problem is that adding the RBS20 somehow created this quasi VLAN or subnet where devices connecting to the original RBR50 and new RBS20 were on one segment while RBS50 and associated stations were on another.

Removing the RBS20 fixes that 

As for the FW, all 3 devices are on the latest firmware i.e. 2.3.5.30 for the Orbi 50 devices and 2.3.5.26 for the Orbi20

during my troubleshooting, I did manually update all 3 to be on the last common firmware version which was 2.3.1.48

[FURRYe38]

Well for the sake of troubleshooting and narrowing down the issue, testing of the RBS20 alone with the RBR with out the RBS50 being online would be a good test to try. Isolation of the RBS20 problem is recommended. Also trying of a different RBS would be valid as well. Even trying the RBS20 at a different location with a Orbi system would be valid if you know someone. 

Other than this, you might contact NG support and see what they say. Overall I agree the RBS20 should not be causing these problems. Thus seems to be a problem with the RBS20 and the RBR for some reason,. 

[CrimpOn]

---

@superczar wrote:

Location makes a difference in that if I were to connect my latop in proximity to the RBR50/RBS20, it would see devices on this segment fine but not  the RBS50 segment - and vice versa

---

By "location", I meant where the RBR20 was physically located.  If you still have it, what happens if you turn it on (literally) in the same room with the router.  (Not where it was located to serve the porch.)

[CrimpOn]

Also, Netgear provides 90 days of free telephone support for new purchases.  Since the RBS20 is "new" from Amazon, have you tried calliing Netgear?

[CrimpOn]

In the previous post that this thread is based on, the Orbi RBR50 was in Access Point mode, and the Orbi units had "static IP's".

• This is still the case?
• "Static" means (a) set locally on the Orbi, or (b) set up in a table on the primary router?

[superczar]

The static IP addresses have been assigned on the primary router (EdgeRouter-X)

 I re-added the RBS20 to the network - Oddly enough, things seemed to be fine initially 

However, devices that were off at the point of addition seem to be unable to view the devices on the RBR-50 after they were switched on.

To validate, I ran an arp query on the device that can't see the RBR devices and the arp table shows the entry for the RBR50 MAC as incomplete i.e. the IP address:MAC updates aren't happening as expected on the devices attached to segment 2 (i.e. RBR50) and segment 1 (i.e. RBS20/RBR50)

It's 2:30 AM here now - I guess I will have to push the rest of the checks to tomorrow but I have this strong hunch that the Orbi running in router mode will maintain some kind of a lookup table shared between nodes that will prevent this scenario.

In AP mode though, it is encountering some bug that is preventing it from relaying the ARP mapping upstream

[CrimpOn]

---

@superczar wrote:

I have this strong hunch that the Orbi running in router mode will maintain some kind of a lookup table shared between nodes that will prevent this scenario.

In AP mode though, it is encountering some bug that is preventing it from relaying the ARP mapping upstream

---

Putting the Orbi in router mode is likely to create problems of a different sort because devices on the "outside" of the Orbi will have no idea where to find devices "behind" the Orbi.  Once something on the inside opens a NAT path through the Orbi to something on the outside, that particular connection will be solid.  But, a device on the outside looking for one on the inside won't find it because the private IP space behind the Orbi router is hidden from the outside.

In the previous thread, the question was asked about the exact model numbers of the three switches. I did not find the answer.

p.s. That Ubiquiti router, "Oh, My"  What an amazing set of capabilities!

[superczar]

The gbe switches are TP Link SG108e , D-link 1008 and the 100mbps switches are TP LinkSF1005D

To set the Orbi in router mode, I will of course be connecting the Orbi WAN port directly to the router and move switch 1 to the LAN port of the Orbi.

However I intend to do this only testing because I use the ER-X for VPN and other firewall rules that the Orbi doesn't support

[CrimpOn]

As long as you are experimenting, could you turn off the IGMP Snooping feature of the TP Link switch?

(I know this sounds silly, but there have been reports that IGMP Snooping has been implicated in Orbi problems in the past.)

[superczar]

That actually was one of the first things I tried .

I even swapped out the sg108 with a dumb switch for testing purposes but that didn't make any difference

[CrimpOn]

With a new purchase, I would really like to hear Netgear support's take on this problem.

With the RBR50 in Access Point mode, adding a second satellite (of any model) should not change anything in regard to connections.

• All the IP assignments come from the router
• It should not matter whether a device connects to the old satellite or the new satellite

When a situation is simply "impossible", there has to be a reason.  There is something that I am not seeing.

[superczar]

Some success at last - Let's hope this works as a permanent solution

Based on the issue at hand, I surmised there is something wrong with the routing tables on the Orbi - quite possibly the RBR50 

I added a static route on the RBR50 to provide 192.168.5.1 (My router) as the gateway for all traffic on the 192.168.5.0/24 subnet 

While this should theroretically not be required as the internal switch on the Orbi should be transparent to all devices connected to it but based on the incomplete ARP tables on the client devices, it did appear that it isn't quite working as expected

And guess what, all devices can see each other now!

I will probably need to test it for 1-2 days to see if there are no more unexpected glitches plus it still doesn't tell me really say anything about what the issue is

My guess is that the Orbi switch even in AP mode works more as a managed switch and something on it wasn't playing nice

With this static route defined, devices are probably performing lookups via the Edgerouter (which on a side note is a ridiculously reliable and powerful devices for he price)

Anyway , as long as it works as a workaround, I am not complaining..

I got the RBS20 for 6500 INR (~90 USD) which is a steal and while i do have a few more days in the return window, I wasn't particularly keen on doing that because of the price paid.

Thanks for all the help folks !

[CrimpOn]

---

@superczar wrote:

I added a static route on the RBR50 to provide 192.168.5.1 (My router) as the gateway for all traffic on the 192.168.5.0/24 subnet 

---

Wow, this is creative.  I would have assumed that all of those features (static routes, port forwarding, triggering, etc.) would not be available when the Orbi is in AP mode.

[superczar]

None of the routing features (i.e. Layer 3) are available in AP mode.

Static routes are a layer 2 switching feature and the Orbi acts like a semi-managed switch when in AP mode.

It's been serveral hours of heavy usage of the network now and things seem to be working fine! 🙂

[superczar]

Unfortunately the static route workaround did not solve the problem completely.

Devices connected to the RBS50 would stop seeing other devices on the LAN after some time.

Having wasted 8+ hours on it, decided to give up and set the Orbi to router mode this morning.

The ER-X handles the dual WAN and passes on the Internet to the RBR50 - The RBR50 is now acting as the primary router for the rest of the network.

I am not facing any of the mentioned issues in Router mode now.

While this setup with its double NAT is certainly not optimal , plus I also lose the ability to VPN to my home from outside 

I still like the Orbi but these odd issues make it very difficult to recommend it to others 

[FURRYe38]

If your going to use the ERX and RBR both in router mode, I would set the RBR into the ERX's DMZ for less hassle with double NAT. I do this for routers i periodically test from time to time behind my primary router. 

[superczar]

Thanks - That's a good tip.

Sadly the issue is back - even in router mode, things worked fine all day but I see dropped LAN connectivity from RBS50 again.

I enabled fast roaming an hour ago - while this doesn't seem related, I will try disabling and retesting.

On a separate note, this does not appear to be an isolated issue 

orbi satellite-2 drops connection to satellite-1 =... - NETGEAR ...https://community.netgear.com › O...