× NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Orbi WiFi 7 RBE973

GS110T/GS110TP Multicast overload bug

JMSladen
Aspirant

GS110T/GS110TP Multicast overload bug

Hi, I have the following switches on a network, and they all suffer from this issue:

 

2x GS110T (v5.4.0.6 - latest?)

2x GS110TP (v5.4.2.30 - latest?)

1x GS110TP (v5.0.5.4)

1x GS110TP (v5.4.2.13)

1x GS110TP (v5.4.2.18)

 

I am using them with a piece of software (Sennheiser WSM) which monitors Radio Mics over broadcast. As soon as this traffic starts the following occours:

All switches either do not repond to most pings (70%+ loss), and when they do they have a very high reponse time (average 600ms+)

the web interfaces on all switches fails to respond, and any open sessions stop functioning until the traffic is stopped.

The traffic continues to pass, and reaches all the PC's ok, it just makes it impossible to monitor the switches, or configure them for anything else.

 

My *guess* is that there is something in the packets that the switch cannot understand, and so the internal CPU is getting maxed out while trying to read it.

 

I have tried with IGMP snooping on and off but there is no difference.

 

I have saved a 25 second wire shark capture.

http://jasmine.19inch.net/~james/wireshark/Wireshark-GS110TP-issues2-filtered.pcapng

When it is started I am pinging one of the switches (PC IP: 192.168.99.41, Switch IP: 192.168.99.210), which continues all the way through.

At 12 seconds I enabled the switch ports with the radio equipment attached.

at 12.1 and 12.2 seconds the broadcast traffic from the equipment starts

at 13.6 seconds there is the first ping request without a response.

For the rest of the capture the ping responses are either very slow, or non-existant

 

Does anyone have any ideas of what I can do to stop the switches overloading, or at least minimize the effect?

 

Cheers

James

 

[edit] @kohdee added multicast in lieu of broadcast to the subject [/edit]

Model: GS110T|ProSAFE 8-port Smart Switch with fiber uplink, GS110TP|ProSAFE 8-port Gigabit PoE Smart Switch
Message 1 of 18

Accepted Solutions
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

 

Sorry for the delay, I had been trying to make sure that everything was stable with the updated firmwares before updating.

 

As you say they were able to provide an updated firmware, that appears to be working fine for the GS110TP models.

 

for the GS110T switches I was given a work around from Engineering, which I will post below for anyone else who has this problem:


"The root cause is, the reserved range multicast packets comes to switch CPU and the IGMP application fowards them to other ports in same VLAN. Since, the packets are coming at high rate, CPU is becoming slow and failing to open GUI connection.

On GS110T, you can try installing an ACL and drop the multicast packets in the range 224.0.0.1 to 224.0.0.255. If you need these packets, then configure ACL with redirect action (redirect the matched packets to desired port)."

 

Cheers

James

View solution in original post

Message 17 of 18

All Replies
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP multicast overload bug

Hi JMSladen,

 

Welcome to the community! 🙂 

 

Kindly answer the questions below:

 

a. How is everything connected? Is the PC with the Sennheiser WSM installed directly connected to each switch?  It would be best if you post a diagram of your existing network setup.

b. What is the behavior of the LED lights on the switches when the problem occurs?

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 2 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP broadcast overload bug

Hi DaneA,

Switches are connected in a star config, with either a GS110T/TP or a GS728TS (unaffected by this issue) in the middle.


The 'central' unit will have between 1 and 3 pc's running the WSM software connected over RJ45, and then between 1 and 5 of the other GS110T/TP switches are connected either over RJ45 or via SFP. In some circumstances I will daisy chain witches on one of the legs depending on the cable situation.

There are no loops on the network (RSTP is enabled and configured (and tested!)), and this bug affects switches that are on legs from the central switch with no other kit connected.

LED's show normally, with flashing for traffic on all ports, but nothing that doesn't appear normal.

have you looked at the Wireshark trace that I have provided, and tried sending that traffic over switches that you have on a test bench?

Cheers
James

P.S. I am aware that this is a Multicast issue, not a Broadcast issue, but I wasn't able to edit the post to say that.
Message 3 of 18
kohdee
NETGEAR Expert

Re: GS110T/GS110TP Multicast overload bug

Fixed the title of the thread. 

Message 4 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP multicast overload bug

@JMSladen,

 

I'll have to inquire your concern to the higher tier of NETGEAR Support.  I'll post in this thread once I got a feedback.

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 5 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP broadcast overload bug

@JMSladen,

 

I just got a feedback from the higher tier of NETGEAR Support regarding your concern.  

 

As per the higher tier of NETGEAR Support, most of the NETGEAR model switches, we put low priority on ping packets to the switch IP address and put higher priority on data packets passing through the switch, this is why you would see no response or high response times on ping, but no issue with traffic passing through the switch. Normally, this shouldn’t affect the access to the web-GUI.  However, if there is a high volume of traffic, it may simply delay the response and could be why the web-GUI doesn’t load.

 

It was recommended that you kindly try to update the firmware of all switches to the latest version then observe.  

 

For the GS110T switch. you can download the latest firmware v5.4.0.6 here.  

 

For the GS110TP, you can download the latest firmware v5.4.2.30 here.  

 

Note:  Be reminded to perform a factory reset on the switches after upgrading the firmware then reconfigure it from scratch in order to start clean using the latest firmware version.  

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 6 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

As per my first post, I already have several of the switches running on the most recent firmware(s), and the problem is still there. I have tried factory resets as well and that makes no difference.

In this situation, the Web GUI IS affected, my use of the pings were to purely demonstrate when things stop working. The GUI stops functioning at exactly the same time.

From the trace provided, it should be possible to see that we are not dealing with a high volume of traffic at all, and that there is some processing that the switch is trying to perform and failing at.

Has anyone actually tried sending the data in the trace across a switch on a test bench yet? That response from higher tier reads as suspiciously like a stock answer to me.

Cheers
James

Message 7 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

The suggestion about having all switches be on the latest firmware version is a way to isolate more of the problem in order to see what would be the result if ever the same problem will occur if all of the switches in the star topology are on the latest firmware version. 

 

The Wireshark packet trace you have provided was checked by the higher tier of NETGEAR Support that is why they came up with the suggestion mentioned.

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 8 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

I believe I have tested the setup before with only the switches running the latest firmware, and the issue was the same.

I will test it again, but it will be next weekend the next time all the equipment is setup together.

Are there any other combinations that they would like proving, such as only GS110TP's, and leave the GS110T's and GS728TS out of the setup?

Are there any other samples that it would be useful for me to capture using wireshark, such as longer duration, or traces covering more of the switches, or showing attempts to use the GUI while the issue is occouring?

Cheers
James
Message 9 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

As for now, there are no other switch combination which the higher tier of NETGEAR Support would like to probe.  Feel free to capture packets using Wireshark (such as longer duration, or traces covering more of the switches, or showing attempts to use the GUI while the issue is being observed) when you perform the test again after having the switches updated on the latest firmware version.   

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 10 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

Just following up on this.  Any updates?  

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 11 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

It will be the weekend before I have any updates for you.

Cheers
James
Message 12 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

 

Sorry for the Delay in getting back to you.

 

I have done a much longer capture for you.

 

http://jasmine.19inch.net/~james/wireshark/Wireshark-GS110TP-issues-long3-unfiltered.pcapng

 

This setup comprised of 4 switches:

1x GS728TS

with 2 x GS110TP linked over SFP and a 3rd GS110TP linked over CAT5e from one of the previous GS110TP's

 

All switches are reconfigured with the latest firmware loaded.

 

I have background pings running to all 3 GS110TP's (192.168.99.200, 192.168.99.210 and 192.168.99.220)

The monitoring software is already running at the start, and continues until roughly 140 seconds, at which point I stop the monitoring, and wait until 202 seconds before starting it again.

 

I have been trying to load the web interface of the switches from the start, but do not get very much from them until after I have stopped the monitoring. I log into all 3 and load the switching ->ports tab ok on all 3. After I restart the monitoring I attempts to navigate to other tabs on all 3, and have very little success.

 

Is there any other information that I can provide for you?

Message 13 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

Thanks for the update.  Since its still the same problem, as per advised of the higher tier of NETGEAR Support, kindly open a chat or online case with NETGEAR Support at anytime.  State your concern and attach the packet captures to the chat/online case.  Indicate the link of this forum thread as well.  This will be escalated to the engineering team for further investigation.  

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 14 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

I just want to follow-up on this.  If ever you have opened a chat or online case with NETGEAR Support, kindly provide us a feedback on what is the status of it.

 

 

Regards,

 

DaneA

NETGEAR Community Team

Message 15 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

I inquired about your case from NETGEAR Support and found out that it is already set to close status.  A firmware (beta) for the GS110TP switch was provided to you by the Level 3 expert and this seemed to fix the issue.   However, the beta firmware is not compatible with the GS110T switch and there will be no fix for it since it is already EOL (End-of-Life).  

 

Feel free to post your future concerns here in NETGEAR Community. 🙂   I encourage you to mark the appropriate reply as the “Accepted Solution” so others can be confident in benefiting from the solution. The NETGEAR Community will look forward to hearing from you and being a helpful resource in the future!

 

 

Cheers,

 

DaneA

NETGEAR Community Team

Message 16 of 18
JMSladen
Aspirant

Re: GS110T/GS110TP Multicast overload bug

Hi DaneA,

 

Sorry for the delay, I had been trying to make sure that everything was stable with the updated firmwares before updating.

 

As you say they were able to provide an updated firmware, that appears to be working fine for the GS110TP models.

 

for the GS110T switches I was given a work around from Engineering, which I will post below for anyone else who has this problem:


"The root cause is, the reserved range multicast packets comes to switch CPU and the IGMP application fowards them to other ports in same VLAN. Since, the packets are coming at high rate, CPU is becoming slow and failing to open GUI connection.

On GS110T, you can try installing an ACL and drop the multicast packets in the range 224.0.0.1 to 224.0.0.255. If you need these packets, then configure ACL with redirect action (redirect the matched packets to desired port)."

 

Cheers

James

Message 17 of 18
DaneA
NETGEAR Employee Retired

Re: GS110T/GS110TP Multicast overload bug

@JMSladen,

 

Thanks for the update.  I'm sure other this will help other people who encountered the same problem and I'm glad to know that the NETGEAR Engineering Team was able to help you. 🙂  

 

 

Cheers,

 

DaneA

NETGEAR Community Team

Message 18 of 18
Top Contributors
Discussion stats
  • 17 replies
  • 5970 views
  • 0 kudos
  • 3 in conversation
Announcements