This topic has been marked solved and closed to new posts due to inactivity. We hope you'll join the conversation by posting to an open topic or starting a new one.
Hi, I have the following switches on a network, and they all suffer from this issue:
2x GS110T (v5.4.0.6 - latest?)
2x GS110TP (v5.4.2.30 - latest?)
1x GS110TP (v5.0.5.4)
1x GS110TP (v5.4.2.13)
1x GS110TP (v5.4.2.18)
I am using them with a piece of software (Sennheiser WSM) which monitors Radio Mics over broadcast. As soon as this traffic starts the following occours:
All switches either do not repond to most pings (70%+ loss), and when they do they have a very high reponse time (average 600ms+)
the web interfaces on all switches fails to respond, and any open sessions stop functioning until the traffic is stopped.
The traffic continues to pass, and reaches all the PC's ok, it just makes it impossible to monitor the switches, or configure them for anything else.
My *guess* is that there is something in the packets that the switch cannot understand, and so the internal CPU is getting maxed out while trying to read it.
I have tried with IGMP snooping on and off but there is no difference.
Sorry for the delay, I had been trying to make sure that everything was stable with the updated firmwares before updating.
As you say they were able to provide an updated firmware, that appears to be working fine for the GS110TP models.
for the GS110T switches I was given a work around from Engineering, which I will post below for anyone else who has this problem:
"The root cause is, the reserved range multicast packets comes to switch CPU and the IGMP application fowards them to other ports in same VLAN. Since, the packets are coming at high rate, CPU is becoming slow and failing to open GUI connection.
On GS110T, you can try installing an ACL and drop the multicast packets in the range 224.0.0.1 to 224.0.0.255. If you need these packets, then configure ACL with redirect action (redirect the matched packets to desired port)."
a. How is everything connected? Is the PC with the Sennheiser WSM installed directly connected to each switch? It would be best if you post a diagram of your existing network setup.
b. What is the behavior of the LED lights on the switches when the problem occurs?
Switches are connected in a star config, with either a GS110T/TP or a GS728TS (unaffected by this issue) in the middle.
The 'central' unit will have between 1 and 3 pc's running the WSM software connected over RJ45, and then between 1 and 5 of the other GS110T/TP switches are connected either over RJ45 or via SFP. In some circumstances I will daisy chain witches on one of the legs depending on the cable situation.
There are no loops on the network (RSTP is enabled and configured (and tested!)), and this bug affects switches that are on legs from the central switch with no other kit connected.
LED's show normally, with flashing for traffic on all ports, but nothing that doesn't appear normal.
have you looked at the Wireshark trace that I have provided, and tried sending that traffic over switches that you have on a test bench?
Cheers James
P.S. I am aware that this is a Multicast issue, not a Broadcast issue, but I wasn't able to edit the post to say that.
I just got a feedback from the higher tier of NETGEAR Support regarding your concern.
As per the higher tier of NETGEAR Support, most of the NETGEAR model switches, we put low priority on ping packets to the switch IP address and put higher priority on data packets passing through the switch, this is why you would see no response or high response times on ping, but no issue with traffic passing through the switch. Normally, this shouldn’t affect the access to the web-GUI. However, if there is a high volume of traffic, it may simply delay the response and could be why the web-GUI doesn’t load.
It was recommended that you kindly try to update the firmware of all switches to the latest version then observe.
For the GS110T switch. you can download the latest firmware v5.4.0.6 here.
For the GS110TP, you can download the latest firmware v5.4.2.30 here.
Note: Be reminded to perform a factory reset on the switches after upgrading the firmware then reconfigure it from scratch in order to start clean using the latest firmware version.
As per my first post, I already have several of the switches running on the most recent firmware(s), and the problem is still there. I have tried factory resets as well and that makes no difference.
In this situation, the Web GUI IS affected, my use of the pings were to purely demonstrate when things stop working. The GUI stops functioning at exactly the same time.
From the trace provided, it should be possible to see that we are not dealing with a high volume of traffic at all, and that there is some processing that the switch is trying to perform and failing at.
Has anyone actually tried sending the data in the trace across a switch on a test bench yet? That response from higher tier reads as suspiciously like a stock answer to me.
The suggestion about having all switches be on the latest firmware version is a way to isolate more of the problem in order to see what would be the result if ever the same problem will occur if all of the switches in the star topology are on the latest firmware version.
The Wireshark packet trace you have provided was checked by the higher tier of NETGEAR Support that is why they came up with the suggestion mentioned.
I believe I have tested the setup before with only the switches running the latest firmware, and the issue was the same.
I will test it again, but it will be next weekend the next time all the equipment is setup together.
Are there any other combinations that they would like proving, such as only GS110TP's, and leave the GS110T's and GS728TS out of the setup?
Are there any other samples that it would be useful for me to capture using wireshark, such as longer duration, or traces covering more of the switches, or showing attempts to use the GUI while the issue is occouring?
As for now, there are no other switch combination which the higher tier of NETGEAR Support would like to probe. Feel free to capture packets using Wireshark (such as longer duration, or traces covering more of the switches, or showing attempts to use the GUI while the issue is being observed) when you perform the test again after having the switches updated on the latest firmware version.
with 2 x GS110TP linked over SFP and a 3rd GS110TP linked over CAT5e from one of the previous GS110TP's
All switches are reconfigured with the latest firmware loaded.
I have background pings running to all 3 GS110TP's (192.168.99.200, 192.168.99.210 and 192.168.99.220)
The monitoring software is already running at the start, and continues until roughly 140 seconds, at which point I stop the monitoring, and wait until 202 seconds before starting it again.
I have been trying to load the web interface of the switches from the start, but do not get very much from them until after I have stopped the monitoring. I log into all 3 and load the switching ->ports tab ok on all 3. After I restart the monitoring I attempts to navigate to other tabs on all 3, and have very little success.
Is there any other information that I can provide for you?
Thanks for the update. Since its still the same problem, as per advised of the higher tier of NETGEAR Support, kindly open a chat or online case with NETGEAR Support at anytime. State your concern and attach the packet captures to the chat/online case. Indicate the link of this forum thread as well. This will be escalated to the engineering team for further investigation.
I just want to follow-up on this. If ever you have opened a chat or online case with NETGEAR Support, kindly provide us a feedback on what is the status of it.
I inquired about your case from NETGEAR Support and found out that it is already set to close status. A firmware (beta) for the GS110TP switch was provided to you by the Level 3 expert and this seemed to fix the issue. However, the beta firmware is not compatible with the GS110T switch and there will be no fix for it since it is already EOL (End-of-Life).
Feel free to post your future concerns here in NETGEAR Community. 🙂 I encourage you to mark the appropriate reply as the “Accepted Solution” so others can be confident in benefiting from the solution. The NETGEAR Community will look forward to hearing from you and being a helpful resource in the future!
Sorry for the delay, I had been trying to make sure that everything was stable with the updated firmwares before updating.
As you say they were able to provide an updated firmware, that appears to be working fine for the GS110TP models.
for the GS110T switches I was given a work around from Engineering, which I will post below for anyone else who has this problem:
"The root cause is, the reserved range multicast packets comes to switch CPU and the IGMP application fowards them to other ports in same VLAN. Since, the packets are coming at high rate, CPU is becoming slow and failing to open GUI connection.
On GS110T, you can try installing an ACL and drop the multicast packets in the range 224.0.0.1 to 224.0.0.255. If you need these packets, then configure ACL with redirect action (redirect the matched packets to desired port)."
Thanks for the update. I'm sure other this will help other people who encountered the same problem and I'm glad to know that the NETGEAR Engineering Team was able to help you. 🙂
