Reply
Highlighted
Guide

[BUG] GS108tv3 blocks udp broadcasts to port 123

Hi Guys,

 

I was evaluating the GS108Tv3 for table/POS access switch usage. I discovered that third party hardware suddenly was not able to communicatie with eachother anymore, so I investigated it, and discovered that sending broadcasts to udp poort 123 is actively dropped by the switch.

It's a bit of an unfortunate port, but that does not justify blocking the port.

 

Test case: take a factory reset GS108Tv3 with any firmware version up to 7.0.4.4 .

Attach a system on one port, and use this script:

root@ultra:~# for i in 122 123 124; do echo crap| socat - UDP4-DATAGRAM:10.255.255.255:$i,broadcast;done

 

It just broadcasts "crap" to udp ports 122, 123 and 124.


On another port listen to the device:

tcpdump -nei eth0 ether host 00:1e:06:31:cc:12 and udp

15:54:59.343666 00:1e:06:31:cc:12 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 60: 10.235.0.1.56381 > 10.255.255.255.122: UDP, length 5
15:55:00.373540 00:1e:06:31:cc:12 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 60: 10.235.0.1.38117 > 10.255.255.255.124: UDP, length 5

 

The switch also sees it drops packets, but it doesn't say why. In the web interface: Monitoring,Ports,Port Detailed Statistics clearly shows two counters:

 

Total Received Packets Not Forwarded

9

Dropped Transmit Frames

9

 

 

Could someone tell me how to report a grave bug like this to netgear?

 

EDIT 1:

 

16:48:31.565478 00:1e:06:31:cc:12 > 33:33:00:00:55:55, ethertype IPv6 (0x86dd), length 67: fe80::21e:6ff:fe31:cc12.56472 > ff02::5555.122: UDP, length 5
16:48:32.075350 00:1e:06:31:cc:12 > 33:33:00:00:55:55, ethertype IPv6 (0x86dd), length 67: fe80::21e:6ff:fe31:cc12.54246 > ff02::5555.123: NTPv4, Client, length 5
16:48:32.584974 00:1e:06:31:cc:12 > 33:33:00:00:55:55, ethertype IPv6 (0x86dd), length 67: fe80::21e:6ff:fe31:cc12.47350 > ff02::5555.124: UDP, length 5
16:49:38.466926 00:1e:06:31:cc:12 > 01:00:5e:01:01:01, ethertype IPv4 (0x0800), length 60: 10.235.0.1.60205 > 224.1.1.1.122: UDP, length 5
16:49:38.977551 00:1e:06:31:cc:12 > 01:00:5e:01:01:01, ethertype IPv4 (0x0800), length 60: 10.235.0.1.40214 > 224.1.1.1.123: NTPv4, Client, length 5
16:49:39.487050 00:1e:06:31:cc:12 > 01:00:5e:01:01:01, ethertype IPv4 (0x0800), length 60: 10.235.0.1.46736 > 224.1.1.1.124: UDP, length 5

 

So it's specific to IPv4 broadcast.

My next EDIT will when I move the management to out-of-band instead of in band, because I fear it might be a filter to specifically protect the host cpu, but the host cpu should just get by with a tiny iptables if it's necessary.

Message 1 of 13

Accepted Solutions
Highlighted
Guide

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

Update:

We got a beta release 7.0.4.7beta that fixed the issue.

Currently the switch has been running production for 2 months I think, and not a problem that's related to that switch.

 

Of course I've seen weird things going on in the switch (upnp, weird configuration daemons and such), but it doesn't stop it from working. So that beta seems good.


Changing the management vlan will be my next achievement, I fear I have to do that with scripted tftp download, sed, tftp upload, reboot. As the web doesn't allow it, and the command line doesn't allow it. Maybe if we use the serial console, but we do not intend to solder one on every switch ;-).

 

Anyway, past the initial problems of getting a support ticket opened (the "owner" is a different person than the tech support here), the support crew was helpful in getting it resolved.

 

Regards,

Ard van Breemen

View solution in original post

Message 13 of 13

All Replies
Highlighted
Guide

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

So:

If I move the management-vlan to out-of-band (which is hard to do, because it refuses to do it from the web interface, you need to upload a configuration file with the management-vlan set) then it still blocks.

 

However: if I configure sntp broadcast syncing or disable sntp at all, suddenly udp port 123 broadcasts are forwarded, independent of the VLAN.

 

I would have settled for an OOB workaround, but I can not settle for a managed switch that I can't time sync. I need port 123 broadcast forwarded, it's a switch.

Message 2 of 13
Highlighted
Guru

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

@YeZ something to push towards Smart Managed Pro QA - specifically for this GS108Tv3 model, and in general for all Smart Managed Pro switch specs ....

Message 3 of 13
Highlighted
Guide

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

@schumaku and @YeZ do you know what is the best way to get attention for this bug from netgear?photo_2020-06-04_20-53-35.jpg I mean we are not really tiny customers. But a switch as the GS108Tv3 in it's current state should not have been sold if it fails to do it's one major task: switch traffic undiscriminately. And I think the fix should not be that hard, because the code currently actively ads an acl for udp port 123 on the switch asic. It can just not do that and everything is fine.

Message 4 of 13
Highlighted
Guru

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

YeZ is with Netgear and can talk to the product management internally. I didn't had a chance to chat with him for about two days why ever.
Message 5 of 13
Highlighted
Guru

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

You can fetch the running config from the switch by Maintenance - Upload - Text Config to see if there is really such a config in place intentionally.
Message 6 of 13
Highlighted
NETGEAR Moderator

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

@Ardje,

 

We apologize for the inconvenience. I already raised this issue to our higher tier and I'll get back to you once we got an update.

 

Welcome to our community! Smiley Happy

 

Regards,

 

John

NETGEAR Community Team

Message 7 of 13
Highlighted
NETGEAR Expert

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

@Ardje 

 

Sorry to reply you later, this's by design on GS108Tv3 that SNTP using unicast mode by default, device drop UDP broadcast over port 123. If you don't mind, please change client mode from unicast to broadcast, the UDP broadcast over port 123 could be forwarding successfully.

Let's know if you have any, thanks.

GS108Tv3_SNTP.PNG

BR,
Netgear Employee
Message 8 of 13
Highlighted
Guru

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123


@Bruce_G wrote:

please change client mode from unicast to broadcast, the UDP broadcast over port 123 could be forwarding successfully.


I'm struggling to understand why the existence of an SNTP client talking unicast (to/from a defined, single IP address!!!) should impact what is going on on both the management and other VLANs. Why should the CPU port "capture" UDP broadcast as this traffic is clearly not intended for it - and then the same on ALL VLANs? This ready like a fat bug to me...

 

And what-if the customer does not have a local (S)NTP server within the broadcast domain (that IP subnet)? The time on the switch would not be set... Many smaller networks relay on external NTP sources.

 

PS. I could understand oddities (impacting traffic on all VLANs) on a Smart Managed Plus switch due to the way the microcontroller for the management is attached - but not here on a Smart Managed Pro device.

 

Curious whart @Ardje will say...

Message 9 of 13
Highlighted
NETGEAR Expert

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

@schumaku 

 

I could fully understand why you have concern. By RFC1305, the format of the NTP Message data area, which immediately follows the
UDP header and port number (123) assigned by the Internet Assigned Numbers Authority to NTP, GS108Tv3 just following RFC to implement. So on customer side, we encourage their services/application to use the reserved port instead of RFC pre-defined port.

 

BR,
Netgear Employee
Message 10 of 13
Highlighted
Guru

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

The CPU port is not the core of the universe, it must behave like yet another connected device and can't eat-up or blackhole the NTP UDP broadcasts... This is not about disputing the NTP RFC.

Message 11 of 13
Highlighted
Guide

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

Hi @Bruce_G , you will understand that a switch that does not forward udp port 123 packets on any vlan, because the management cpu that is in it's own management vlan, is using unicast as it should, is unacceptable? It's perfectly fine for me if it doesn't forward it on the management vlan, because it's a table switch after all. And if you have read the thread, you would see, I've already discovered that it always blocks.

 

But the management of the switch should not interfere with switching.

Now on to broadcasting: to work around a bug in a switch I have to add broadcasting ntp servers on each management domain. Since the switches are pretty limited, I also need to have multiple management domains and per domain a local server that does unicast ntp and broadcasts it to one or more, or have a hell of an l2 design.


I get that you must protect the host cpu, but it's a bug caused by a cheap work around against a possible ntp exploit. Make it less cheap by only filtering the management vlan. If people don't use management vlans, they also don't care about udp port 123.

 

My target networks are not office tables, it's large halls with machines where being able to get access to a network port is a security breach on it's own. Our need for network ports is that high that we included switches on our own hardware. But these are machine internal switches.

And with machines I mean PoS.

 

The machines can't be changed. They have been certified by more governmental bodies on their functioning than this switch has seen QA. The protocol over 123 used by these machines is not NTP.

 

In the mean time, my ticket to netgear support has revealed the following:

For now it only affects Linux+RTL SoC based small switches, not the Linux+Broadcom SoC based switches. I assume it's still a bug somewhere in netgears use of the RTL-sdk, as the sdk's are that different. I can not compare it to the similar DLINK devices that also use the RTL8380, since I do not have a dlink. But at least dlink's source code drop is not an insult to the GPL license.

I am not going to talk about the ECOS+Broadcom SoC based switches. No. Don't ask about them.
The ticket has landed at engineering, so I hope they implement a work around like just limiting the acl to the management vlan.

Anyway, you can clearly feel the difference between the two, because you can't change the management vlan on an RTL switch.

Message 12 of 13
Highlighted
Guide

Re: [BUG] GS108tv3 blocks udp broadcasts to port 123

Update:

We got a beta release 7.0.4.7beta that fixed the issue.

Currently the switch has been running production for 2 months I think, and not a problem that's related to that switch.

 

Of course I've seen weird things going on in the switch (upnp, weird configuration daemons and such), but it doesn't stop it from working. So that beta seems good.


Changing the management vlan will be my next achievement, I fear I have to do that with scripted tftp download, sed, tftp upload, reboot. As the web doesn't allow it, and the command line doesn't allow it. Maybe if we use the serial console, but we do not intend to solder one on every switch ;-).

 

Anyway, past the initial problems of getting a support ticket opened (the "owner" is a different person than the tech support here), the support crew was helpful in getting it resolved.

 

Regards,

Ard van Breemen

View solution in original post

Message 13 of 13
Discussion stats
  • 12 replies
  • 602 views
  • 6 kudos
  • 4 in conversation
Announcements