NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.

Forum Discussion

matslundqvist's avatar
matslundqvist
Aspirant
Nov 19, 2020

MS510TXPP not getting more than 1Gb on 5G End to End

EDIT: Tried posting yesterday, doesn't show up in the board. This is my 2nd attempt.

 

Hi, I have two switches in two locations, CAT6A copper between them in the 10GbE port approx 35m cable length. Each switch has one MultiG adapter attached to a 5GbE port (AQC107).

 

<Server> ----5GbE---> <SW01> -----10GbE-----<SW02>----5GbE----<Client>

Firmware 6.7.0.37

 

The switch, client and server all report wirespeed (5G and 10G respectively).

And I'm not getting more than 1Gbit speeds, I and I cannot figure out why.

 

I ran Iperf3 between server and client. I usually run Windows on the client, but I booted a live usb and ran it in Linux to eliminate software and driver issues. The server is a N54L microserver running ubuntu 16.04.

I've switched cabling (except the site to site cable) and ports.

I did cabletests within the switch web gui.

 

Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.1.127, port 49146
[  5] local 192.168.1.22 port 5201 connected to 192.168.1.127 port 49148
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   112 MBytes   940 Mbits/sec                  
[  5]   1.00-2.00   sec   112 MBytes   940 Mbits/sec                  
[  5]   2.00-3.00   sec   112 MBytes   941 Mbits/sec                  
[  5]   3.00-4.00   sec   112 MBytes   938 Mbits/sec                  
[  5]   4.00-5.00   sec   112 MBytes   941 Mbits/sec                  
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec                  
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec                  
[  5]   7.00-8.00   sec   112 MBytes   941 Mbits/sec                  
[  5]   8.00-9.00   sec   112 MBytes   936 Mbits/sec                  
[  5]   9.00-10.00  sec   112 MBytes   941 Mbits/sec

I adjusted the frame size to 8000, just for testing, and the mtu on the server and client. No change.

Client:

Settings for enp29s0:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	                        2500baseT/Full 
	                        5000baseT/Full 
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	                        2500baseT/Full 
	                        5000baseT/Full 
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: 5000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: g
	Wake-on: d
	Link detected: yes

Server:

Settings for enp2s0:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Advertised link modes:  100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Speed: 5000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: external
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: g
	Wake-on: g
	Link detected: yes

But I'm wondering if this has anything to do with it? Mode and status are not equal in speed.

ifIndex 	9
Port Type 	Normal
Port Channel ID 	Not a LAG member
Port Role 	Designated
STP Mode 	Enabled
STP State 	Forwarding
Admin Mode 	Up
LACP Mode 	Disabled
Physical Mode 	10000 Mbps Full
Physical Status 	1000 Mbps Full
Link Status 	Up
Link Trap 	Enabled
Octets Received 	656538914
Packets Received 64 Octets 	64979709
Packets Received 65-127 Octets 	101787144
Packets Received 128-255 Octets 	52967283
Packets Received 256-511 Octets 	21846058
Packets Received 512-1023 Octets 	14582797
Packets Received > 1024 Octets 	400375923
Total Packets Received Without Errors 	181170406
Unicast Packets Received 	169041868
Multicast Packets Received 	8178275
Broadcast Packets Received 	3950263
Total Packets Received with MAC Errors 	0
Jabbers Received 	0
Fragments Received 	0
Undersize Received 	0
Alignment Errors 	0
Rx FCS Errors 	0
Overruns 	0
802.3x Pause Frames Received 	0
Total Packets Transmitted (Octets) 	590400263628
Total Packets Transmitted Successfully 	475368508
Unicast Packets Transmitted 	469433573
Multicast Packets Transmitted 	3250005
Broadcast Packets Transmitted 	2684930
Total Transmit Errors 	0
Tx FCS Errors 	0
 Tx Oversized 	0
Total Transmit Packets Discarded 	0
Single Collision Frames 	0
Multiple Collision Frames 	0
Excessive Collision Frames 	0
802.3x Pause Frames Transmitted 	0
EAPOL Frames Received 	0
EAPOL Frames Transmitted 	3

Here's the only logs I could find.

18 Nov 2020 21:05:17 UTC+1:00%AAA-I-CONNECT: New http connection for user admin, source 192.168.1.127 destination 192.168.1.252 ACCEPTED
18 Nov 2020 20:18:22 UTC+1:00%AAA-I-DISCONNECT: http connection for user admin, source 192.168.1.109 destination 192.168.1.252 TERMINATED
18 Nov 2020 20:15:17 UTC+1:00%STP-W-PORTSTATUS: mg7 of instance 0: STP status Forwarding
18 Nov 2020 20:15:15 UTC+1:00%STP-W-PORTSTATUS: mg6 of instance 0: STP status Forwarding
18 Nov 2020 20:15:12 UTC+1:00%LINK-I-Up: mg7
18 Nov 2020 20:15:11 UTC+1:00%LINK-I-Up: mg6
18 Nov 2020 20:15:07 UTC+1:00%LINK-W-Down: mg7
18 Nov 2020 20:15:03 UTC+1:00%LINK-W-Down: mg6
18 Nov 2020 20:14:26 UTC+1:00%STP-W-PORTSTATUS: xmg9 of instance 0: STP status Forwarding
18 Nov 2020 20:14:21 UTC+1:00%LINK-I-Up: xmg9
18 Nov 2020 20:14:13 UTC+1:00%LINK-W-Down: xmg9
18 Nov 2020 20:14:08 UTC+1:00%STP-W-PORTSTATUS: xmg9 of instance 0: STP status Forwarding
18 Nov 2020 20:14:03 UTC+1:00%LINK-I-Up: xmg9
18 Nov 2020 20:13:59 UTC+1:00%LINK-W-Down: xmg9
18 Nov 2020 20:12:50 UTC+1:00%AAA-I-CONNECT: New http connection for user admin, source 192.168.1.109 destination 192.168.1.252 ACCEPTED
18 Nov 2020 19:38:21 UTC+1:00%SNMP-W-SNMPAUTHFAIL: Access attempted by unauthorized NMS: 192.168.1.127
18 Nov 2020 19:38:02 UTC+1:00%AAA-I-DISCONNECT: http connection for user admin, source 192.168.1.143 destination 192.168.1.252 TERMINATED
18 Nov 2020 19:30:24 UTC+1:00%COPY-N-LOGGINGFILECOPY: start log messages related to file copy operations
18 Nov 2020 19:30:22 UTC+1:00%COPY-N-LOGGINGFILECOPYSTOP: stop log messages related to file copy operations
18 Nov 2020 19:29:59 UTC+1:00%AAA-I-CONNECT: New http connection for user admin, source 192.168.1.143 destination 192.168.1.252 ACCEPTED
18 Nov 2020 19:07:14 UTC+1:00%SNMP-W-SNMPAUTHFAIL: Access attempted by unauthorized NMS: 192.168.1.143
18 Nov 2020 15:50:46 UTC+1:00%AAA-I-DISCONNECT: http connection for user admin, source 192.168.1.143 destination 192.168.1.252 TERMINATED
18 Nov 2020 15:39:27 UTC+1:00%AAA-I-CONNECT: New http connection for user admin, source 192.168.1.143 destination 192.168.1.252 ACCEPTED
06 Dec 2018 16:40:50 UTC+1:00%STP-W-PORTSTATUS: g1 of instance 0: STP status Forwarding
06 Dec 2018 16:40:45 UTC+1:00%LINK-I-Up: g1
06 Dec 2018 16:40:41 UTC+1:00%LINK-W-Down: g1
06 Dec 2018 16:40:23 UTC+1:00%STP-W-PORTSTATUS: g1 of instance 0: STP status Forwarding
06 Dec 2018 16:40:19 UTC+1:00%LINK-I-Up: g1
06 Dec 2018 16:40:15 UTC+1:00%LINK-W-Down: g1
06 Dec 2018 16:40:11 UTC+1:00%STP-W-PORTSTATUS: g1 of instance 0: STP status Forwarding
06 Dec 2018 16:40:06 UTC+1:00%LINK-I-Up: g1
06 Dec 2018 16:39:59 UTC+1:00%LINK-W-Down: g1
06 Dec 2018 08:14:30 UTC+1:00%NT_poe-I-Notification: PoE class mismatch detected On unit 1
06 Dec 2018 08:06:40 UTC+1:00%INIT-I-Startup: Cold Startup
06 Dec 2018 08:05:19 UTC+1:00%SYSLOG-N-LOGGING: Logging started.
06 Dec 2018 08:04:46 UTC+1:00%STP-W-PORTSTATUS: mg6 of instance 0: STP status Forwarding
06 Dec 2018 08:04:45 UTC+1:00%STP-W-PORTSTATUS: mg7 of instance 0: STP status Forwarding
06 Dec 2018 08:04:42 UTC+1:00%LINK-I-Up: mg6
06 Dec 2018 08:04:41 UTC+1:00%STP-W-PORTSTATUS: xmg9 of instance 0: STP status Forwarding
06 Dec 2018 08:04:41 UTC+1:00%LINK-I-Up: mg7
06 Dec 2018 08:04:40 UTC+1:00%STP-W-PORTSTATUS: g1 of instance 0: STP status Forwarding
06 Dec 2018 08:04:39 UTC+1:00%STP-W-PORTSTATUS: g2 of instance 0: STP status Forwarding
06 Dec 2018 08:04:36 UTC+1:00%LINK-I-Up: xmg9
06 Dec 2018 08:04:36 UTC+1:00%LINK-I-Up: g1
06 Dec 2018 08:04:35 UTC+1:00%LINK-I-Up: Vlan 1
06 Dec 2018 08:04:35 UTC+1:00%LINK-I-Up: g2
06 Dec 2018 08:04:32 UTC+1:00%NSFP-I-SFPGibicDetected: xg10 SFP port is present, module type - 10G BASE-SR
06 Dec 2018 08:04:32 UTC+1:00%LINK-I-Up: loopback1
06 Dec 2018 08:04:32 UTC+1:00%LINK-W-Down: xg10
06 Dec 2018 08:04:32 UTC+1:00%LINK-W-Down: xmg9
06 Dec 2018 08:04:32 UTC+1:00%LINK-W-Down: mg8
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: mg7
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: mg6
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: mg5
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: g4
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: g3
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: g2
06 Dec 2018 08:04:31 UTC+1:00%LINK-W-Down: g1
06 Dec 2018 08:04:30 UTC+1:00%NT_poe-I-Notification: PoE class mismatch detected On unit 1
06 Dec 2018 07:04:30 UTC+1:00%SNMP-I-CDBITEMSNUM: Number of startup configuration items loaded: 138
06 Dec 2018 07:04:30 UTC+1:00%SNMP-I-CDBITEMSNUM: Number of running configuration items loaded: 138
06 Dec 2018 07:04:28 UTC+1:00%MLDP-I-MASTER: Switching to the Master Mode.
06 Dec 2018 07:04:24 UTC+1:00%Entity-I-SEND-ENT-CONF-CHANGE-TRAP: entity configuration change trap.
06 Dec 2018 07:04:24 UTC+1:00%Environment-I-FAN-STAT-CHNG: FAN# 1 status changed to operational.
06 Dec 2018 07:04:19 UTC+1:00%INIT-I-InitCompleted: Initialization task is completed

The only test I have left is to move the client machine to the other location and plug in to the same switch.

 

Is there any speed test I can do from the switch? Iperf3 Switch to switch?

 

I appreciate any help you can offer.

BR/Mats

5 Replies

  • schumaku's avatar
    schumaku
    Guru - Experienced User
    Nov 19, 2020

    Two things hit my eyes:

     

    1. The server does not announce the 2.5G and 5G link modes - suspect this isn't a multiGig capable adapter and/or driver:

     

    Settings for enp2s0:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Advertised link modes:  100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Speed: 5000Mb/s

     

    2 The switch interconnect on that xg9 (the 10G port, #9) does link/negotiate on 1G only why ever (that's the obvious bottleneck) as you correctly spotted:

     

    ifIndex 	9
Port Type 	Normal
Port Channel ID 	Not a LAG member
Port Role 	Designated
STP Mode 	Enabled
STP State 	Forwarding
Admin Mode 	Up
LACP Mode 	Disabled
Physical Mode 	10000 Mbps Full
Physical Status 	1000 Mbps Full
Link Status 	Up

     

    This isn't about yet another speed test - it's about finding the issues: Adapter/driver resp. cable/patches...

     

     

    • matslundqvist's avatar
      matslundqvist
      Aspirant
      Nov 19, 2020

      It's a true multigig adapter, that's certain. I found an old mellanox 10G card with a twinax that I inserted into another server (and in SW01) in the offsite location, and the iperf3 test was up to 4,7gbit. I did the same with the client, on SW02. So I've excluded the cable, nic and the server as a cause (in both sites). - Which leaves the switch interconnect.

      Physical Mode 	10000 Mbps Full
Physical Status 	1000 Mbps Full

      So you mean that Physical Status reflects the actual speed?

      I checked the 5G port for the server (which I just benchmarked at 5G) and it reports the same:

      Physical Mode 	5000 Mbps Full
Physical Status 	1000 Mbps Full

      If the cable was bad, wouldn't there be a lot of transmission errors in the port details?

      And if the cable was bad, wouldn't it negotiate a lower speed (i.e physical mode)?

      Is there a way to "force" lower speed on the port?

      • schumaku's avatar
        schumaku
        Guru - Experienced User
        Nov 19, 2020
        matslundqvist wrote:

        Is there a way to "force" lower speed on the port?

        This is prohibited by the IEEE standard my friend - autonegoation is mandatory from GbE and up.

         

        Open a ticket with Netgear please.

NETGEAR Academy

Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology! 

Join Us!

ProSupport for Business

Comprehensive support plans for maximum network uptime and business peace of mind.

 

Learn More