× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Device starts dropping pings during backups

Rburn99
Aspirant

Device starts dropping pings during backups

I have 2 Pro 6 business nas units. One is the central location for our vms and the other is for backups.

Firmware: 4.2.27

X-raid2

6 2tb drives

 

Under regular use and when idle pings are solid <1ms without a hiccup. But during intensive read/write operations (backup jobs), the device starts dropping pings. Once several pings in a row get dropped the jobs jail with Device Not Found type errors. The read/write operations are to CIFS shares. Backup jobs were from vmware vdr and vdp appliances and now from Veeam Backup & Replication

Model: ReadyNASRNDP6310|ReadyNAS Pro 6
Message 1 of 13
mdgm-ntgr
NETGEAR Employee Retired

Re: Device starts dropping pings during backups

There is a 4.2.28 firmware release now available.

How many backups are you running at once? Have you staggered your backup jobs?

Message 2 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

Hello,

 

Just one backup. I run a ping logger.. everything is fine. I manually start a backup job... once the job begins transfering data pings start dropping. I may get 10-20 minutes into the job with one or two dropped pings every so many seconds but once it misses three in a row the backup job throws in the towel.

Message 3 of 13
StephenB
Guru

Re: Device starts dropping pings during backups

Is this happening on both NAS?

 

Are you using NIC teaming?

 

Also, are you seeing errors in the ethernet stats on the two NAS?

Message 4 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

The other nas is running fine. But it also is not under the load of transferring hundreds of gigs worth of data plus whatever instructions go on during a backup job. I do not use nic teaming. I wanted to try that but having never done it before and because I only have the production environment to work with I am hesitant to experiment. Yes there are errors listed but the vm nas (not causing any disconnect issues) seems to have far more.

 

Backup NAS (drops pings under load of backup job)

TX packets3094012358
TX errors0
TX dropped0
TX overruns0
TX carrier0
RX packets2137332576
RX errors180867
RX dropped394264
RX overruns0
RX frame180867
Collisions

0

 

 

VM NAS (not causing any noticable issues)

TX packets11032907002
TX errors0
TX dropped0
TX overruns0
TX carrier0
RX packets7154619993
RX errors4294967295
RX dropped3290316
RX overruns0
RX frame4294967295
Collisions0
Message 5 of 13
StephenB
Guru

Re: Device starts dropping pings during backups

I think you do need to resolve the high rx error rate on the VM NAS.

 

Just to clarify - these backups are being done from a third server?  So data is being read from the VM NAS over the network, and then being written to the backup NAS over the network?

Message 6 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

Yes for the most part. I have a couple physical machines that are just backing up a few folders to the backup nas also. My personal physical pc is designated as the 'backup server' in veeam and the vm's from the vm nas are backed up to the backup nas. Before that a vm from vmware running on the vm nas was backing up the vms to the backup nas. We constantly had problems with it and decided to try a third party solution (veeam). For the most part it works but advanced functions like synthetic backups where it reads the incrementals from the backup nas and recreates the full will not work. The errors are always due to the backup nas disappearing from the network (ie: too many consecutive dropped pings)

 

How would I resolve high rx error rates?

Message 7 of 13
StephenB
Guru

Re: Device starts dropping pings during backups

If you have a managed switch, then look at the stats there as well (and if possible look at the stats along the path between your desktop PC and the VM NAS, as well as between the VM server and the NAS).  

 

If it is practical try changing the ethernet cable(s) connecting the VM NAS to the switch.  They should be cat 5e or better.

 

Are you teaming the two NICs in the NAS?  If so, what method are you using?

 

Also, are you running pingplotter from your desktop PC, or a different device? 

 

Message 8 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

All switches on the network are Netgear GS716T

I am using premade cat6 cabling. I have spools of cat6 so I could definitely make up my own but I am doubtful that is the problem.

I am not using nic teaming.

I ran pingplotter on multiple machines cross checking to make sure when a device stops responding I could be sure which end was responsible.

There are no errors to speak of on the switch itself.

 

InterfaceTotal Packets received without ErrorsPackets received with ErrorsBroadcast Packets receivedPackets transmitted without ErrorsTransmit Packet ErrorsCollision FramesTime since counters last cleared 
g120787278320855649313305069710017 day 6 hr 57 min 3 sec
g21201664787028954913486190400017 day 6 hr 57 min 4 sec
g37594382118755723376150017 day 6 hr 57 min 4 sec
g400000017 day 6 hr 57 min 4 sec
g511372450603021167821117667700320017 day 6 hr 57 min 4 sec
g6141011358740017 day 6 hr 57 min 4 sec
g73638806231240641267056230017 day 6 hr 57 min 4 sec
g82865370141311506050017 day 6 hr 57 min 4 sec
g926036187536045237064309281356260017 day 6 hr 57 min 4 sec
g107494890141343160630017 day 6 hr 57 min 5 sec
g111110100652534283672395143080017 day 6 hr 57 min 5 sec
g1210604968032034103105490603210017 day 6 hr 57 min 5 sec
g133430683645065350132992035930017 day 6 hr 57 min 5 sec
g1400000017 day 6 hr 57 min 5 sec
g155660641295102720017 day 6 hr 57 min 5 sec
g1600000017 day 6 hr 57 min 5 sec
PORTSLAGSAll
   
 
Message 9 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

I did just notice that maximum frame size on the nas connected ports are 1518. Must have forgotten to set that to 9000 again when I replaced the switch a while ago in attempts to address these issues.

Message 10 of 13
StephenB
Guru

Re: Device starts dropping pings during backups


@Rburn99 wrote:

I did just notice that maximum frame size on the nas connected ports are 1518. Must have forgotten to set that to 9000 again when I replaced the switch a while ago in attempts to address these issues.


Is this on the backup NAS or the VM NAS?

 

It does make sense to have everything use the same MTU, so maybe fix that first.

Message 11 of 13
Rburn99
Aspirant

Re: Device starts dropping pings during backups

Both.. all 16 ports on the switch were 1518. I have set the two nas box (vm and backups) to 9216. I don't want to run the backup during work as I don't want to field a bunch of why is my report taking so long phone calls 🙂 I don't know if this will fix things either though as I had swapped out the switch to address these issues (although at the time it was with vm's backup appliance periodically failing). And that switch I know I had set up with jumbo frames. Is there a way to reset the error count on nas boxes, obviously without doing factory resets 😛

 

And to my original questions.. Would maxing out the processing power of these readynas units be a likely cause for the units to stop responding to ping requests by connecting devices? And would maxing out the processing power to such an extent be a likely condition to begin with?

 

Message 12 of 13
StephenB
Guru

Re: Device starts dropping pings during backups

The switch configuration is the ethernet (layer-2) max frame size, the NAS configuration is the IP payload (layer 3), which is smaller.  The PC configuration is also usually the IP payload.  So 1500 in the NAS corresponds to 1518 in the switch.  If you use vlans, 1500 in the NAS corresponds to 1522 in the switch. 

 

I'd set the NAS (and the PC and the VM server) to use 1500 for now.  Alternatively you could set them to 9000 (with switch at 9022 or more).  

 

I don't think you can clear the counts with 4.2.x firmware - what you can do is copy them to a spreadsheet, and then subtract.

 

 

 

 

 

 

 

 

 

 

 

Message 13 of 13
Top Contributors
Discussion stats
  • 12 replies
  • 2377 views
  • 0 kudos
  • 3 in conversation
Announcements