× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: RN102 USB backup speed usb3 OS6.10.3

piet8stevens
Aspirant

RN102 USB backup speed usb3 OS6.10.2

I have bought new backup HDD (WD Elements) for my RN102. Started backup of some 3TB but only getting speed of 12MB/s. HD is formatted EXT4.

I saw a thread from 2016 talking about slow USB3 backup speeds, related to NTFS drivers for OS version 6.4 and 6.5.

 

Any ideas what I can do to increase this backup speed?

Thx.

Message 1 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.2


@piet8stevens wrote:

 

Any ideas what I can do to increase this backup speed?

First of all, use the rear USB ports instead of the front one.  The rear ports are USB 3.0, the front is USB 2.0

 

You might also want to connect the disk to a PC (reformatting it of course), and then testing disk performance there.  Maybe try to copy the same folder over an ethernet connection, and see what speed you get.

Message 2 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Hello, thank you for our reply.

The speeds I reported are from the back USB3 ports. I had the RN102 reformat the HDD to ext4.

When I connect the drive directly to my laptop (windows 10) via USB3, I get copy speeds between 10 and 210 MB/s, depending on file size. Here, the hdd is formatted NFS. In total, 6,305 items were copied, for a total of 64 GB in roughly 6 minutes (I looked at the PC clock, did not time the seconds), ie, roughly 180 MB/s. 

 

I copy from an SSD on the laptop.

On the NAS, I have WD Red drives in raid 1. The HDD I copy to is a WD green one.

 

So, on the laptop, the speed is roughly 18x faster than on the NAS. That seems to be quite a factor!

Model: RN102|ReadyNAS 100 Series 2- Bay
Message 3 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.2


@piet8stevens wrote:

In total, 6,305 items were copied, for a total of 64 GB in roughly 6 minutes (I looked at the PC clock, did not time the seconds), ie, roughly 180 MB/s. 


This was from the NAS?  I'm guessing not, because the speed is too high there:

  • a gigabit ethernet connection can't carry more than 120 MB/sec with any NAS
  • The RN100 series isn't fast enough to deliver more than about 75 MB/s over gigabit ethernet.

So something isn't quite right with the measurement.

 

Assuming it was a local copy, I'd try doing a copy from the NAS to the USB drive, and see what performance you get.  You could also try NasTester ( http://www.808.dk/?code-csharp-nas-performance ).  The idea here is to check the performance of the RN102 itself.

 

FWIW, I don't recommend using ext formatting for backups unless you have another machine that supports ext.  A backup you can't read isn't very useful.

Message 4 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Thank you for your reply.

 

The 6,305 items copies were from the laptop to the backup drive connected directly to the laptop via usb3. Speed is roughly 180MB/s, i.e., a measure for what is possible over usb3, as per the recommendation in your first reply.

 

Using the same drive from the RN102 to the backup drive connected direclty via a USB3 on the back, it gives the 10-12MB/s

 

So, the question is why the usb3 on an RN102 for a direct copy is so slow.

Message 5 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.2


@piet8stevens wrote:

 

So, the question is why the usb3 on an RN102 for a direct copy is so slow.


Yes.  And one possible reason is that the RAID array is underperforming.  That's why I suggested that you test the transfer speed of the NAS itself.

Message 6 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Sorry, you are right - I should have run the test first before replying.

From my Windows 10 laptop, I have run the test over my LAN.

These are the results using the default file size and iterations (400 and 5 respectively):

Iteration 1: 11.61 MB/sec

Iteration 2: 10.70 MB/sec

Iteration 3: 10.60 MB/sec

Iteration 4: 11.46 MB/sec

Iteration 5: 11.06 MB/sec

Average (W): 11.09 MB/sec

 

Iteration 1: 10.54 MB/sec

Iteration 2: 10.10 MB/sec

Iteration 3: 10.57 MB/sec

Iteration 4: 10.80 MB/sec

Iteration 5:  9.89 MB/sec

Average (R): 10.38 MB/sec

 

So, it seems it is an issue with RAID read/write times, rather than a usb3 issue. Thank you for providing advice and the link to the testing software.

What can I do about RAID read/write times?

 

Note: re. my LAN, I used iperf3 to check speed across the network to the NAS, and had about 750-760 MB/s between my desktop and the 2 RN102s, so the speed is not restricted by the LAN. I report between desktop and NAS because I run Ubuntu on my desktop and I have not installed iperf3 on my windows laptop. but the desktop and laptop are connected to the same switch on the network.

 

 

Message 7 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.2


@piet8stevens wrote:

 

So, it seems it is an issue with RAID read/write times, rather than a usb3 issue. Thank you for providing advice and the link to the testing software.

What can I do about RAID read/write times?

 


It does look that way.  Since you do have some familiarity with linux, you could check the RAID read speed directly on the NAS with ssh, using dd to copy to /dev/null.  

 

You could also look at the disk health - perhaps using smartctl -x.

 

If you've never run a balance on the volume, then I do suggest doing that.

Message 8 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Thank you for the suggestions and continued support. I am learning a lot.

1. via ssh:

1a. dd

- on home directory: write speed of 83.6MB/s, read 132MB/s (after clearing cache)

- on raid: write speed 9.4MB/s, read 21.7MB/s (after clearing cache)

- usb3 hdd: write speed 75.7MB/s, read 103MB/s (after clearing cache)

1b. smartctl: I installed it on the NAS, but when I try to run it on the raid, it asks for a type to be specified (-d option). Which type should I specify?

2. balance: started on the local admin page.

 

 

Message 9 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Additional info:

1. after balance, write speed on the NAS raid (with dd) has increased to 16.7MB/s. Read speed is still the same at 21.4MB/s. (raid)

2. I also did a hdd speed check on the hdd connected via usb3 to my Windows laptop: write speed (dd) is 215MB/s. read speed 2.7GB/s (!)

 

Message 10 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.2


@piet8stevens wrote:

 

1b. smartctl: I installed it on the NAS, but when I try to run it on the raid, it asks for a type to be specified (-d option). Which type

 


It should have been already installed.

 

On options, smartctl -a /dev/sda will give you the smart stats for the first drive.  It is also possible to test the drives with --test=long (though you can also do that from the volume settings wheel in the GUI).

 

Personally I like -x.  That adds in an error log (among other things), which did find some issues with one of my WD60EFRX drives.  But the output is long.

 

I'd also download the log zip file from the web ui, and look for btrfs or disk errors in the system.log and kernel.log.  Note you can also see that stuff with journalctl in ssh.  Though if you do that you'll want to use a filter.  For instance

# journalctl --no-tail -r | grep -i btrfs

 

Also, how full is the data volume?

 


@piet8stevens wrote:

 

1a. dd

- on home directory: write speed of 83.6MB/s, read 132MB/s (after clearing cache)

- on raid: write speed 9.4MB/s, read 21.7MB/s (after clearing cache)

 

after balance, write speed on the NAS raid (with dd) has increased to 16.7MB/s. Read speed is still the same at 21.4MB/s. (raid)

 


This is a little curious.  Do you mean /home by "home directory"?  That actually is mount point on the data volume.

If you mean / then that is it's own volume - but it is also RAID-1.

 

 

If you have a spare disk, you could try doing a factory install with just that disk in place, and measure the speed after setup.  Best to label the real disks by slot as you remove them (powered down) so you can restore them to the same slots later on.

 

If that gives you the expected speeds, then if both operational disks are healthy, then I'd suggest is to do a factory default with the operational drives in place (painful), rebuild the NAS, and restore the data from backup.  The idea there is to get a clean file system. 

 

 

Message 11 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

Thank you for the additional suggestions. I just got 2 new WD Red HDDs and will try as you say. Am pretty busy so report back not before Saturday.

Message 12 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

Hi, I was finally able to installl the new WD 6TB hdds in the NAS but after 3 hours of trying, am at my wits end. I have already done 2 factory resets and an OS reinstall from the boot menu, but each time, the NAS insists on resyncing my HDDs, which are empty. I searched internet and they should be reformatted by the factory reset so I do not understand why this resyncing is happening. After these factory resets, I also had a problem with the admin password, which is why I had to to an OS reinstall.

Is there any way to abort/avoid the resyncing? I also tried DESTROY on the volume, FORMAT on the hdds (which failed for the one in the #2 slot with a message about the disk may be part of an active volume or may contain the last system OS mirror).

Whatever I do, it seems I just get more delay. Resyncing takes something like 22 hours.

Any help would be much appreciated.

Message 13 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

<deleted>

Message 14 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.2

I tried one more thing: I reformatted the 2 HDDs on my PC (to BTFRS or something like that), then did another factory restore in the NAS, had to go to another OS reinstall to get the password right. The a FORMAT COMPLETE (or something like that), create new volume. And then, it started another resync, 73 hours.

 

And I tried one more: deleted partitions on my PC, no format. Then HDDs back in NAS and restart NAS. This time, no OS reinstall needed. But after new volume created -> resync, 59 hours.

 

Please help me.

Message 15 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.3


@piet8stevens wrote:

 but each time, the NAS insists on resyncing my HDDs, which are empty. I searched internet and they should be reformatted by the factory reset so I do not understand why this resyncing is happening.


This is normal - the NAS is building the RAID array mirror.  This requires syncing disk 2 to disk 1.

 

The fact that there is no data in the file system isn't relevant.  The RAID array is a virtual disk, and all blocks need to be synced (whether used or not).

Message 16 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

Thank you for your reply. But I am afraid I still do not understand. I set up these 2 HDDs also as a soft raid1 on a raspberry pi 4 (using madadm) in order to do some testing and there, this was not needed.

Also, as I was typing this, the Nas prompted me for my admin password again, and started the setup wizard again, based on new HDD detection. I had not done anything. And now, at the end of the wizard, I get an error that it failed to set admin password.

Message 17 of 27
Sandshark
Sensei

Re: RN102 USB backup speed usb3 OS6.10.3

Then the RasPi probably used the "assume clean" option of MDADM RAID creation.  That's not really a good idea except when using it for recovery processes.

 

Just let it do it's job.  While access is slower, you do still have access during sync.  Had you not kept re-starting, it might have been done by now.

Message 18 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

OK, thank you for your patience. After 22 hours of resyncing, I was able to test as follows:

yyy# sync; dd if=/dev/zero of=tempfile bs=1M count=1024;sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.8596 s, 121 MB/s – <= write speed

yyy# /sbin/sysctl -w vm.drop_caches=3

vm.drop_caches = 3

yyy# sync; dd if=tempfile of=/dev/null bs=1M count=1024;sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.24733 s, 172 MB/s <= read speed

So, much better than before (like 10x). I am not sure why this is so, but will now proceed to copy back the backup via usb3 to see what speeds I can get there and report back. Once I have completely restored the NAS content, I can then test on the old hdds to see whether if I clean them and resync, they provide similar performance to the new hdds.

 

I appreciate all the support I have received here. Thank you.

 

Unfortunately, when I wanted to shut down the system, it said I would lose all settings as no volume has been created. I do not understand as I had created a volume and I can write on it. What to do?

Message 19 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

OK, good news: after 22 hours of resyncing, the NAS is up and running. I enabled ssh to test speed with this result:

yyy# sync; dd if=/dev/zero of=tempfile bs=1M count=1024;sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.8596 s, 121 MB/s – <= write speed

yyy# /sbin/sysctl -w vm.drop_caches=3

vm.drop_caches = 3

yyy# sync; dd if=tempfile of=/dev/null bs=1M count=1024;sync

1024+0 records in

1024+0 records out

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.24733 s, 172 MB/s <= read speed

 

So, 10-12x faster than before. I will now first restore the data from usb3 hdd backup and then re-initialize the old hdds and check the speed on those, newly installed. I will report back on those speeds, too.

 

I really appreciate all your help in getting me this far. Thank you!

 

Unfortunately, I do have another problem now: I wanted to shut down the system to move it to its usual location, but got a warning I would lose all my settings as no volume had been created. But I did create a volume. How to get around this problem?

OK, I found the solution: I had not changed the admin password. Once I did so, the warning went away.

 

Message 20 of 27
Sandshark
Sensei

Re: RN102 USB backup speed usb3 OS6.10.3

There are actually a number of things that don't work correctly if you don't change the admin password.  Netgear does a poor job (do they say it anywhere?) documenting this, and none of the error or warning messages tell you that's the reason.

 

Glad you figured it out and let others know of that restruction.

Message 21 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

Unfortunately, after all this time, I am back at the original problem: with newly formatted, freshly resynced new HDDs, the restore speed from usb3 hdd plugged in at the back in a usb3 port is running at a leisurely 28MB/s. Faster than at the start of all this work, but not dramatically so.

 

Anything else I can do? Is this a normal speed? Compared to usb3 spec and hdd speed as measured (121MB/s), it seems low.

 

I tried but failed to connect the usb3 HDD to my Ubuntu PC and back up over the network to see whether that would be faster. Each time, it failed with a connection lost during file copy, although my ssh from my PC to the NAS had no issues.

Message 22 of 27
StephenB
Guru

Re: RN102 USB backup speed usb3 OS6.10.3


@piet8stevens wrote:

Unfortunately, after all this time, I am back at the original problem: with newly formatted, freshly resynced new HDDs, the restore speed from usb3 hdd plugged in at the back in a usb3 port is running at a leisurely 28MB/s. Faster than at the start of all this work, but not dramatically so.

What protocol are you using for the restore?

Message 23 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

Sorry for not being complete. I used the backup in the GUI from source local share usb_hdd_1 path to destination local share: path. At this speed, the restore will take something like 15 hours (1.5 TB).

Message 24 of 27
piet8stevens
Aspirant

Re: RN102 USB backup speed usb3 OS6.10.3

Update: avg speed is now 19.8MB/s, 23h48' running time for 1.6TB transferred. Almost done. Question remains: how to accelerate this? Is this normal for usb3 with a disk write speed of about 121MB/s?

Message 25 of 27
Top Contributors
Discussion stats
  • 26 replies
  • 2178 views
  • 8 kudos
  • 3 in conversation
Announcements