× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: RN104 Incremental Backup to USB3

Concept4
Aspirant

RN104 Incremental Backup to USB3

My daily backup to USB seems to be taking forever recently, similar amount of data but it's just not finishing in time. I've hovered over the backup whilst it is running and the overview pop-up says 10.3mb/s... is that about right? This is an RN104 using USB3 back bottom port to USB3 HDD.

 

I've formatted the USB3 HDD and reset up the backup.

 

With the above, how can I setup incremental to USB3? I tried using some info I found on the net but it didn't work. Anybody any ideas?

 

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 1 of 23

Accepted Solutions
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

What do I need to use for the destination path if.... the hdd is named USB_HDD_16 and the folder I want to copy to is Digital?

 

The NAS I'm pushing/copying from has rsync enabled in the settings.

 

Again, thanks.

View solution in original post

Message 15 of 23

All Replies
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

A few hours in and the speed has gone up to 18mb/s.

Message 2 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3

  • What firmware are you running?
  • How is the USB drive formatted?
  • How is the backup job set up now? 

 

The best way to do an incremental backup is to use rsync.

Message 3 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Thanks for the reply. Info below...

 

FW 6.9.5

USB drive formatted to EXT4

Backup is set to daily, local share to usb... it removes contents and then does a full backup

 

It's now 7hrs in and is transferring at 22.3mb/s... it started mid-morning

Message 4 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

FW 6.9.5

USB drive formatted to EXT4

Thanks.  Personally I'd use NTFS, since I want to be able to access the backup from Windows.  But if you have other linux systems, EXT4 is a reasonable choice.  If not, I suggest you change to a different format.

 


@Concept4 wrote:

 

It's now 7hrs in and is transferring at 22.3mb/s... it started mid-morning


I think you must mean MB/s (megabytes, not megabits).  This is slow, but it could be a slow drive - particularly if it uses SMR.  What model is it?

 


@Concept4 wrote:

 

Backup is set to daily, local share to usb... it removes contents and then does a full backup

 


So that isn't incremental.  You can make it incremental by switching to rsync - making the source "remote" and using 127.0.0.1 for it's IP address. 

 

But it is wise to set it up to do a full backup every now and then.  I've had USB drives fail on me, and more than once I've found that old files on the drive looked ok, but couldn't be read.  Doing the occasional full backup should lower the odds of that.

Message 5 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Thanks again....

 

We don't access via Windows so always stuck to what was recommended on here many moons ago. Would be happy any format that is quick to be honest.

 

Yeah, MB/s not mbps.... sloppy typing.

 

The drive is a Samsung D3 Station

I've tried doing the rsync to 127.0.0.1 but I wasn't able to find the share. Our Nas has 3 shares and I only back 2 of the 3 as one is just for file sharing not storage. Will adding 127.0.0.1 do the entire NAS? Would be good if I can get this working.

 

Message 6 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

 

We don't access via Windows so always stuck to what was recommended on here many moons ago. Would be happy any format that is quick to be honest.


Let's start here: 

  • What would you do if your NAS failed, and you needed the backed up files?
  • How would you confirm that your files are actually being backed up?

@Concept4 wrote:

 

I've tried doing the rsync to 127.0.0.1 but I wasn't able to find the share. 

 


Well, you do need to enable rsync for the source shares.  You don't need to browse for them.  There is a guide here: https://kb.netgear.com/29741/How-do-I-back-up-data-between-two-ReadyNAS-OS-6-systems-using-the-backu...

 


@Concept4 wrote:

Will adding 127.0.0.1 do the entire NAS? 

 


127.0.0.1 is a special IP address that is also known as localhost.  It's always the local machine.  What is backed up depends on the other backup job settings.  

 

If you want to back up the entire NAS, you'd need to configure the backup job with a local source, and make the USB destination remote.  So the USB share would need to have rsync enabled (read/write) for that to work.  Personally I like one backup job per share myself.

 


@Concept4 wrote:

The drive is a Samsung D3 Station


Ok.  That model is too small to be SMR.

 

Still, it doesn't seem to be all that fast.  https://www.scan.co.uk/products/4tb-samsung-stshx-d401tdb-d3-station-usb-30-20-external-desktop-hard...

 

If you do the math on the two measurements in the above review, you'll get about 25 MB/s on the small file test, and about 57 MB/s on the large file test.

Message 7 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Thanks some good info here, many thanks for the help.

 

We have a live NAS which backs up to a backup NAS, i then use the USB3 drive as another backup from the backup NAS. Our director takes this home, we swap between a few external drives so we have a 36 hr backup. I only access these USB3 drives via the readynas rather than direct but direct access could be helpful. The USB3 drive helps in case of fire.

 

Is there a way of encrypting the ext HDDs?  Although they may not be used soon, we've got new premises linked via fibre so I could have a live nas in one location and the backup in the other location. Just thinking, should I mirror live to backup and then continue backing up to USB3 to have a 24 hr log, that could be a good setup.

 

I'm going to give this a bash using the 127.0.0.1, I've always preferred backing up shares individually so will continue. If one fails the others continue etc and when accessing the backup they are in a similar order.

 

Again thanks for the help. Much appreciated.

 

 

Message 8 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

We have a live NAS which backs up to a backup NAS, i then use the USB3 drive as another backup from the backup NAS. Our director takes this home, we swap between a few external drives so we have a 36 hr backup. I only access these USB3 drives via the readynas rather than direct but direct access could be helpful. The USB3 drive helps in case of fire.

 


Thx for the info.  NTFS would give you direct access - performance is slightly slower than EXT, but not much.

 


@Concept4 wrote:

Just thinking, should I mirror live to backup and then continue backing up to USB3 to have a 24 hr log, that could be a good setup.

 


That could also work well. 

 


@Concept4 wrote:

 

Is there a way of encrypting the ext HDDs? 

Unfortunately no.  It would be a nice feature.  If you have an application server, you could connect the USB hard drives to that, and use a backup program that has that feature (and maybe also get versioning).

Message 9 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

I like the thinking with the server, we have a DC server I could use to do this. Will check that out. Never thought of that, always used the NAS.

 

Thanks for all the help.

Message 10 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Just tried a backup using 127.0.0.1 as the Host but it failed.

 

On the USB HDD I have 3 folders for the 2 shares on the backup NAS. I'm trying to backup one share and put them in the relevant folders.

 

Any ideas on the connection issue?

 

Attached my settings, I might have done something wrong here...

Message 11 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3

Is rsync enabled for the Digital share?

Message 12 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Yeah i have rsync on, added a screenshot.

 

Thanks

Message 13 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

Yeah i have rsync on, added a screenshot.

The sharename needs to be in the path, and it's not.  You only have the subfolder.

 

And is rsync also on for the source share?

Message 14 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

What do I need to use for the destination path if.... the hdd is named USB_HDD_16 and the folder I want to copy to is Digital?

 

The NAS I'm pushing/copying from has rsync enabled in the settings.

 

Again, thanks.

Message 15 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

I think I have done it, I've used... USB_HDD_16/Digital

Message 16 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Looks like I'm all setup, I missed the link to the guide you supplied yesterday.

 

Thanks for all the help on this.

Message 17 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3

Good.  If you go onto the advanced options tab, you'll see an option called "remove deleted files on target".  That will delete files on the USB drive that you previously deleted on the NAS.  

 

If you want the USB folders to always match the NAS shares when the backup is completed, then you'll want to check that box.  The scenario here is recovering from a NAS failure - when you just want to restore the shares on the NAS.

 

But if you want use the backup to retrieve files that you accidently deleted,  then you won't want to check it.

 

Personally I check it, and use snapshots to recover from accidental deletion.

 

 

Message 18 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Thanks Stephen... yeah I have that option ticked on the USB.

 

At the minute we have a live NAS which is then backed up via rsync to our backup NAS, this is every night. It works well but we are 8 hrs out of sync. Then we backup to USB3 HDD which is then took offsite, 24hrs out of sync but it's a safety net.

 

Our live nas is an older model, NV+ v2 and the backup in the Rn104.

 

I was thinking of mirroring live to backup or rsyncing more frequently. Wasn't sure the best setup with the NAS's we have, what would you recommend?

 

I was then thinking daily snapshots on the backup NAS so if something goes missing we have 28days of snapshots.

 

I was then going to backup the backup NAS to USB so we have an extra copy in case of complete disk failure. Does this sound ok or is it over kill.

 

I've never used snapshots to retrive a file, can you just access individual files without loading an entire revision of all files?

 

Message 19 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Sorry for another message, was just looking at snapshots...

 

If we did the mirror, live nas to backup nas... am I right in thinking we could setup snapshots on the backup every 2 hours between 8am/6pm (office hours) and then setup to just save for a week etc. This would give us a weekly backup with stages to access existing data.

 

Our work is fast turnaround so we usually know about missing files within a day or we may need to revert to older files if a client decides 'actually I prefer what you did from day 1'. We do save old artwork files to help but sometimes human error just kicks in.

 

Then if we backup the backup nas we have the same saved data on a single USB HDD incase of nas/drive failure.

Message 20 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

 

I was thinking of mirroring live to backup or rsyncing more frequently. Wasn't sure the best setup with the NAS's we have, what would you recommend?

 


You won't get full coherency if you do "pull" back ups (running on the RN104) during working hours.  You might not get it if you do "push" backups either - I don't own an NV+ v2, so I have no hands-on experience with 5.x firmware.

 

You might also find that some backups fail (due to locked files for instance).

 

Performance will drop some when a backup is running - which might be a consideration with a v2 NAS (which doesn't have a lot of memory, and has slow processor).

 

Coherency might not matter (particularly if you don't have live databases on the main NAS).  

 

You could certainly up the rate from the NV+ v2 -> RN104 to hourly and see what happens. 

 


@Concept4 wrote:

I was then thinking daily snapshots on the backup NAS so if something goes missing we have 28days of snapshots.


That's what I do - using custom snapshots.  That lets you set retention.  You can balance the retention limit to keep the space usage reasonable.

 


@Concept4 wrote:

I've never used snapshots to retrive a file, can you just access individual files without loading an entire revision of all files?


Yes.  You can browse into a snapshot from the admin web ui, and restore an individual file.

 


@Concept4 wrote:

I was then going to backup the backup NAS to USB so we have an extra copy in case of complete disk failure. Does this sound ok or is it over kill.

I don't think it's overkill.  Personally I keep at least three copies of everything.  I used to be ok with a single backup, but I sometimes found (the hard way) that some files on the backup couldn't be read.  FWIW, this was pre-NAS.  I haven't lost data since I switched to the three copy approach.

 

Ransomware is another scenario where a USB copy would be needed - especially if you up the rate on the NV+ v2->RN104 backup jobs.  Snapshots can help recover from ransomware, but you need a lot of free space.  The RN104 will start deleting snapshots if the volume reaches 90% full.  To guarantee that it won't do that during a ransomware attack you'd need ~60% free space on the RN104 (since all the files will be deleted/encrypted on the main NAS, the snapshot space will rise sharply).

 

 

Message 21 of 23
StephenB
Guru

Re: RN104 Incremental Backup to USB3


@Concept4 wrote:

If we did the mirror, live nas to backup nas... am I right in thinking we could setup snapshots on the backup every 2 hours between 8am/6pm (office hours) and then setup to just save for a week etc. This would give us a weekly backup with stages to access existing data.

 


That should work too.  The space usage of the snapshots depends on the "churn" - how often files are deleted, modified, or renamed.

 

There's no way to sync the snapshots with the rsync updates, so they will sometimes be taken in the middle of a backup run.

 

Another option is to upgrade your main NAS to an RN214 or RN424, and put the snapshots on there.  Then you could re-purpose the NV+ v2 as a secondary backup (ideally placing it offsite).  Then you could probably eliminate the USB drives.

Message 22 of 23
Concept4
Aspirant

Re: RN104 Incremental Backup to USB3

Thanks again Stephen, as always great help.

 

I think we kept the RN as the backup as it USB3 and I never got rsync working.

 

I think I'll do as you say, swap the NAS boxes around.

 

Message 23 of 23
Top Contributors
Discussion stats
  • 22 replies
  • 3404 views
  • 4 kudos
  • 2 in conversation
Announcements