× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Add option for FrontView Rsync to verify file copies

jmoomaw
Aspirant

Add option for FrontView Rsync to verify file copies

RE: viewtopic.php?f=4&t=57199&p=323193

Replicating terabytes of data between NAS boxes, and especially USB-drives to NAS carries a reasonable risk of corrupted bytes. Apparently there are no FrontView backup options for doing copy verification. Please provide a checkbox option on FrontView backup jobs to perform data verification, esp for Rsync.

Based on dialog in above topic this option isn't turned on and there's no simple way for users to enable this. I'm surprised this isn't already enabled given most backup software offers this option, esp if the Rsync installed on the ReadyNAS natively has that capability.

I'm in process of building a 2nd ReadyNAS to install at a remote location for disaster recovery, and need to copy over terabytes of information from the master to it before I enable the Replication application. Apparently my plan to use Rsync is little different than doing a drag&drop from a Windows client between the two machines. I'm now considering running CRC-capable sync-software to validate the copied files before putting into production, although I have no idea how long this comparison will take. Alternative is to use a Windows client along with 3rd-party file syncing application for doing all file copies. Only testing will tell which is faster.

Further I'm quite certain my ISP isn't going to tolerate the NAS boxes syncing more than tens of gigabytes per month between machines (if that much) so the big-volume changes need to be done using USB drives. But I'm left with the same corruption vulnerability... if I use current version of FrontView Rsync to build the outbound disk, and Rsync to read the inbound disk, I have third generation files on NAS with no confidence they are valid copies. Of course, when the day comes I need to copy them back the problem compounds. The whole mission of the backup NAS box is compromised if there's no practical means to create verified file copies. I'd prefer the NAS machines could do this w/o me needing external computers and applications for verified file moves.

Regarding the issue of a content mismatch for existing files (destination file doesn't match content of source) which would otherwise be considered the same (date and size match), it would be good to have the option of at least finding and flagging those. The problem would probably be that you wouldn't know which one was correct without further investigation (either one is subject to media corruption) where copying a corrupted source file isn't what you'd want to do.

I haven't looked to see if the ReadyNAS Replication product has any kind of copy verification option, but if it doesn't I'm back to the same problem with that app, and not having confidence in my backup machine. Trying to do CRC-checking of source & destination files is probably not practical in this scenario.
Message 1 of 4
StephenB
Guru

Re: Add option for FrontView Rsync to verify file copies

Doesn't Rsync already do this (http://en.wikipedia.org/wiki/Rsync)??? It seems to me that all you need to do is run it again, and there would be no files copied if everything verified.
Message 2 of 4
jmoomaw
Aspirant

Re: Add option for FrontView Rsync to verify file copies

Please read the thread in the link at the top of the request... the content verification switch isn't turned on, it's only using size and dates to recognize a difference in the files. This isn't strictly a robust comparison. It appears the Rsync app on the NAS may have the capability to do verification at copy time but the FrontView interface doesn't have the user lever to set it. Once the copy is made it would be nice if Rsync could do a content comparison and fix problems found over time. Blocks and bits do fail.
Message 3 of 4
StephenB
Guru

Re: Add option for FrontView Rsync to verify file copies

Wow. I agree with you - I see little value in using rsync over incremental NFS given the way this is described. Certainly the idea of starting with NFS and switching to rsync seems questionable.

Given that the disk speed is generally faster than the network, it would be useful to have both ReadyNAS be able to generate / compare hash codes on the files in the share, and create a list of differences. Even if an add-on could only generate the full list, you could use a PC to compare the lists.
Message 4 of 4
Top Contributors
Discussion stats
  • 3 replies
  • 3898 views
  • 0 kudos
  • 2 in conversation
Announcements