NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.

Forum Discussion

BennTech's avatar
BennTech
Aspirant
Aug 14, 2014

Backup job copies ALL files instead only changes

I have a ReadyNAS NVX running 4.2.26. It has a share used for our ESXi server backups. I then have a ReadyNAS backup job that runs everyday and copies the backups from the NVX share to a USB drive attached to the NVX. The USB drive gets swapped weekly (or when my client remembers, which is more like monthly) among a set of 3 drives and stored off-site.

My problem is that our daily incremental backups are only 5-10GB, but instead of copying just these new files, the job copies ALL 2TB of old backup files EVERY DAY. And, yes, I DO have "Schedule full backup: First time" and no, "Remove the contents of the backup destination" is NOT checked, and yet the job still copies ALL files every day.

I can watch real-time in a folder comparison utility as the job re-copies existing files in seemingly random order. This is killing by backups--taking over 200-400x longer to complete and wearing out my drives 200-400x faster. Help!

15 Replies

Replies have been turned off for this discussion
  • ESXi attaches the ReadyNAS share as a store via NFS, then vmProtect writes to the ESXi store.

    FIXED! (Sorta.) I am happy to report that rsync fixes the problem--rsync copies only the missing files and ignores the granularity difference between EXT to NTFS. And StephenB is correct on username/password--none needed. This works on both NVX v4.2.26 and NV+ v4.1.6.

    Big downside is that rsync writes the truncated time (nearest second) to the destination time. Be nice if it used that for comparisons only but copied the FULL time (let the destination truncate to its highest granularity). E.g., I've previously used the non-rsync method to copy files without issue from NTFS system to ReadyNAS then ReadyNAS to NTFS USB. Works fine because the granularity was already truncated by the original NTFS system to 7 decimals. However, if I try the same with rsync, the copies get truncated to the nearest second, and their timestamps no longer match the original. Now if I compare timestamps with something other than rsync (which isn't included in Windows...see illiterate user comment above)--or even rsync but with the default, untruncated granularity--the timestamps don't match and I've got the same problem again. Argh! I guess that's why my clients pay me the big bucks to bang my head against the desktop over problems like this so they don't have to...hmmm...maybe they're not such stupid users after all. :)
  • StephenB's avatar
    StephenB
    Guru - Experienced User
    Thanks for providing your resolution.

    On the windows issue, robocopy has quite a few options, some of which might help.
  • Robocopy is my go-to copy app because it's one of the few that accurately copies all file attributes, and even works on folder timestamps, which virtually every other copy program neglects. It has an /FFT switch with 2 second granularity.

    However, robocopy has similar issues with timestamps. Without /FFT, older versions checked and copied to millisecond granularity, while newer versions copy full NTFS granularity, resulting in the same problem as above. Stupid, but even so, it's rarely encountered because it requires comparing the same files with different versions of robocopy (i.e., different OS), unlike the above which fails on the same machine repeating the exact same operation.
  • StephenB's avatar
    StephenB
    Guru - Experienced User
    I was thinking about /maxage in particular. If you doing these copies periodically, you might not need strict timestamp checks.
  • Nope. Can't use /maxage. My clients have multiple USB drives that rotate between one onsite and others stored offsite. The swapped drive is weeks out-of-date.

NETGEAR Academy

Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology! 

Join Us!

ProSupport for Business

Comprehensive support plans for maximum network uptime and business peace of mind.

 

Learn More