NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
RN104 OS 6.7.1 -> 6.7.4 breaks rsync backup of Time Machine to External Disk?
Our Macs use Private Time Machine on ReadyNAS, and we have been backing up the collective Time Machine "share" to external USB/eSATA disk using a remote: Rsync over SSH backup job (in the Admin Page). Like so, e.g.:
---------------------------------------- Backup Job Name: Time Machine (M W F Sa) Backup Job Type: Incremental Protocol: rsync+ssh Backup Source: [timemachine:~timemachine~]/ Backup Destination: [remote:rsync+ssh]/127.0.0.1://media/USB_HDD_4/TimeMachine/MWFSa Backup Start Time: Mon May 29 2017 2:59:41 Backup Finish Time: Mon May 29 2017 4:02:14 Backup Status: Success
That above is actually the last successful run of this job, before upgrading ReadyNAS OS 6.7.1 to 6.7.4 on 5/29/2017 at 22:49.
Since then, every time the job (and its (Su T Th) sibling) runs, it finishes in just a few seconds with a log entry like:
sending incremental file list sent 177 bytes received 20 bytes 78.80 bytes/sec total size is 0 speedup is 0.00
and no actual data is copied.
Testing quite a bit, I found that the fundamental problem is that the rsync backup job is copying at most only the top-level directories (i.e., not recursively), so if we have:
/data/.timemachine/chris/ /data/.timemachine/ellie/lunitari.sparsebundle/ /data/.timemachine/tom/bacamac.sparsebundle/ /data/.timemachine/tom/phaedrus.sparsebundle/
...running the backup job to an empty destination dir results in empty chris, ellie, and tom directories being created:
sending incremental file list ./ chris/ ellie/ tom/ sent 159 bytes received 31 bytes 380.00 bytes/sec total size is 0 speedup is 0.00
After lots more experimenting and testing (incl. running rsync in a shell), I have found that it seems to be related to the --one-file-system option for rsync.
Using an example rsync command I found in another thread, I was able to narrow it down to that option by comparing:
root@NASbox:~# rsync -v -8 --timeout=30 --recursive --links --times --devices --specials --one-file-system --modify-window=2 --owner --group --no-acls --no-perms -delete /data/.timemachine/ /media/USB_HDD_4/TimeMachine/SuTTh/test sending incremental file list ./ chris/ ellie/ tom/ sent 159 bytes received 31 bytes 380.00 bytes/sec total size is 0 speedup is 0.00
and (removing --one-file-system option):
root@NASbox:~# rsync -v -8 --timeout=30 --recursive --links --times --devices --specials --modify-window=2 --owner --group --no-acls --no-perms -delete /data/.timemachine/ /media/USB_HDD_4/TimeMachine/SuTTh/test sending incremental file list ./ chris/ ellie/ ellie/.AppleDB/ ellie/.AppleDB/cnid2.db ellie/.AppleDB/db_errlog ellie/.AppleDB/lock ellie/.AppleDB/log.0000000003 ellie/lunitari.sparsebundle/ ellie/lunitari.sparsebundle/Info.bckup ellie/lunitari.sparsebundle/Info.plist ellie/lunitari.sparsebundle/com.apple.TimeMachine.MachineID.bckup
...
and off we go...
...and finally the weirdest part: It only misbehaves if the source dir is /data/.timemachine/. If it's a different path like /data/.timemachine/tom/ or even /data/.test/, it works as expected, even with the --one-file-system option:
root@NASbox:~# rsync -v -8 --timeout=30 --recursive --links --times --devices --specials --one-file-system --modify-window=2 --owner --group --no-acls --no-perms -delete /data/.timemachine/tom/ /media/USB_HDD_4/TimeMachine/SuTTh/test sending incremental file list ./ .AppleDB/ .AppleDB/cnid2.db .AppleDB/db_errlog .AppleDB/lock .AppleDB/log.0000000106 bacamac.sparsebundle/ bacamac.sparsebundle/Info.bckup bacamac.sparsebundle/Info.plist bacamac.sparsebundle/com.apple.TimeMachine.MachineID.bckup bacamac.sparsebundle/com.apple.TimeMachine.MachineID.plist bacamac.sparsebundle/com.apple.TimeMachine.Results.plist bacamac.sparsebundle/com.apple.TimeMachine.SnapshotHistory.plist bacamac.sparsebundle/token
...
...and off we go...
I understand that my example rsync command line is not exactly what is actually used by the backup job (e.g., no SSH and no snapshot), but I believe it is close enough to have worked in isolating the problem.
I'm no kind of expert on btrfs and its snapshots, etc., but thinking hard about the --one-file-system thing and /data/.timemachine/ vs. /data/.timemachine/tom/ vs. /data/.test/, could this be related to the fact that /data/.timemachine is a btrfs subvolume?
Anyway, to repeat: these backup jobs were working fine prior to the 6.7.1 -> 6.7.4 upgrade, and effectively fail every time since.
Any ideas, or can anyone replicate?
Thanks!
FYI, 6.7.5-T299 (Beta 1) seems to have resolved this.
7 Replies
Replies have been turned off for this discussion
Hi capaz
This's a issue , we will fix it on later firmware
capaz,
Thanks for your excellent explaination. I am piping in to say I have a very similar problem after updating from 6.7.1 to 6.7.4, but I'm using RSync (push) from my main ReadyNAS to my secondary ReadyNAS in the same LAN, so it seems the Rsync routines in 6.7.4 have been messed at the same time as (or likely by) the 6.7.4 firmware. New (top level) directories are copied if they are just below the 'root', but nothing inside them.
One solution may be to give the Netgear firmware programmers more compensation - they seem to have made some real blunders in the past few months. They need motivation to 'dot their I's, and cross their T's' - in other words, to pay more attention to details.
Sheesh - it's like they have lately been delivering firmware as if it's a once-off for some cheap Chinese electronics.
Netgear - wake up! It's time to deliver better quality software. You are marketing this line as premium, robust products.
These rsync problems appear to have been an issue since 6.7.1.
And here's the larger issue: Every time someone on these forums has a serious issue with their ReadyNAS (and I personally have had many of them), the admins wag a finger at them if they don't have a backup. Rsync is a basic and vital platform for those routine backups — and yet it's been broken for the last couple of messy, problematic firmware builds. I haven't been able to complete a successful backup clone for weeks.
If buggy firmware is an inevitability, Netgear at least needs to make sure that rsync works so we can safeguard against data loss.
nsne wrote:the admins wag a finger at them if they don't have a backup. Rsync is a basic and vital platform for those routine backups — and yet it's been broken for the last couple of messy, problematic firmware builds. I haven't been able to complete a successful backup clone for weeks.
If buggy firmware is an inevitability, Netgear at least needs to make sure that rsync works so we can safeguard against data loss.
Admin or not, you do need to have a backup.
BUT I completely agree that rsync being broken is a critical issue. Exactly because backup is one of the most critical part of storage. Hopefully NETGEAR will be prompt on releasing a fix. I understand why you guys aren't happy. I too had some issues with my ReadyNAS lately and I too use rsync for a lot of my backups. Luckily for me, I don't fully rely on either for my backup plan.
I can't comment much on the 6.7 firmwares, but until now, for sure it hasn't be glorious.
FYI, 6.7.5-T299 (Beta 1) seems to have resolved this.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
