NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
ReadyNAS hangs (fatally) during NAS-to-NAS backups
I've just acquired a Synology DiskSation (DS413j to be precise) to serve as a backup to my existing workhorse ReadyNAS NV. [X-RAID RAIDiator 4.1.6]
Backup creation has not gone smoothly.
The backup jobs are run on the ReadyNAS, "pushing" data across to the DiskStation.
If I create a small backup job, of a single small folder, I can successfully push that across to the DiskStation using either NFS or rsync, so I know it's all basically sound...
But when I set the backup to copy the entire share via NFS (some 5TB of material), the backup routine, on the ReadyNAS, gives up part way through and hangs itself so thoroughly that not only am I unable to access the ReadyNAS via the FrontView web interface, it won't even respond to a press-and-hold reset via the button on the front panel. I have found no recourse other than disconnecting from power. The DiskStation continues to respond normally throughout.
When I reconnect power, and restart the ReadyNAS, it insists on spending many hours doing a RAID resync operation. With that out of the way, I am able to resume the backup until such time as it hangs itself again. The backup logs are empty following a restart.
Several restarts later I finally had what appeared to be a full backup, at which point I upgraded to RAIDiator 4.1.10, applied factory defaults and pulled all the data back from the DiskStation. The data was pulled back in four blocks (first big folder, second big folder, third big folder, everything else) and went without a hitch.
Questions (to which I can find no answers on the forum):
Are there any known issues with pushing large backups from a ReadyNAS?
Why does it hang itself so dramatically?
Any suggestions as to a cause of the problem?
Suggestions for mitigation?
Is there a better way to recover from hang-ups?
What, if anything, might I have been doing wrong?
Is there a better way to to it?
??????????
Backup creation has not gone smoothly.
The backup jobs are run on the ReadyNAS, "pushing" data across to the DiskStation.
If I create a small backup job, of a single small folder, I can successfully push that across to the DiskStation using either NFS or rsync, so I know it's all basically sound...
But when I set the backup to copy the entire share via NFS (some 5TB of material), the backup routine, on the ReadyNAS, gives up part way through and hangs itself so thoroughly that not only am I unable to access the ReadyNAS via the FrontView web interface, it won't even respond to a press-and-hold reset via the button on the front panel. I have found no recourse other than disconnecting from power. The DiskStation continues to respond normally throughout.
When I reconnect power, and restart the ReadyNAS, it insists on spending many hours doing a RAID resync operation. With that out of the way, I am able to resume the backup until such time as it hangs itself again. The backup logs are empty following a restart.
Several restarts later I finally had what appeared to be a full backup, at which point I upgraded to RAIDiator 4.1.10, applied factory defaults and pulled all the data back from the DiskStation. The data was pulled back in four blocks (first big folder, second big folder, third big folder, everything else) and went without a hitch.
Questions (to which I can find no answers on the forum):
Are there any known issues with pushing large backups from a ReadyNAS?
Why does it hang itself so dramatically?
Any suggestions as to a cause of the problem?
Suggestions for mitigation?
Is there a better way to recover from hang-ups?
What, if anything, might I have been doing wrong?
Is there a better way to to it?
??????????
8 Replies
Replies have been turned off for this discussion
- Andrew,
To reply, what I would give are suggestions.
(a) Can you split up Backup jobs to be smaller, or deal with a folder(s) rather than a share?
(b) Are you keeping backup logs? If yes, eg after 2 Tera of data transferred, the Backup_00x
logs fill up the Operating partition, and the whole system locks out.
(c) If so, periodically delete the Backup_00x logs. An occasional OS reinstall would
delete temporary files, and reload the OS.
(d) Any suggestions as to a cause of the problem? ** See above
Is there a better way to recover from hang-ups? ** No. You've no alternative
What, if anything, might I have been doing wrong?
Is there a better way to to it? *** Reduce backup sizes. 5 Tera is alot of data.
The NV processors are not as capable as the Pro/business unit range.
Regards Marto - Marto,
Thanks for the heads-up.
(a) I can, but would have to move quite a few folders to new locations. I don't want to increase the number of shares, unless I have to, since it burns drive mappings on the PC.
Is it possible to move folders directly on the NAS without having them having to make a round trip to the (Windows) PC in the process?
(b) Logs were deleted prior to each attempt. Having seen the size of some of the logs from the successful partial runs, I rather suspected that this might come into it.
Dare one request that the next update to RAIDiator traps this condition such that the backup fails more gracefully? Or even offer a "not logged" option.
(c) I'll think about that one, it sounds "scary".
Thanks. - Marto73, your statement "The NV processors are not as capable as the Pro/business unit range" insinuates the Pro/Business range of products would not exhibit this issue, am I to take that to mean the OS partition size in the Pro is different than the OS partition size of the NV? More simply, would a Pro model backing up the same set of data not crash the unit by filling up the OS partition with logs?
Or, as MDGM stated in this thread -> http://www.readynas.com/forum/viewtopic.php?f=64&t=61430&p=345452
- am I to take from this the Pro models have more advanced log handling functions the NV does not have?There are already precautions in place to limit how big the backup job logs get. However in rare cases for whatever reason sometimes the logs aren't truncated and you get this problem.
Or, has the issue been made irrelevant by moving the logs off the OS partition in the x86 line (as indicated in this thread from 2011 -> http://www.readynas.com/forum/viewtopic.php?f=10&t=56118&p=330010,He roots around the NAS and makes a couple of simultaneous changes (listed at bottom)
1. Logging has been moved from the hidden partition on the disk to a varlog volume to provide more space to log issues to avoid overwriting but running out of space may have been the cause of the lock.
?Would you like to monitor it for a few weeks and then, if there are no further issues, we can likely say that the original issue was that the backup logs had filled the OS partition? If that does end up being the case, I’ve already spoken with the developers regarding this issue and they are working on the best way to address it.
I'm just trying to get a handle on the 'OS partition full due to backup logs" issue as it pertains to the different families of ReadyNAS. - The Sparc models have a 2GB OS partition. The newer models have a 4GB one. It's possible that backup job logs or other logs could fill up the OS partition of any ReadyNAS model if they grow too large somehow.
If you have SSH access you can easily check how full the OS partition is by doing
# df -h
And use the du command to check how much space files/directories use up. - At least you were able to get your NV to rsync with your DiskStation. I have a NV+ (v1) and I can't figure out what step I've missed that would get my NV+'s backup utility to transfer files to my DS412+.
rsync is enabled on the DS412+. I've specified the IP address of my DS412+, listed NetBackup as the path, user root, and my DS's administrator password, but my NV+ fails to connect. Pressing the "Test connection" button on FrontView Backup says "Error connecting to xxx.xxx.x.xxx".
xxx.xxx.x.xxx is the local IP address of my DS412+.
Starting the backup job produces the following log:INCREMENTAL Backup started. Sat Apr 27 12:46:37 PDT 2013
Job: 001
Protocol: rsync
Source: [backup]/
Destination: xxx.xxx.x.xxx::NetBackup
rsync: failed to connect to xxx.xxx.x.xxx (xxx.xxx.x.xxx): No route to host (65)
rsync error: error in socket IO (code 10) at clientserver.c(122) [sender=3.0.9]
Backup failed. Sat Apr 27 12:46:40 PDT 2013.
Reason for failure:
Error encountered copying data from source path /backup/ ==> xxx.xxx.x.xxx::NetBackup due to unknown reason. Please see log.
Of course, I am a bit concerned that even if I do remedy this, I'll just end up in the same situation you are with it hanging. My NV+ has a 2TB volume that is 92% full...and I put a 3TB volume in the DS412+ hoping to alleviate my capacity issue. - Yesterday, I got fed up trying to rsync and just started using the PC to copy files between the NASs. I was getting around 20-30Mbps and the copying went on for a few hours with the time estimate staying around 8 hours and 30 minutes. Then my PC froze. Completely froze. Only option was to power down.
Tried again today. Copying went on as expected for an hour or so, then the PC froze again.
I need a better solution for this.
If I could remember how to format my external USB drive, I could directly connect it to the NAS and transfer that way I suppose... - If this is a Windows PC then the NAS will already recognize the format.
The lockups are concerning, since of course you likely have incomplete transfers/corrupt files.
I generally use robocopy when copying lots of data to/from the PC. - I finally figured out what was preventing rsync from working...and could potentially be responsible for the lock ups.
Even though I originally had both NAS devices set to have their IP assigned via DHCP, the new DS412+ somehow got manually assigned to the same IP address that my router had reserved / assigned my NV+. At some point, the DS412+ must have had a different IP address, because I had reserved that particular address in my router's DHCP reservation list.
I have no idea how any of this has been working because I have been logging on to the DS412+'s web interface using the IP address it was originally assigned and could still log on to the IP address of the NV+ using the one it was being assigned via DHCP. I only discovered the conflict when my NV+ started reporting a conflict this morning...the first such warning I have received since I started setting up the DS412+ a week ago.
Anyway, as soon as I assigned both NAS devices to static IP addresses, I was able to initiate an rsync backup job. The only thing now is I'm questioning the performance. Looking at my DS412+'s performance monitor, network activity seems to be hovering around 5MB/s...write speeds getting no higher than 40MB/s.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
