NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Rsync disconnects my external USB drive
I read through a number of posts on here and the benefits of Rsync keeping mirror copies without having to perform entire backups. That's a good thing if you have terabytes of data to backup and therefore a big time saver on incremental updates. So as suggested on a number of posts, I initially ran a standard backup job using Frontview so I could get my full backup for each share to my external USB drive. I then changed it to use Rsync where my source was the NAS and path was the external USB path/folder. The destination was changed to be the NAS local share. I hit go on the job and it runs and completes quickly seeing there are no differences between the source & destination at this time. I then drop a small text file into the NAS share like I always do when backing up my files. I then proceed to hit go on the backup job and it completes without any issues. HOWEVER, it results in my text file being deleted from the NAS share rather than mirroring over to the external USB drive. This is actually the opposite of what I expected. I further confirmed it by placing the text file on the external USB HD folder and re-run the job. The logs show it copies the file over to the NAS. The reverse is true and deleting the file from the USB HD folder and re-running the job shows the file is removed from the NAS.
So, I figure changing the path under the source to be the local NAS share and the destination to be the USB drive path/folder would do the trick. Note: Rsync is still the protocol being used here. Anyhow, retesting does show it works as I would think. So placing a file on the NAS share and re-running the job shows it is copied over to the external HD folder. Unfortunately, the the USB drive is disconnected/unmounted when the job completes. I have to recycle the USB drive or unplug the USB cable and plug it back in for the NAS to see it. Again, the job does complete correctly but for some reason it disconnects the USB drive.
Is this normal and the way the client/server functions work for Rsync? I would say most of the posts on this forum and otherwise are written using Rsync between two NAS which may be its intended purpose. For the time being, I have reverted back to using the built in backup job function. Otherwise, my first job would turn off the USB drive and result in subsequent jobs failing.
I'm thinking this is a bug but hoping someone else can confirm it either way.
So, I figure changing the path under the source to be the local NAS share and the destination to be the USB drive path/folder would do the trick. Note: Rsync is still the protocol being used here. Anyhow, retesting does show it works as I would think. So placing a file on the NAS share and re-running the job shows it is copied over to the external HD folder. Unfortunately, the the USB drive is disconnected/unmounted when the job completes. I have to recycle the USB drive or unplug the USB cable and plug it back in for the NAS to see it. Again, the job does complete correctly but for some reason it disconnects the USB drive.
Is this normal and the way the client/server functions work for Rsync? I would say most of the posts on this forum and otherwise are written using Rsync between two NAS which may be its intended purpose. For the time being, I have reverted back to using the built in backup job function. Otherwise, my first job would turn off the USB drive and result in subsequent jobs failing.
I'm thinking this is a bug but hoping someone else can confirm it either way.
11 Replies
Replies have been turned off for this discussion
- In the first paragraph it sounds like you did an incremental backup with rsync with the same source and destination folder? That seems likely to malfunction, though I am a bit surprised about a deletion.
The unmounted drive sounds like your real concern though. I agree that is not good behavior for an unattended backup. StephenB wrote: In the first paragraph it sounds like you did an incremental backup with rsync with the same source and destination folder? That seems likely to malfunction, though I am a bit surprised about a deletion.
The unmounted drive sounds like your real concern though. I agree that is not good behavior for an unattended backup.
The source/destination were different. I took a few snap shots of what I had that may be better than my written explaination. However, I don't see how I can attach those to this posting. It seems you have to link them. :(- I couldn't get the images to work so here are my settings:
Step 1:
drop down: Remote: Rsync Server
Host: 172.28.1.1
Path: media/Videos/Adult
Remove deleted files on source is checked/enabled
A quick note about the path in Step1. The media folder is NOT preceeded with a / like is done in the path for Step 2. The test/job will fail if this is included.
Step 2:
drop down: Share: USB_HDD_1
Path: /media/Videos/Adult
Maybe you can just clarify what you meant to say above...NASguru wrote: So as suggested on a number of posts, I initially ran a standard backup job using Frontview so I could get my full backup for each share to my external USB drive. I then changed it to use Rsync where my source was the NAS and path was the external USB path/folder. The destination was changed to be the NAS local share. StephenB wrote:
Maybe you can just clarify what you meant to say above...NASguru wrote: So as suggested on a number of posts, I initially ran a standard backup job using Frontview so I could get my full backup for each share to my external USB drive. I then changed it to use Rsync where my source was the NAS and path was the external USB path/folder. The destination was changed to be the NAS local share.
Sure, I grabbed this outline from another post on the process of using Rsync. I've seen simiar steps in other posts but as I said previoulsy they are mostly written for NAS to NAS using Rsync.
1, Enable Rsync in Servers/Standart Files Protocals
2, Open share list, enable Read/Write in Rsync of USB Share.
3, Add a backup job, set source as remote rsync, input your NAS's IP in Host and Path is USB folder name.
4, Set destination as a share folder.
In step 3 above, I set the path to be the local NAS share instead of the USB folder name and in step 4 above I set the destination to be the USB share folder instead of being the NAS share. I don't want the USB share to be treated as the server since it's not where I make my changes to the files. I add/delete files off the NAS which is MUCH quicker than transfer rates to and from a USB drive attached to the NAS and hence my preference.
So bottom line, should my USB drive be disconnected using the settings I have listed in my screen shot?- Ok, so at the end you had the NAS share as the source, and the USB drive as the destination. You are correct that you don't want the the share to be on the USB drive, you just want the backup to go there.
I'm not seeing the screen shots, just the word image in plain-text. But the USB drive ought to remain mounted. At least, this guide from mdgm (http://www.rnasguide.com/2010/09/23/rsy ... -usb-disk/) doesn't suggest otherwise.
I'm sure there are other posters here who use your approach, maybe they can chime in. NASguru wrote: I grabbed this outline from another post on the process of using Rsync. I've seen simiar steps in other posts but as I said previoulsy they are mostly written for NAS to NAS using Rsync.
1, Enable Rsync in Servers/Standart Files Protocals
2, Open share list, enable Read/Write in Rsync of USB Share.
3, Add a backup job, set source as remote rsync, input your NAS's IP in Host and Path is USB folder name.
4, Set destination as a share folder.
I backup my entire NAS (c: drive) to a USB using the front button, and then turn-off the USB drive until next backup.
The backup job has the Step 1 Source as "Volume: c"
The Step 2 Destination is "Remote: Rsync Server" with Host of localhost and Path of GOFLEX_EXT2 (this last will be named different for you). I do have "Remove deleted files" checked here also.
Then under Step 4 I have "Schedule Full backup" set as "first time".
I don't understand why you "Set destination as a share folder". This feels like the problem IMO.
In any case my backup button writes only changes to my external USB drive, deleting any files that have been deleted from the NAS.
The only trouble I had with the above was re: doing the first FULL backup, i.e. I already had copied volume: c to the drive by another method, thus needed to tell the Rsync process to treat my initial run as an incremental. Here's what I found that helped me:
http://carlhutzler.com/blog/2009/08/26/ ... he-get-go/StephenB wrote: Ok, so at the end you had the NAS share as the source, and the USB drive as the destination. You are correct that you don't want the the share to be on the USB drive, you just want the backup to go there.
I'm not seeing the screen shots, just the word image in plain-text. But the USB drive ought to remain mounted. At least, this guide from mdgm (http://www.rnasguide.com/2010/09/23/rsy ... -usb-disk/) doesn't suggest otherwise.
I'm sure there are other posters here who use your approach, maybe they can chime in.
Correct, that's how I had it setup and I've seen that guide from MDGM before.maxblack wrote: NASguru wrote: I grabbed this outline from another post on the process of using Rsync. I've seen simiar steps in other posts but as I said previoulsy they are mostly written for NAS to NAS using Rsync.
1, Enable Rsync in Servers/Standart Files Protocals
2, Open share list, enable Read/Write in Rsync of USB Share.
3, Add a backup job, set source as remote rsync, input your NAS's IP in Host and Path is USB folder name.
4, Set destination as a share folder.
I backup my entire NAS (c: drive) to a USB using the front button, and then turn-off the USB drive until next backup.
The backup job has the Step 1 Source as "Volume: c"
The Step 2 Destination is "Remote: Rsync Server" with Host of localhost and Path of GOFLEX_EXT2 (this last will be named different for you). I do have "Remove deleted files" checked here also.
Then under Step 4 I have "Schedule Full backup" set as "first time".
I don't understand why you "Set destination as a share folder". This feels like the problem IMO.
In any case my backup button writes only changes to my external USB drive, deleting any files that have been deleted from the NAS.
The only trouble I had with the above was re: doing the first FULL backup, i.e. I already had copied volume: c to the drive by another method, thus needed to tell the Rsync process to treat my initial run as an incremental. Here's what I found that helped me:
http://carlhutzler.com/blog/2009/08/26/ ... he-get-go/
Unfortunately for me, backing up the whole volume isn't an option at this time since my external USB drive doesn't have enough capacity. As for my destination, it's a folder on the USB drive named with the same path/folder as the one on the NAS. I probably shouldn't have said share as that can easily be confused with the folders created under the NAS quotas. That aside, I will try setting the rsync up under step 2 while the source is simply the share on the local NAS. That may give a different result. It's funny you should mention the first FULL backup. I did my first full backup using the built in copy command for the default backup jobs for each folder and then switch to Rsync. However, I fell back to the default method for backup jobs since my NAS was disconnecting my external USB drive while using Rsync or so I thought. On Monday my schedule ran for the backup jobs on 3 folders that I have setup and wouldn't you know it the job was backing up the whole folder again instead of doing an incremental move. As if that wasn't enough, my external USB drive was again disconnected after the first job completed which of course failed the other two jobs that were already in progress. So at this point, I'm not convinced it's a Rsync problem now. I should mention my external USB Wester Digital drive is one of those Green Desktop versions so it could be a firmware thing as well?
BTW, thanks for that link. I'll force the incremental on the next go around so I don't have to cancel or sit through it again. I'm also going to setup a small test folder/share instead of hashing this out with 500 GB worth of data. :D- I finally had some time to circle back on this issue. The good news is I figured out how to make this work where the NAS no longer disconnects my external USB drive. Here are my settings to my backup job for those that may be interested:
Step 1:
Drop down is set to: Remote: Rsync Server
Host is set to: 172.28.1.11
Path: media/Videos/Kids
Remove deleted files on source is checked/enabled
step 2:
Drop down is set to: USB Device (Rear Bottom Port)
Path: /media/Videos/Kids
Unmount usb drive after backup is NOT checked.
Please notice in Step 2 the drop down is NOT set to share: USB_HDD_1. From what I can tell, the default behavior of this selection is to unmount from the external USB drive after the backup job is completed. Unfortunatley, the NAS is never able to mount the USB again following the job so any and all other jobs using it fail. So, changing to use USB Device (Rear Bottom Port) allows the option to unmount be selected and by default it is not enabled which is exactly what I was looking for in the destination. I did not try any other USB port on the NAS or the relevant options for those ports but expect the results to be the same.
That being said, I did notice that this issue (unmounting the external USB drive) did NOT appear when backing up the share FTP1 using the share: USB_HDD_1 as the destination in the drop down along with the relevant path. I have no idea why, but the NAS treats the default media folder differently than the FTP1 folder that I created under the shares. I checked all the settings for both in Frontview and see no difference between them that would result in a different outcome when running the same protocol on the backup jobs. However, SSH to the NAS shows the permissions for the media folder has a 't' at the end of it. Does anyone know if that is the reason or what that means?
drwxrwxrwx 2 nobody nogroup 4096 2012-12-17 14:48 FTP1
drwxrwxrwt 5 nobody nogroup 4096 2012-09-04 17:09 media
Bonus question, does anyone know how to backup a user folder with Rsync. :D I'm having issues with step one failing to communicate with the NAS. I believe it's either the path and/or due to not being able to set Rsync permissions for users like you can for shares. Thoughts?
Step 1:
Drop down is set to: Remote: Rsync Server
Host is set to: 172.28.1.11
Path: homes/John/
Remove deleted files on source is checked/enabled
I also tried the following for the path but that also didn't work. c/home/John
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
