My ReadyNAS is very full up, and I'm trying to copy stuff of it without any success.
First, I was having trouble with those stupid DS_Store files, so I decided to compress the files in order to make it easier to move them.
It failed the first time, because I had the temerity to include an asterisk in the zip file name. So I got rid of that, and tried again. This time the error was far too vague:
"Reason for failure: Error encountered copying data from source path ZIPFILE1 ==> /USB_HDD_1/AUDIO due to unknown reason. Please see log."
As that IS the log, AFAICT, what does it mean by "Please see log."? Is there another log I can look at to help figure out why this backup failed? Is it an input or an output problem?
If I can't get it to work, I'm going to have to copy this stuff, all nearly 900GB of it, manually, by copying it to the desktop, and then up to the drive I want it on, and that will take days!
This USB drive is brand new, I've only had it for a few days, and it hasn't shown any other signs of corruption.
Many thanks.
[Mac Mini, OS 10.9.5]
Model: ReadyNASRND4410v2|ReadyNAS NV+ v2 4TB (4 X 1TB Desktop)
The file system of the USB drive is FAT32. The ReadyNAS firmware is 5.3.8, and it's a 4-bay ReadyNAS nv+ v2. (Apparently I can update it to 5.3.11, but I don't know if that will make any difference to this issue?)
Sometimes, it just truncates a file that's bigger than 4.29GB (so this appears to be the limit for this USB drive), and carries on, but at other times, it stops. I haven't figured out what the difference is.
What I'd really like to be able to do, and I've no idea if this is possible, is to say to the backup: okay, backup everything you can, and give me a list of the things you couldn't (I imagine because of characters in filenames, or file size), and then I can do them manually later.
Also, I've got another USB drive, which I know for a fact has over 600GB spare on it, but when I connect it up to either of my ReadyNASes (I also have a v1, which I'm actually copying this data to, via the USB drive), they say it has no free space. It would be useful if I could use this drive as well in this process. According to Disk Utility, its file system is "Mac OS Extended (Journaled)".
But what would be really, really useful, is if the backup could actually tell me what the error is, rather than just "unknown", and the system log doesn't SEEM to have any more detail. It just says "Please see log."!!
Thanks Stephen. No, unfortunately I can't use NTFS, because I can't format those on my Mac. Anyway, I can't format this drive as anything other than what it is, as it has data on it.
There's nothing wrong with the USB drive, or at least Disk Utility says so, anyway. (It's the same drive I was copying stuff onto from the other NAS box last week.) Any thoughts on the other USB drive with 600+GB available on it?
My main question, apart from what use is an 'unknown error', is, is there a way I can make the backup process bypass the errors, and just make a note of them, rather than stopping? It's extremely time-consuming this way!
And also, is there a way of copying directly form NAS to NAS, without using an intermediary drive, and without using Rsync, which AFAICT, wants to incrementally backup the whole volume? (Unless that's not true, in which case, how do I do it, please?)
And also, is there a way of copying directly form NAS to NAS, without using an intermediary drive, and without using Rsync, which AFAICT, wants to incrementally backup the whole volume? (Unless that's not true, in which case, how do I do it, please?)
You can backup up NAS-to-NAS with NFS, CIFS, or rsync. And you can back up individual shares with any of these protocols. I use incremental rsync to back up each share of my pro-6 daily to various backup ReadyNAS.
If the protocols are enabled on the v2, you can configure the backup jobs to run on the v1 (or vice versa, but I am more familar with the v1). What is confusing you on the backup job configuration?
Thanks Stephen. No, unfortunately I can't use NTFS, because I can't format those on my Mac.
Well, the format is the issue with the file size. There are also character limits with FAT32 that might be getting in your way. There are inexpensive ways to format as NTFS on the mac (paragon being one). If you are willing to get paragon, you could also go with ext formatting, which has faster copying than NTFS (especially on the v1).
Or you can copy the data over the network to a USB drive connected to the mac. If you have wired ethernet, that is the fastest way to do it. And you can format the drive to anything the mac supports.
You can backup up NAS-to-NAS with NFS, CIFS, or rsync. And you can back up individual shares with any of these protocols. I use incremental rsync to back up each share of my pro-6 daily to various backup ReadyNAS.
If the protocols are enabled on the v2, you can configure the backup jobs to run on the v1 (or vice versa, but I am more familar with the v1). What is confusing you on the backup job configuration?
I've set up Rsync on both boxes. I couldn't use CIFS, because it doesn't appear to be available on the v2.
I'm doing this on the v2, rather than the v1, because although I much prefer the v1's GUI, it doesn't allow me to select paths in the backup. If I click 'Browse', it gives me an empty window which I can't close without rebooting the browser. This means having to dump everything on the top level, which is very messy.
The two boxes are connected to, and by, a Netgear Portsafe 8 Port Gigabit Switch, model GS108. I could connect the two NAS boxes directly, if I need to.
When I set up the Backup Job, and tested the connection, I got the message 'Cannot connect to host.', with the code 7077010011. These are the settings I used:
BACKUP SOURCE [this is the v2, which is called Media2]
Type: share
Name: share: films
Host: [empty, and unclickable]
Path: FILMS/Other1
Log In:
Password:
BACKUP DESTINATION [this is the v1, which is called Media1]
Type: remote
Name: remote: Rsync Server
Host: Media1
Path: FILMSNEW/FINALBACKUPS [with greyed out Browse button, but at least I can type in the Path field, which I can't do on the v1]
Login: admin
Password: ********
This is what I get when I try the backup:
-------------------------------------------------------------------------------- FULL Backup started. Wed Mar 9 14:25:11 GMT 2016
rsync: getaddrinfo: Media1 873: Name or service not known rsync error: error in socket IO (code 10) at clientserver.c(122) [sender=3.0.9]
Backup failed. Wed Mar 9 14:25:12 GMT 2016.
Reason for failure: Error encountered copying data from source path /Films/FILMS/Other1 ==> Media1::FILMSNEW/FINALBACKUPS due to unknown reason. Please see log.
CIFS is windows file sharing and it is available on the v2. In backup jobs it is usually called "Windows/NAS (Timestamp)".
Not on mine, it isn't, unless it's called something else in Services. I've got the choices of: SMB, AFP, NFS, FTP, ReadyDLNA, Rsync, UPnP, HTTP, HTTPS, SNMP, and genie.
Anyway, we're going with Rsync...
I'm doing this on the v2, rather than the v1, because although I much prefer the v1's GUI, it doesn't allow me to select paths in the backup.
Browse is grayed out with current v1 firmware if you select "remote: rsync server" But you can certainly enter the path.
Browse is even greyed out if I select a USB drive. And on the v1, no, I can't. Don't know why, if you say I should be able to. When I backed up stuff to the USB drive, it all went on to the top level, which is really messy and annoying.
Is rsync enabled on the V1 destination share?
Ah! Didn't realise I needed to do that as well as enable it for the box. Done it now, and have rebooted both boxes for good measure. Still got the same error.
Is /Films/FILMS/Other1 the source path you actually want?
Did you try the IP address instead of Media1 for the destination?
Yes and yes. Hadn't tried the IP address before, but have now, and still get the same error.
TBH, this is really now just a proof of concept, as it were, rather than an actual necessity. The remaining folder is small enough to copy using the 1TB USB drive, which on past form would take about six hours. I would like to know how to copy from one NAS to another, but by the time I come to need to do it again the GUI will have changed again even from what it is now.
So, if you're happy to guide me through this anyway, and get it to work, that'd be great. But if not, I'm happy to leave it here.
BTW, thanks for the heads up on Paragon. I knew of it, but didn't realise it now has verify, repair and format options. Will definitely get that, then! Am I right in thinking that the NAS doesn't like Apple's 'Mac OS Extended (Journaled)' format?
CIFS is windows file sharing and it is available on the v2. In backup jobs it is usually called "Windows/NAS (Timestamp)".
Not on mine, it isn't, unless it's called something else in Services. I've got the choices of: SMB...
SMB is the same as CIFS.
Browse is even greyed out if I select a USB drive. And on the v1, no, I can't. Don't know why, if you say I should be able to. When I backed up stuff to the USB drive, it all went on to the top level, which is really messy and annoying.
I don't use USB drives for NAS backup. However, I do use rsync NAS-NAS, and I did confirm that I could manually enter a path on the v1 using remote:rsync server as the NAS source, and also on the NAS destination (using share: sharename). I am running 4.1.15-T3 on the v1 NAS.
TBH, this is really now just a proof of concept, as it were, rather than an actual necessity. The remaining folder is small enough to copy using the 1TB USB drive, which on past form would take about six hours. I would like to know how to copy from one NAS to another, but by the time I come to need to do it again the GUI will have changed again even from what it is now.
The v1 and v2 GUIs won't be changing, as both are end-of-life products.
If you think it's worth completing the exercise, I suggest creating a small test share on both machines, doing a straight share-to-share backup (using only the share name in the path). I could share my settings if that fails.
NFS is another protocol that is worth using - rsync is more robust on permissions/owners/file dates, but NFS is quite a bit faster.
BTW, thanks for the heads up on Paragon. I knew of it, but didn't realise it now has verify, repair and format options. Will definitely get that, then! Am I right in thinking that the NAS doesn't like Apple's 'Mac OS Extended (Journaled)' format?
Both the v1 and v2 documentation say they only support FAT32, NTFS, ext2, and ext3.
CIFS is windows file sharing and it is available on the v2. In backup jobs it is usually called "Windows/NAS (Timestamp)".
Not on mine, it isn't, unless it's called something else in Services. I've got the choices of: SMB...
SMB is the same as CIFS.
Oh, I do love consistency!! 🙂
Browse is even greyed out if I select a USB drive. And on the v1, no, I can't. Don't know why, if you say I should be able to. When I backed up stuff to the USB drive, it all went on to the top level, which is really messy and annoying.
I don't use USB drives for NAS backup. However, I do use rsync NAS-NAS, and I did confirm that I could manually enter a path on the v1 using remote:rsync server as the NAS source, and also on the NAS destination (using share: sharename). I am running 4.1.15-T3 on the v1 NAS.
Fair enough, don't know why I can't, then. Should it make any difference if I select "share: USB_HDD_1" or "USB Device: Front Port"? (I know they're technically the same thing, but obviously there must be some difference, or they wouldn't be listed separately.)
I just backed up the last folder onto the USB drive, and am now backing it up onto the v1. BUT there was an error! At least, the email, which this time actually got sent, told me so, but the file it complained about is fine, and the whole folder has been copied. The file is an mp4 film, and is perfectly playable. It is not over 4.29GB, and it has no characters in its name apart from alpha ones. I hate it when stuff like that happens! 😞
TBH, this is really now just a proof of concept, as it were, rather than an actual necessity. The remaining folder is small enough to copy using the 1TB USB drive, which on past form would take about six hours. I would like to know how to copy from one NAS to another, but by the time I come to need to do it again the GUI will have changed again even from what it is now.
The v1 and v2 GUIs won't be changing, as both are end-of-life products.
Ah no, I knew that, I was talking about the GUI for whatever version I next get. If it's like the v2 one, Dashboard, I'd have to think seriously about it, because I think it's horrible. The v1 interface, FrontView, is nice, all in one box, not sprawling all over the page, and with a left-hand menu that doesn't move! Anyway, aesthetics aside,…
If you think it's worth completing the exercise, I suggest creating a small test share on both machines, doing a straight share-to-share backup (using only the share name in the path). I could share my settings if that fails.
NFS is another protocol that is worth using - rsync is more robust on permissions/owners/file dates, but NFS is quite a bit faster.
Yes, definitely, thanks, that'd be great. But talking of permissions, etc, I think they need to be sorted out first, at least on the v2. On the v1, I can login through the Finder as Guest, and I have read/write access. On the v2, I can't log in as Guest, it fails. So the ownerships &/or permissions for the two boxes are not the same, and they need to be. There's no reason why anyone logging in as Guest (because that will only ever be me) should not have full read/write access. But I don't know how to give Guest access, and I can't find anything about it in the manual. (Another reason for Guest access is that I don't get 'hidden' shares like 'admin' and 'homes', which I don't need.)
And, as an aside, but an important one, do you know why it sometimes takes so long to log in to the v2 (not the v1) via the Finder? Sometimes it can literally take minutes, even when absolutely nothing else is going on on the computer. And sometimes I need to restart the Finder, and even reboot the computer, to get it to log in at all.
Both the v1 and v2 documentation say they only support FAT32, NTFS, ext2, and ext3.
The v2 manual doesn't talk about guest (or anonymous) access. If making the share read/write for everyone doesn't do it, then perhaps create a single user account, and set up credentials on every PC. You'll still see the home share on the v2 for that user, since the v2 firmware doesn't let you configure that.
You probably can't create a username of "guest", but you could use anonymous (or anon).
The v2 manual doesn't talk about guest (or anonymous) access. If making the share read/write for everyone doesn't do it, then perhaps create a single user account, and set up credentials on every PC. You'll still see the home share on the v2 for that user, since the v2 firmware doesn't let you configure that.
No, it doesn't, though on the v2, there's an option of 'allow anonymous access' for each share, not for the whole box. Setting those has allowed me to log in as Guest, exactly as I wanted, so I don't see 'admin' and 'homes'. I'm assuming that's because unless there is anonymous access to at least one share, it won't allow it at all. 🙂
So, okay, I'm happy with the permissions and ownerships now, and AFAICT, they're the same for both boxes! 🙂
I've set up two shares with NFS, 'V1_SHARE' and 'V2_SHARE', and I've setup a Backup, and tested the connection, and it worked!!! 🙂 Not as the hostname, but as the IP address. So anyway, definitely on track to try the backup now. In the share I'm copying from there's a folder with asterisks in its name, and in it are two movies, one of which is 5.34GB, and the other is 'M*A*S*H.mp4'. If it can copy these, then I'm happy, and I just wish I'd done it this way to start with! Ah well! 🙂
So, I'm just waiting for the final copy from the USB drive to finish, and then I can try this backup. 🙂
Tried the backup, and it sot of worked, and it sort of didn't.
I can see the new folder and files in the Finder window, and I can open and play them.
BUT…
"Error encountered copying data from source path /V1_SHARE/ ==> 192.168.0.4:/V2_SHARE due to permission denial on one or more files."
And the log is full of : "Permission denied" and "Operation not permitted".
The NFS settings for the source share, on the v1 are: "Default access: Read/write."
And the settings for the destination share, on the v2, are: "Share Access: Any Host - Read/Write"
Also, the file that caused the last backup to the USB drive to say it had failed, even though it hadn't, has not caused a failure when it was copied to the other NAS!
BTW, happy to include screenshots, but not sure how. I could use Dropbox, but I can't guarantee I'll remember to keep the images there. Is there anywhere I can upload images to?
`./.AppleDB/log.0000000001' -> `/job_001/.AppleDB/log.0000000001' cp: cannot create regular file `/job_001/.AppleDB/log.0000000001': Permission denied `./.AppleDB/__db.005' -> `/job_001/.AppleDB/__db.005' cp: cannot create regular file `/job_001/.AppleDB/__db.005': Permission denied `./.AppleDB/__db.003' -> `/job_001/.AppleDB/__db.003' cp: cannot create regular file `/job_001/.AppleDB/__db.003': Permission denied `./.AppleDB/lock' -> `/job_001/.AppleDB/lock' cp: cannot create regular file `/job_001/.AppleDB/lock': Permission denied `./.AppleDB/db_errlog' -> `/job_001/.AppleDB/db_errlog' cp: cannot create regular file `/job_001/.AppleDB/db_errlog': Permission denied `./.AppleDB/__db.006' -> `/job_001/.AppleDB/__db.006' cp: cannot create regular file `/job_001/.AppleDB/__db.006': Permission denied `./.AppleDB/__db.002' -> `/job_001/.AppleDB/__db.002' cp: cannot create regular file `/job_001/.AppleDB/__db.002': Permission denied `./.AppleDB/__db.001' -> `/job_001/.AppleDB/__db.001' cp: cannot create regular file `/job_001/.AppleDB/__db.001': Permission denied `./.AppleDB/cnid2.db' -> `/job_001/.AppleDB/cnid2.db' cp: cannot create regular file `/job_001/.AppleDB/cnid2.db': Permission denied `./.AppleDB/__db.004' -> `/job_001/.AppleDB/__db.004' cp: cannot create regular file `/job_001/.AppleDB/__db.004': Permission denied cp: preserving times for `/job_001/.AppleDB': Permission denied `./.DS_Store' -> `/job_001/.DS_Store' cp: failed to preserve ownership for `/job_001/.DS_Store': Operation not permitted `./Temporary Items/.AppleDouble/.Parent' -> `/job_001/Temporary Items/.AppleDouble/.Parent' cp: preserving times for `/job_001/Temporary Items/.AppleDouble/.Parent': Operation not permitted cp: preserving times for `/job_001/Temporary Items/.AppleDouble': Operation not permitted cp: preserving times for `/job_001/Temporary Items': Operation not permitted `./Network Trash Folder/.AppleDouble/.Parent' -> `/job_001/Network Trash Folder/.AppleDouble/.Parent' cp: preserving times for `/job_001/Network Trash Folder/.AppleDouble/.Parent': Operation not permitted cp: preserving times for `/job_001/Network Trash Folder/.AppleDouble': Operation not permitted cp: preserving times for `/job_001/Network Trash Folder': Operation not permitted `./* ASTERISK FOLDER' -> `/job_001/* ASTERISK FOLDER' `./* ASTERISK FOLDER/M*A*S*H [1970].avi' -> `/job_001/* ASTERISK FOLDER/M*A*S*H [1970].avi' cp: failed to preserve ownership for `/job_001/* ASTERISK FOLDER/M*A*S*H [1970].avi': Operation not permitted `./* ASTERISK FOLDER/Skyfall [2012].mkv' -> `/job_001/* ASTERISK FOLDER/Skyfall [2012].mkv' cp: failed to preserve ownership for `/job_001/* ASTERISK FOLDER/Skyfall [2012].mkv': Operation not permitted cp: cannot create directory `/job_001/* ASTERISK FOLDER/.AppleDouble': File exists cp: failed to preserve ownership for `/job_001/* ASTERISK FOLDER': Operation not permitted `./.AppleDouble/.DS_Store' -> `/job_001/.AppleDouble/.DS_Store' cp: failed to preserve ownership for `/job_001/.AppleDouble/.DS_Store': Operation not permitted `./.AppleDouble/.Parent' -> `/job_001/.AppleDouble/.Parent' cp: failed to preserve ownership for `/job_001/.AppleDouble/.Parent': Operation not permitted cp: failed to preserve ownership for `/job_001/.AppleDouble': Operation not permitted `./.AppleDesktop/.volinfo' -> `/job_001/.AppleDesktop/.volinfo' cp: preserving times for `/job_001/.AppleDesktop/.volinfo': Operation not permitted cp: preserving times for `/job_001/.AppleDesktop': Operation not permitted cp: failed to preserve ownership for `/job_001/.': Operation not permitted
Backup failed. Thu Mar 10 17:41:49 GMT 2016.
Reason for failure: Error encountered copying data from source path /V1_SHARE/ ==> 192.168.0.4:/V2_SHARE due to permission denial on one or more files.
Note the "choose file" option when click on the "insert image" icon. That uploads a local file to the forum database, so you don't need to use a web link.
Yes, they did copy. But I only know that by empirical observation, not by seeing a nice completion message! 😞
I tried BlueHarvest, which does get rid of these invisible files, but it waits for them to be created, and then deletes them, rather than stopping them from being created. This means I still can't get a clean backup, or at least a clean backup which tells me it is so.
I enabled invisible files in the Finder to see what was going on, and the files are definitely created, and then deleted, and not once, but twice. There had been invisible files in the original share, but I deleted them before starting the backup for the second time, so they were definitely being created, not copied.
I don't really care about these files myself, as there are no PCs on this network, so no-one's ever gonna see them. But I do care about backups with clean completions.
What I really don't get at all, is what on earth this even has to do with Apple and OS X anyway. It's not doing the copying, which continues after the computer is switched off. Its scratch disk is not being used, and nor are its file system or OS. So it must be the NAS that's creating these files, not OS X. Mustn't it?
It appears that if you ask on Apple forums, it's Netgear's 'fault', and if you ask on Netgear forums, it's Apple's. But if it can't be fixed, that's academic!
OSX is creating these files on the share. Then the NAS is unable to copy them.
I'm sorry, but I absolutely do not believe this. My computer might be amazing, but it is not magic, Arthur C Clarke notwithstanding. It is switched off. It is not plugged in. How can it possibly be creating files on the NAS? It can't, it just can't. The NAS is doing it, and it is really annoying, to say the least.
I specifically posted this question on this part of the forum, because it's about using ReadyNASes in OS X. No offence meant, but is there anyone here who does know about such things? BTW, that link doesn't work, it just takes me to a page to reply to my own message!
You see that it (and many other similar posts) all say that OSX created these problem files in the share when you copied the main files to the NAS from the mac.
There are some suggestions there on how to prevent them from being formed in the first place.
Thanks Stephen. Sorry for the delay, I've been otherwise occupied for a few days!
Anyway, I still do not understand how my computer (the Mac) can create files on another computer (the NAS), when it (the Mac) is switched off. And until I understand that, nothing else makes sense. I'm in touch with the chap who makes BlueHarvest, as he is a Mac person, so perhaps he knows what's going on here, and can help me get to the bottom of it. Otherwise, it feels a bit like I'm asking questions in English, and getting answers back in French, only some of which I understand! (I do appreciate you're trying to help here, but I'm very frustrated at the apparent complete lack of Mac support. And as for how these NASes operate in different (Mac) user accounts, there's no consistency at all!)
Also, as I still don't believe a word of this, I'm going to try my own experiment. I'm going to borrow a Windows PC, which has never been a Mac in its life, download a movie (on a different network), and then put it up on the NAS, on this network. I shall do this both with the Mac switched on and connected to the LAN, so I can monitor for these files (having switched BlueHarvest off and enabled invisible files in Finder), and with it switched off.
To quote Captain Oates, "I may be some time!" But, like Arnie, "I'll be back!" 🙂
Anyway, I still do not understand how my computer (the Mac) can create files on another computer (the NAS), when it (the Mac) is switched off. And until I understand that, nothing else makes sense.
I never said the mac was creating files when it was turned off. The metadata files are created when the files are copied from the mac to the NAS shares (and some might also be created when mac browses the shares). They are hidden files, so you won't normally see them.
Per the logs, the issue is that these pre-existing and hidden files aren't being copied successfully from the share to the backup.
Are you saying that you've turned the mac off, then deleted the metadata files, confirmed that the deletions were successful, and then ran the backup (with the mac off for all the steps)?
I never said the mac was creating files when it was turned off.
No, you didn't, and nor did I say you did. Anyway, as I said, I'm going to do some experimenting, with the help of Ross from BlueHarvest. I shall apply Sherlock Holmes's maxim of eliminating the impossible, and will be left with the truth, however improbable or nonsensical.
l
