I've just done a complete set of onsite and offsite backups. The existence of the .AppleDouble files and a couple of other special Mac file types has confounded my verification of backups. I'd like a quick sanity check if possible.
On a NAS directory containing ~700 GB, ~86k files, which consists of mostly media, .mp3, mkv and .jpg files are the following differences between the original NAS and backup directory cause for any suspicion that my rsync backups have missed files:
I've used du -sh when logged into a local Mac on the network
-Original directory on NAS: 659 GB -Same directory backed up on USB drive: 702 GB
~40 GB is a pretty big difference, obviously
I also get different counts of files by about 100 files using "find /directory -type f | wc"
I'm very keen on any general advise for the verification of backups. This should be easier...
^ Thanks. The USB backup drive indeed has the larger file count and space used. I do have the option selected to delete files not on source. However, this was a first time (new BU drives) back up process so this was a full backup. As such I expected a very high correspondence between sizes and file counts.
Sizes (as stored on the disk) certainly might differ if the USB drive is not formatted as ext, or if the cluster size is different. However, 6% is more than I'd expect.
The extra files are something else again. Perhaps write the complete directory list to a file, and diff them???
External USB disks are always formatted as ext3. Yes 6% or so bigger was my concern. In another backed up directory, my single largest, 3+ TB, mostly .mkv files, the file counts and space used are identical, using du and find /directory -type f | wc.
I've tried diff but the output is perhaps 10 of thousands of lines and most it just list differences in those darn .AppleDouble files/directories...
What would be ideal would be a unix command to list all files in all original directories but not in the backup directory but exclude all files in all .AppleDouble directories in the original directories... Might have to post that request on a unix forum somewhere as I'm pretty much a noob...
I've tried diff but the output is perhaps 10 of thousands of lines and most it just list differences in those darn .AppleDouble files/directories...
Don't use diff, use a visual comparison program that shows the files side-by-side with differences highlighted. WinMerge is nice if you can run Windows apps; I don't know what's good for Mac.
swamp2 wrote:
What would be ideal would be a unix command to list all files in all original directories but not in the backup directory but exclude all files in all .AppleDouble directories in the original directories... Might have to post that request on a unix forum somewhere as I'm pretty much a noob...
Try this (which I haven't tested). It will create a file called "original.txt" at the root of your original directory, and a file called "backup.txt" in the root of your backup directory. Each file will contain a list of all the subdirectories and files in those directories, with relative pathnames and sorted for easy comparison:
cd /original_directory find . | sort > original.txt cd /backup_directory find . | sort > backup.txt
Then use your visual differencing tool to compare /original_directory/original.txt with /backup_directory/backup.txt.
Can't remember if you need quotes around .AppleDouble or not
This is fantastic, thanks so much. It seems to work without quotes and is now extremely close to what I need. Can I bug you for one more syntax help and clarify function?
Will recursively compare dir1 and dir2. It will find all items that are present in any directory in dir1 that is not also in the same location in dir2. It will exclude all results in directories in dir1 containing the text ".AppleDouble". Correct?
Is it easy to exclude an additional directories based on the presense of a different string. I basically need a logical OR in the grep.
