ReadyNAS DR taking too long with IBM domino databases
I would like to know is there going to be improvement with ReadyDR copying IBM Domino nsf files because it doesn't seem to copy just a change it copies everything because every backup is taking 7 hours.
The same thing is happening when I use readynas share to store vhdx file that I use for windows server backup it is taking days to backup up instead of change which shouldn't be that much it is copying everything so either I have something configured wrong or that is just how it is.
In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr.
That explains the problem. Even if you use rsync in the backup job to do incremental backups, it will copy the entire file if something in it changes. So if one block in NSF file changes, the backup job copies the entire file again. ReadyDR (and btrfs snapshots generally) will treat the entire file as new - so they will also copy the entire file.
There is an option in rsync that uses checksums on each 2kb block, and that will only rewrite changed blocks in the nsf file. But that option isn't used in the ReadyNAS backup jobs. If the domino server is running linux, you could try running rsync with that option on it, instead of using the NAS backup job. Or you could try using ssh and running rsync manually on the ReadyNAS. If that works, you could create your own cron job to do the domino->readynas backup.
I don't know of any good way to handle this if the domino server (or host of the vhdx files) is running windows. I guess you could write a utility that compares two files, and re-writes blocks on the destination that differ.
No matter how you do it, if you only update changed blocks on the NAS you will end up with fragmented files. And if they are defragmented, then the entire file will (again) be treated as new. So you should probably turn off auto-defrag on the shares holding these files, and run scheduled defrags using the volume page setting.
I would like to know is there going to be improvement with ReadyDR copying IBM Domino nsf files because it doesn't seem to copy just a change it copies everything because every backup is taking 7 hours.
The same thing is happening when I use readynas share to store vhdx file that I use for windows server backup ...
Appreciate feedback
ReadyDR is built on the snapshot feature of BTRFS. It is incremental in the sense that it will only copy blocks that were written since the previous backup. But it isn't copying differences from the original file. That is, if a block is rewritten with the same data it had before, it will be copied again.
So how much data is transfered depends on the details on how the file on the NAS is updated. If the domino server is directly updating the NSF file on the NAS, then only those updated blocks will be transfered with ReadyDR. But if a backup process is copying the full updated NSF file from the domino server to the NAS, then ReadyDR will transfer the entire file - even if the updated file is identical to the original that was already on the NAS.
Another scenario: if the file is defragmented, then the full file is re-written by the file system. So again, the full file would end up transfered by ReadyDR.
Re: ReadyNAS DR taking too long with IBM domino databases
Thank you for the feedback.
I will explain how it is set up so maybe I did it wrong way. IBM now HCL Domino is located on the server and everynight it gets shut down for few hours for the backup. In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr.
In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr.
That explains the problem. Even if you use rsync in the backup job to do incremental backups, it will copy the entire file if something in it changes. So if one block in NSF file changes, the backup job copies the entire file again. ReadyDR (and btrfs snapshots generally) will treat the entire file as new - so they will also copy the entire file.
There is an option in rsync that uses checksums on each 2kb block, and that will only rewrite changed blocks in the nsf file. But that option isn't used in the ReadyNAS backup jobs. If the domino server is running linux, you could try running rsync with that option on it, instead of using the NAS backup job. Or you could try using ssh and running rsync manually on the ReadyNAS. If that works, you could create your own cron job to do the domino->readynas backup.
I don't know of any good way to handle this if the domino server (or host of the vhdx files) is running windows. I guess you could write a utility that compares two files, and re-writes blocks on the destination that differ.
No matter how you do it, if you only update changed blocks on the NAS you will end up with fragmented files. And if they are defragmented, then the entire file will (again) be treated as new. So you should probably turn off auto-defrag on the shares holding these files, and run scheduled defrags using the volume page setting.
Re: ReadyNAS DR taking too long with IBM domino databases
For users that have domino now it is HCL Domino I have enabled DAOS but after some reading it is only for new attachments and have now added the following command Compact -c -daos on mail file name and it has reduced the size of the nsf file. Now the backup has gone from 6 hours to 2 hours and there are still few more databases to go which will probable reduce it well under an hour.
