× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

ReadyNAS DR taking too long with IBM domino databases

denz09
Aspirant

ReadyNAS DR taking too long with IBM domino databases

I would like to know is there going to be improvement with ReadyDR copying IBM Domino nsf files because it doesn't seem to copy just a change it copies everything because every backup is taking 7 hours. 

 

The same thing is happening when I use readynas share to store vhdx file that I use for windows server backup it is taking days to backup up instead of change which shouldn't be that much it is copying everything so either I have something configured wrong or that is just how it is. 

 

Apreciate feedback 

Model: ReadyNAS-3100|ReadyNAS-3100
Message 1 of 6

Accepted Solutions
StephenB
Guru

Re: ReadyNAS DR taking too long with IBM domino databases


@denz09 wrote:

In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr. 

 


That explains the problem.  Even if you use rsync in the backup job to do incremental backups, it will copy the entire file if something in it changes.  So if one block in NSF file changes, the backup job copies the entire file again.  ReadyDR (and btrfs snapshots generally) will treat the entire file as new - so they will also copy the entire file.

 

There is an option in rsync that uses checksums on each 2kb block, and that will only rewrite changed blocks in the nsf file.  But that option isn't used in the ReadyNAS backup jobs.  If the domino server is running linux, you could try running rsync with that option on it, instead of using the NAS backup job. Or you could try using ssh and running rsync manually on the ReadyNAS.  If that works, you could create your own cron job to do the domino->readynas backup.

 

I don't know of any good way to handle this if the domino server (or host of the vhdx files) is running windows.  I guess you could write a utility that compares two files, and re-writes blocks on the destination that differ.

 

No matter how you do it, if you only update changed blocks on the NAS you will end up with fragmented files.  And if they are defragmented, then the entire file will (again) be treated as new.  So you should probably turn off auto-defrag on the shares holding these files, and run scheduled defrags using the volume page setting.

 

 

 

 

View solution in original post

Message 4 of 6

All Replies
StephenB
Guru

Re: ReadyNAS DR taking too long with IBM domino databases


@denz09 wrote:

I would like to know is there going to be improvement with ReadyDR copying IBM Domino nsf files because it doesn't seem to copy just a change it copies everything because every backup is taking 7 hours. 

 

The same thing is happening when I use readynas share to store vhdx file that I use for windows server backup ...

 

Appreciate feedback 


ReadyDR is built on the snapshot feature of BTRFS.  It is incremental in the sense that it will only copy blocks that were written since the previous backup.  But it isn't copying differences from the original file.  That is, if a block is rewritten with the same data it had before, it will be copied again.  

 

So how much data is transfered depends on the details on how the file on the NAS is updated. If the domino server is directly updating the NSF file on the NAS, then only those updated blocks will be transfered with ReadyDR.  But if a backup process is copying the full updated NSF file from the domino server to the NAS, then ReadyDR will transfer the entire file - even if the updated file is identical to the original that was already on the NAS.  

 

Another scenario: if the file is defragmented, then the full file is re-written by the file system.  So again, the full file would end up transfered by ReadyDR.

Message 2 of 6
denz09
Aspirant

Re: ReadyNAS DR taking too long with IBM domino databases

Thank you for the feedback.

 

I will explain how it is set up so maybe I did it wrong way. IBM now HCL Domino is located on the server and everynight it gets shut down for few hours for the backup. In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr. 

 

 

Message 3 of 6
StephenB
Guru

Re: ReadyNAS DR taking too long with IBM domino databases


@denz09 wrote:

In the readynas backup I have created a job to copy the files from the server to the share and then it gets a daily snapshot and gets transferred over readydr. 

 


That explains the problem.  Even if you use rsync in the backup job to do incremental backups, it will copy the entire file if something in it changes.  So if one block in NSF file changes, the backup job copies the entire file again.  ReadyDR (and btrfs snapshots generally) will treat the entire file as new - so they will also copy the entire file.

 

There is an option in rsync that uses checksums on each 2kb block, and that will only rewrite changed blocks in the nsf file.  But that option isn't used in the ReadyNAS backup jobs.  If the domino server is running linux, you could try running rsync with that option on it, instead of using the NAS backup job. Or you could try using ssh and running rsync manually on the ReadyNAS.  If that works, you could create your own cron job to do the domino->readynas backup.

 

I don't know of any good way to handle this if the domino server (or host of the vhdx files) is running windows.  I guess you could write a utility that compares two files, and re-writes blocks on the destination that differ.

 

No matter how you do it, if you only update changed blocks on the NAS you will end up with fragmented files.  And if they are defragmented, then the entire file will (again) be treated as new.  So you should probably turn off auto-defrag on the shares holding these files, and run scheduled defrags using the volume page setting.

 

 

 

 

Message 4 of 6
denz09
Aspirant

Re: ReadyNAS DR taking too long with IBM domino databases

Thank you for that detailed reply that has given me something else to think about and try. Unfortunately Domino is on the Windows server.

 

It is disappointing why netgear is abandening readyshare replicate which was a supperior service with an inferior service it just doesn't make sence. 

Message 5 of 6
denz09
Aspirant

Re: ReadyNAS DR taking too long with IBM domino databases

For users that have domino now it is HCL Domino I have enabled DAOS but after some reading it is only for new attachments and have now added the following command Compact -c -daos on mail file name and it has reduced the size of the nsf file. Now the backup has gone from 6 hours to 2 hours and there are still few more databases to go which will probable reduce it well under an hour. 

 

 

Message 6 of 6
Top Contributors
Discussion stats
  • 5 replies
  • 1213 views
  • 2 kudos
  • 2 in conversation
Announcements