NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
Where is the data in ReadyDR Jobs?
We have 2 ReadyNAS 4312x machines, a ReadyNAS 3312, and a ReadyNAS 428 at our company. One question I had about ReadyDR Jobs was where the data for those snapshots physically exist after they have been transferred from one machine to the next. For example, imagine this hypothetical situation: On NAS1 we have a folder called BusinessData that we snapshot hourly during business hours as this data changes constantly and we sometimes have to get data off these snapshots. We want to span this BusinessData folder across our other three NAS machines to keep our data safe in case of a failure. Instead of constantly moving data from NAS1 to NAS2, NAS3, and NAS4, where each machine would use resources to receive and then snapshot the data themselves, we simply use ReadyDR Jobs to move the snapshots from NAS1 to all other NAS machines. When looking at the snapshot folders on, say, NAS4, we can see all the data as if it were just a regular copy.
My question is: If I have 40 snapshots of a folder (BusinessData) that is approximately 1TB large, why do these snapshots in total not take up 40TB? I assume it has something to do with the data being referenced instead of literally copied, but I need to know how all of this works in case of a disaster. Is there some original copy from which all the snapshots simply say: "The data at this point in time is the original copy +/- these changes"? Any explanations or links to explanations would be appreciated.
Jason
Sandshark wrote:
The big advantages of ReadyDR seem to be synchronization of snapshots between the sending and receiving NAS and it's inherent versioning.
But the inability to simply put the backup NAS into service in place of the primary if the primary goes down was a big negative for me, much more so than any advantages of ReadyDR.
That was my assessment too. I also use rsync with snapshots enabled on the source and destination.
Sandshark wrote:
It may have some space saving advantages as well, but I couldn't identify any in my use case. It probably depends on how much churn there is in your data.ReadyDR is built on top of btrfs send, which will only send the blocks that were modified since the last backup. Incremental rsync backup jobs will only send the files that were modified since the last backup.
So ReadyDR can be more efficient - and that will be significant will be if you are backing up iSCSI LUNS or live databases. Rsync will need to copy the entire LUN or the entire database file every time it is updated. I don't have LUNS or live databases on my NAS, but if I did I'd likely back them up with ReadyDR.
Since ReadyDR is backing up snapshots, the backup is always coherent (the source isn't changing while the backup is being made). If rsync is running on the source machine, then it is also coherent - the system will make a snapshot, and then back that snapshot up. If rsync is running on the destination machine, the backup is not coherent (so you need to run it off-hours if you need that).
Another benefit of ReadyDR is that you can run multiple ReadyDR backups in parallel (normal backup jobs are queued, and run sequentially).
JasonBMechDev wrote:
Instead of constantly moving data from NAS1 to NAS2, NAS3, and NAS4, where each machine would use resources to receive and then snapshot the data themselves, we simply use ReadyDR Jobs to move the snapshots from NAS1 to all other NAS machines.
In my experience, the resouces needed to create a snapshot are inconsequential.
Either way, the receiving machines are using resources to receive the data. It is possible that ReadyDR can determine which blocks need to be sent more efficiently than incremental Rsync can. That's not something I looked at in my own (fairly brief) test of ReadyDR. I run daily backups, which complete fairly quickly - and the receiving NAS are dedicated to backup, so their resource use isn't a concern.
The time it would take to bring a backup system online definitely was a concern for me. You might want to try doing that on one of your backups, so you know the full procedure, and so you can set appropriate expectations.
4 Replies
JasonBMechDev wrote:
I assume it has something to do with the data being referenced instead of literally copied, but I need to know how all of this works in case of a disaster.
It's fundamental to the way BTRFS works. If you have a main share, and make a single snapshot, then right after the snapshot is created it takes up no space. The data blocks are shared between the share and the snapshot. If you then update a block in a file in the main share, the original block is still kept, and is pointed to be by the snapshot metadata. The file in the main share contains the updated block (and the file is fragmented). Similarly if you have multiple snapshots - the data blocks are shared.
JasonBMechDev wrote:Instead of constantly moving data from NAS1 to NAS2, NAS3, and NAS4, where each machine would use resources to receive and then snapshot the data themselves, we simply use ReadyDR Jobs to move the snapshots from NAS1 to all other NAS machines. When looking at the snapshot folders on, say, NAS4, we can see all the data as if it were just a regular copy.
ReadyDR doesn't work that way. It does put copies of snapshots on other machines, but they do not appear there as if they were just a regular copy -- they are read-only. They are designed primarily to be sent back to the originating machine for a restore, but can also be cloned (just like any other snapshot) to make a normal read/write share on the receiving NAS.
I use rsync backup jobs between my main and backup NAS and do have snapshots enabled on both sides, which serves as pseudo-versioning on the backup. I looked into switching to ReadyDR.
The big advantages of ReadyDR seem to be synchronization of snapshots between the sending and receiving NAS and it's inherent versioning. It may have some space saving advantages as well, but I couldn't identify any in my use case. It probably depends on how much churn there is in your data.
But the inability to simply put the backup NAS into service in place of the primary if the primary goes down was a big negative for me, much more so than any advantages of ReadyDR.
Sandshark wrote:
The big advantages of ReadyDR seem to be synchronization of snapshots between the sending and receiving NAS and it's inherent versioning.
But the inability to simply put the backup NAS into service in place of the primary if the primary goes down was a big negative for me, much more so than any advantages of ReadyDR.
That was my assessment too. I also use rsync with snapshots enabled on the source and destination.
Sandshark wrote:
It may have some space saving advantages as well, but I couldn't identify any in my use case. It probably depends on how much churn there is in your data.ReadyDR is built on top of btrfs send, which will only send the blocks that were modified since the last backup. Incremental rsync backup jobs will only send the files that were modified since the last backup.
So ReadyDR can be more efficient - and that will be significant will be if you are backing up iSCSI LUNS or live databases. Rsync will need to copy the entire LUN or the entire database file every time it is updated. I don't have LUNS or live databases on my NAS, but if I did I'd likely back them up with ReadyDR.
Since ReadyDR is backing up snapshots, the backup is always coherent (the source isn't changing while the backup is being made). If rsync is running on the source machine, then it is also coherent - the system will make a snapshot, and then back that snapshot up. If rsync is running on the destination machine, the backup is not coherent (so you need to run it off-hours if you need that).
Another benefit of ReadyDR is that you can run multiple ReadyDR backups in parallel (normal backup jobs are queued, and run sequentially).
JasonBMechDev wrote:
Instead of constantly moving data from NAS1 to NAS2, NAS3, and NAS4, where each machine would use resources to receive and then snapshot the data themselves, we simply use ReadyDR Jobs to move the snapshots from NAS1 to all other NAS machines.
In my experience, the resouces needed to create a snapshot are inconsequential.
Either way, the receiving machines are using resources to receive the data. It is possible that ReadyDR can determine which blocks need to be sent more efficiently than incremental Rsync can. That's not something I looked at in my own (fairly brief) test of ReadyDR. I run daily backups, which complete fairly quickly - and the receiving NAS are dedicated to backup, so their resource use isn't a concern.
The time it would take to bring a backup system online definitely was a concern for me. You might want to try doing that on one of your backups, so you know the full procedure, and so you can set appropriate expectations.
I just did a bit more testing with ReadyDR and re-discovered one of the other things I didn't like about it: As is the case with all snapshots, the amount of space taken up by a ReadyDR share does not appear in the GUI except were all snapshot space for the volume is added up.
But I got to wondering if there is another advantage. Are ReadyDR snapshots subject to the same snapshot pruning as snapshots of the shares on the NAS? If they are not, then it adds a lot of protection against the scenario where ransomware encrypts all the files and the backup NAS has to start pruning snapshots (and, thus, deleting the unencrypted copies) in order to make room for the new encrypted files. Since ReadyDR share snapshopts seem to follow all the other rules of other snapshots, I suspect there is no difference. But it would be good if someone could confirm or deny.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
