NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Whole-volume defragmentation on volume with nothing but auto-defrag shares?
Hello ReadyNAS community, I have owned a RN526X and EDA500 expansion for several years now, and both have been working great so far. I have a general maintenance question about whole-volume disk ...
ReadyNASser wrote:
I use ReadyDR for scheduled backups to the EDA500. I don't have any of its read-only ReadyDR shares set to auto-defrag, as it seems to not be needed (a manual whole-volume defragmentation typically completes in a few seconds, indicating that nothing much was fragmented).
There is a trade-off here that you might not be aware off. Snapshots only hold changes from the main share. When you modify a file in the main share (for example a update a database), then the system fragments the file. The original blocks end up in the snapshots, and the changed blocks are in the main share.
When you defrag that share, then the full original file ends up in the snapshot - which increases the total disk space needed for the share.
ReadyNASser wrote:
But is this still advisable to do from time to time, even with auto-defrag enabled on every share? Is it doing something that per-share auto-defrag cannot?
That's a good question. The only difference I can see is that auto-defrag is likely triggered by some threshold, and the manual defrag isn't. But there's nothing documented on this.
If the manual defrag is taking 8 hours, it clearly is doing something. The last time I ran it on my NAS it took about 3 hours (and autodefrag is set on most of my shares).
StephenB wrote:There is a trade-off here that you might not be aware off. Snapshots only hold changes from the main share. When you modify a file in the main share (for example a update a database), then the system fragments the file. The original blocks end up in the snapshots, and the changed blocks are in the main share.
When you defrag that share, then the full original file ends up in the snapshot - which increases the total disk space needed for the share.
Thanks StephenB. That would explain why my free space decreased by about 600 GB just after I last ran volume defrag. Snapshots occupy just over 770 GB on the volume.
StephenB wrote:
That's a good question. The only difference I can see is that auto-defrag is likely triggered by some threshold, and the manual defrag isn't. But there's nothing documented on this.
If the manual defrag is taking 8 hours, it clearly is doing something. The last time I ran it on my NAS it took about 3 hours (and autodefrag is set on most of my shares).
I noticed after I ran the most recent volume defrag, the nightly ReadyDR backup to the EDA500 of my music share took nearly four hours to complete (and I hadn't modified any files), probably because the volume defrag had moved files and snapshots around, and those changes then needed to be copied to the EDA500.
I like ReadyDR because it works in the background, can run multiple jobs at once, and is a copy of a snapshot of a share; I've never used the file-based backup functionality of the NAS, but presume it would need to copy things over in the same way as ReadyDR after volume defrag is run.
ReadyNASser wrote:
I've never used the file-based backup functionality of the NAS, but presume it would need to copy things over in the same way as ReadyDR after volume defrag is run.The file-based backups (for instance rsync) just copy over files that have changed since the previous backup. You can use snapshots on the destination share to get some versioning. Defragging a file shouldn't result in copying it, since the file attributes (size, permissions, timestamps, etc) wouldn't have changed.
FWIW, if the backup source is local to the NAS, then the NAS will create a temporary snapshot of the source and back that up. That does ensure coherency.
ReadyDR will copy all the data blocks that have changed in the snapshot, so it will re-copy defragged files. On the other hand, it only copies the changed data blocks (while Rsync needs to copy the entire file).
ReadyNASser wrote:
Thanks StephenB. That would explain why my free space decreased by about 600 GB just after I last ran volume defrag. Snapshots occupy just over 770 GB on the volume.
If you are using the "Smart" snapshots, then the monthly snapshots are retained indefinitely - they eventually fill the NAS, and require manual deletion. Personally I use the "Custom" setting - taking snapshots only when there are changes, and explicitly limiting retention to 3 months. That takes up about 5% of the space used (though of course that will vary depending on the amount of change).
StephenB wrote:The file-based backups (for instance rsync) just copy over files that have changed since the previous backup. You can use snapshots on the destination share to get some versioning. Defragging a file shouldn't result in copying it, since the file attributes (size, permissions, timestamps, etc) wouldn't have changed.
FWIW, if the backup source is local to the NAS, then the NAS will create a temporary snapshot of the source and back that up. That does ensure coherency.
Interesting, thanks for the info. I may try some file-based backup operations eventually (probably using some external drives plugged into the NAS via USB), so that's good to know.
StephenB wrote:If you are using the "Smart" snapshots, then the monthly snapshots are retained indefinitely - they eventually fill the NAS, and require manual deletion. Personally I use the "Custom" setting - taking snapshots only when there are changes, and explicitly limiting retention to 3 months. That takes up about 5% of the space used (though of course that will vary depending on the amount of change).
I use "Custom" snapshots only when there are changes as well. After trying out "Smart" snapshots for a little while, I received a few warnings about volume capacity being less than 30% free, and switched. After experimenting with snapshot retention settings, I now no longer need to manually remove any, which is a real timesaver.
ReadyNASser wrote:
Interesting, thanks for the info. I may try some file-based backup operations eventually (probably using some external drives plugged into the NAS via USB), so that's good to know.One useful trick here - you can use rsync for USB backups if you use the loopback IP address (127.0.0.1). If you use the loopback address on the backup source, the NAS won't make a temporary snapshot. Using loopback for the destination USB will result in the temporary snapshot - but it is a bit trickier to set up. You'd have to create a share on the USB drive, and enable rsync for it.
My own backups (NAS->NAS) generally run on the destination NAS. I don't worry about coherency, since the backups are run off-hours (and I'm not backing up anything where loss of coherency is a big deal).
ReadyNASser wrote:
After experimenting with snapshot retention settings, I now no longer need to manually remove any, which is a real timesaver.Totally agree. I really don't understand why the "Smart" snapshots have no retention setting. It makes the feature completely useless to me (actually it's worse than useless, it actually does damage by filling up the file system).
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
