NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
ReadyNAS os 6 Snapshots
IS there a good description anywhere of how snapshots are implemented? I ask because they are taking up a lot of space on my system and I want to get rid of some, BUT I want to understand whether a s...
They are standard BTRFS snapshots, so you can Google for more info.
They consist of additional pointers to data that has not changed and unique data that has changed since the snapshot. So, they take up some additional data, and the amount taken up depends on how much "churn" there is in the share. Shares used for tempaorary files or those with files that are changed often (like some databases) are not good candidates for snapshots unless you can accept the space useage. Defrag is also contrary to snapshot size management. Data moved for defrag will have the old location of the data retained in snapshots and the new in location the active files. While it's equal data, it's not "the same data", and the snapshot system only knows whats not the exact same.
There was a version update some time back that could cause retention of "invoisible" snapshots. If you think you could be suffering from that, the only solutions are via SSH (done yourself or by paid Netgear support) or destroying and re-creating at least the share, if not the whole volume. In doing that, you lose all yor snapshots, of course.
Ah, thank you. That's what I needed to get started. Much appreciated!
There are some hidden snaps (system reported it couldn't delete the share due to non deleted snapshots) but those are on a share created for Owncloud which is no longer used so I can deal with that when I come to flatten and rebuild the NAS as there's no data in it.
The majority are very visible, both in the snapshot directories on the network shares and through the admin interface. The versions don't seem to coincide with the schedules as far as I can see, even allowing for creeping under 5% free space and auto trimming of snapshots. This triggered the requirement to do something as performance really has fallen off for day to day file writing!
I'm building a FeeNAS box at the moment (couldn't justify £1K on the size and performance Netgear box I'd like for the drives when I have chassis, mobo with fast SATA and network ports and core I7 processor sitting around) so other than performance issues in the short term I may just reorder my data with copy to that once up, flatten the RN104 then use it as a backup target for the FreeNAS box and pension off the old duo that's currently the online backup of the important data.
That will bring in a whole new topic - finding something to do the backup between the two, while leaving them essentially independant. Currently I use RSYNC, but that has limitations when you rename and delete stuff. Manageable with small file systems that are primarily single person use but not good otherwise.
rapple wrote:
The majority are very visible, both in the snapshot directories on the network shares and through the admin interface. The versions don't seem to coincide with the schedules as far as I can see, even allowing for creeping under 5% free space and auto trimming of snapshots. This triggered the requirement to do something as performance really has fallen off for day to day file writing!
You can disable the feature to include them in the network shares - on Windows PCs they can still be accessed by right-clicking files or folders, and selecting "Restore Previous Versions".
I also recommend switching them to "Custom" snapshots instead of the default "Smart" ones. The monthly snapshots taken in the "Smart" mode are retained indefinitely, and that will take a fair amount of space over time. With "Custom" you can set retention, and there is also a setting to only take snapshots if the share has changed.
One thing Sandshark didn't mention - that is the relationship between snapshot space usage and fragmentation. For example, imagine you have an unfragmented 1 gigabyte video file in the main share and in the snapshot. It only takes 1 gigabyte of on-disk storage, since the data blocks are shared. Now change the metadata tag in the video file - which only changes the first block of the file in the main share. What happens now is that the file in the main share is fragmented (with the first block being a fragment). Total on-disk storage is 1 gigabyte + 1 block. If you then defragment the file, BTRFS will stop sharing the data blocks, and the on-disk storage will grow to 2 gigabytes.
rapple wrote:
That will bring in a whole new topic - finding something to do the backup between the two, while leaving them essentially independant. Currently I use RSYNC, but that has limitations when you rename and delete stuff. Manageable with small file systems that are primarily single person use but not good otherwise.
I use Rsync myself, with custom snapshots set on the destination shares. The snapshot schedule takes the snapshots shortly before the Rsync backup runs. The Rsync backup jobs are configured to delete files in the destination that are no longer on the source - so the destination mirrors the source. What I like about this is that the snapshots give me roll-back to older backups if I need it.
You could do this on backups that use the RN104 as the destination.
rapple wrote:
There are some hidden snaps (system reported it couldn't delete the share due to non deleted snapshots) but those are on a share created for Owncloud which is no longer used so I can deal with that when I come to flatten and rebuild the NAS as there's no data in it.
FWIW, they are visible in ssh, and you can delete them manually using the btrfs subvolume delete command.
StephenB wrote:[RA] Can't work out how to unquote my sections in the RT Editor, so I'll use italics!
You can disable the feature to include them in the network shares - on Windows PCs they can still be accessed by right-clicking files or folders, and selecting "Restore Previous Versions".
[RA] In all my years in IT (and it's many) I've never noticed this before! Thank you. Will save me much time when I next screw up.
One thing Sandshark didn't mention - that is the relationship between snapshot space usage and fragmentation.
[RA] Interesting, won't be a problem with my Video files due to the way I prep them, but the ~ 1TB FLACs get updated quite often.
I use Rsync myself, with custom snapshots set on the destination shares. The snapshot schedule takes the snapshots shortly before the Rsync backup runs. The Rsync backup jobs are configured to delete files in the destination that are no longer on the source - so the destination mirrors the source. What I like about this is that the snapshots give me roll-back to older backups if I need it.
[RA] If I understand correctly you are using the snapshots here as a safety net for RSYNC ops on destination and give yourself a couple of (maybe more) months of "no I didn't want to move/get rid of that". Interesting approach. Needs care on the snapshot schedule to suit the "didn't want to do that" but guarantees a 100% backup for a period.
Now I know what I need to retain, having deleted the other snapshots I regained about .75 TB of space!
Should give me enough time to understand FreeNAS config fully and test it out properly before using in anger.
Thanks Both
rapple wrote:
[RA] If I understand correctly you are using the snapshots here as a safety net for RSYNC ops on destination and give yourself a couple of (maybe more) months of "no I didn't want to move/get rid of that". Interesting approach. Needs care on the snapshot schedule to suit the "didn't want to do that" but guarantees a 100% backup for a period.
Correct. I set the snapshot retention for 3 months for most of the destinations - which gives me a reasonable time window, and doesn't take a lot of disk space.. Some shares are set lower - in particular I have a share where I save PC image backups. Both the secondary backup on the destination NAS and the source share are set to 2 weeks, in order to keep the on-disk usage reasonable.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
