Forum Discussion

Virtuoso

Jun 25, 2016

snapshots and defrag - revisited...

I've seen several places in this forum where it's mentioned that defragmenting a share that has snapshots WILL cause the reference links between the primary files and the snapshot versions to be brok...

Other Discussions

garyd9

Virtuoso

Jun 25, 2016

I did some fairly extensive testing...

First, it appears that the breaking of reference links (snapshots) happens at a FILE level. Then, it will only happen IFF that particular file is fragmented when the defrag is run. So, the problem isn't nearly as bad as it's made out to be.

An example: assume you have a share with 200 files, each 1 MB in size. Of the 200 files, one of them is fragmented. Then you made a snapshot of the drive. (0 extra space used.) Then you run a defrag on the volume. During the defrag, the ONE file that was fragmented will be split from the snapshot, causing one extra megabyte of space to be used. That's it.

In that example, if none of the files were fragmented before the snapshot was made, then the defrag wouldn't have used any extra space.

If the files weren't fragmented, you made a snapshot, and then you modified one file (and caused it to fragment), the filesystem itself would use "copy on write" to make copies of the portions of the file that you modified, and modify only those portions. (So, the snapshot would reference the entire old file, and the non-snapshot would partially reference new data and partially reference the old file.) At this point, the non-snapshot file is fragmented, so when a defrag runs, it will "finish" splitting the file from the snapshot. It might be easier if you just assume that any modifications to a file that has snapshot copies will cause an entirely new file to be allocated on the disk (even if that's only partially true to begin with, it becomes fully true if you defrag.)

One other example is if you have those 200 files, and one of them is fragmented before you snapshot. Then you make 10 snapshots. So far, no extra space is being used. Then modify one file. Now if you defrag, then only one extra copy is made (and all 10 snapshots would share the still fragmented copy.) This assumes that netgear's s/w is not running the defrag on the snapshots. (It appears that netgear only is running the defrag on non-snapshot subvolumes. mdgm-ntgr, can you verify that?)

StephenB
Guru - Experienced User
Jun 28, 2016
garyd9 wrote:

First, it appears that the breaking of reference links (snapshots) happens at a FILE level. Then, it will only happen IFF that particular file is fragmented when the defrag is run.

Correct. Though there are some use cases where files are essentially always fragmented when snapshots are frequently taken. An SQL database is one example. If the current database is in a snapshot, then any database update will fragment the file. That would also happen with a torrent.
- garyd9
  Virtuoso
  Jun 28, 2016
  taking this a bit further...
  
  The frontview UI, for shares and LUNs has a configuration option called "Bit Rot Protection (Copy-on-write)" As far as I can tell, at the btrfs level, this controls the "C" attribute (as visible with lsattr and set with chattr.) btrfs documentation calls this a 'nocow' or 'nodatacow' attribute.
  
  A normal person (or even one with a technical background) would probably assume that turning of copy on write would prevent copy on write from occurring. This would be useful for preventing duplicate copies being made on defrag.
  
  Through no fault of Netgear, however, btrfs really mis-represents this attribute. It does NOT prevent copy on write. Oh, it will prevent you from doing "cp --reflink" from the shell, but that isn't something that most readyNAS users would be messing with anyway. It does NOT prevent snapshots and snapshots on nodatacow files, directories and subvolumes will STILL make use of COW regardless of this attribute. (I'm not sure if you can snapshot a subvolume mounted with the nodatacow mount option.)
  
  What this attribute DOES block is the data checksum that helps with "bit rot" detection.
  
  So... I have no idea what GOOD the attribute actually does. I suppose it might make disk writes a TINY bit faster (and perhaps btrfs scrubs a bit faster if all your shares have COW/bitrot protection turned off.)