Should I ever run a Defrag operation if I only every insert/append to NAS (with no Snapshots enabled

Question

I'm back... :)&nbsp;Just making sure my "theory is correct", as I am not sure of the nuances with BTRFS.&nbsp;Do I ever need to run a Defrag operation for the use case below?&nbsp;All I had scheduled was Disk Test&nbsp;and Scrub operations, but as I learnt this week I should also run&nbsp;Balance&nbsp; operations periodically to optimize meta-data / allocation.&nbsp;Use CaseI ever only copy files to my ReadyNAS, and as a result have no snapshots configured as there is no need.So virtually never delete or overwrite the files&nbsp; The exception is when a file copy operation fails for whatever reasons so it's a partial copy-on-write, but that was/is extremely rare.&nbsp;TIA

Sandshark · Answer

rn_enthusiast&nbsp;wrote:&nbsp;I have a little self-made script that will pull filesystem stats every couple of weeks and dump that into a file. Admittedly, I look at the file probably every 3 months cause mostly I just forget about it :) But then again, if an issue does arise I am likely able to fix it anyway. For others it seems more&nbsp;reasonable with a "proactive" approach, indeed.I do something similar every day so it's fresh when I look at it.&nbsp; It over-writes the old file, and it takes insignificant CPU time, so it does no harm to run more often.&nbsp; On a system with drive spin-down enabled, it would wake them up, so that is a consideration.&nbsp; Scheduling it in the same time frame as backup jobs would insure the drives are already awake.&nbsp; On my backup systems which only power on for backups, they even RSYNC the report to my main NAS so I don't have to power them on to check their status.

rn_enthusiast · Answer

I have a similar use case to yours. My NAS is primarily a media library where files are written once and then read when played back. Rarely are files changed or deleted. I have never defragged it and it is as smooth as day one. This is a CoW filesystem and fragmentation will occur over time, especially when using snapshots. In the other big CoW filesystem, ZFS, you can't even defrag - it is not an option.

If you NAS runs fine and you have no performance issues, then don't defrag. That is my opinion. It is worth keeping in mind that defragging can increase data usage on the volume as it can cause data bloat because snapshot reflinks might have to be broken as part of the defrag process.

https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-filesystem

I stay away from defrag unless I really have to and in that case I would use the command line to defrag individual files that causes performance issues, rather than the entire volume.

StephenB · Answer

rn_enthusiast&nbsp;wrote:
&nbsp;
If you NAS runs fine and you have no performance issues, then don't defrag. That is my opinion. It is worth keeping in mind that defragging can increase data usage on the volume as it can cause data bloat because snapshot reflinks might have to be broken as part of the defrag process.
&nbsp;

Personally I do defrag my volumes every three months via the volume maintenace schedule, and haven't had any issues with space usage doing that.&nbsp;&nbsp;
&nbsp;
But overall, I agree that it's fine not to run them (and disable the auto defrag option for the share), and only do defrags when you run into performance issues due to fragmented files.&nbsp;
&nbsp;
Defragmenting a file that is in a snapshot (or otherwise reflinked) can increase the on-disk usage.&nbsp; But this only happens if the file was modified in place on the NAS.&nbsp; Personally I rarely do that (though re-tagging media files is one exception).&nbsp;&nbsp;
&nbsp;
Platypus69&nbsp;&nbsp;wrote:
... as I learnt this week I should also run&nbsp;Balance&nbsp;operations periodically to optimize meta-data / allocation.
&nbsp;

Just to expand on this a bit.&nbsp; BTRFS allocates space in chunks - and the usual chunk size is 1 Gigabyte.&nbsp; It's not that good at keeping the chunks full, so you often end up with quite a few partially filled chunks.&nbsp; This results in free space that the file system can't actually use.&nbsp; Even in your particular usage, you will end up with these partially filled chunks.
&nbsp;
One thing that the balance does is that it consolidates these partially filled chunks - which creates fully empty ones that the file system can use.
&nbsp;
Also, defragging a file can increase the number of partially filled chunks, since it is basically rewriting the file into unallocated space - leaving some of the original chunks partially filled.
&nbsp;

rn_enthusiast · Answer

Back in the days, in support we did come across some extreme cases where people literally acutally ran out of storage mid/post-defrag. So, it is certainly a thing that can happen depending on how the snapshot/reflink situation is - something not easy to see/control from a user-level.

It is something that can potentially occur. For me, it is the same as with balance. I run it only if I need to but then again, I would know if/when I need to. So, it is fine that Netgear allows people to do it on a scheduled basis to ensure these tasks are done - I'm not against that :)

StephenB · Answer

rn_enthusiast&nbsp;wrote:
For me, it is the same as with balance. I run it only if I need to but then again, I would know if/when I need to.

Part of the puzzle here is that with BTRFS there is a big difference between free space and unallocated space.&nbsp; BTRFS needs enough unallocated space in order to work correctly.&nbsp; If it runs totally out of unallocated space, a balance might not be able to fix it.&nbsp;&nbsp;
&nbsp;
Unfortunately, the admin UI doesn't report the amount of unallocated space - so you either need to monitor it with SSH commands, or you need to use free space as a rough proxy for unallocated space.&nbsp; And that is rough - what started this discussion was a volume that had ~700 GiB of free space, but no unallocated space.
&nbsp;
I think for almost all users here, the best approach is to be generous on the amount of free space you have, and run periodic balances to make sure the system maintains enough unallocated space. Most posters don't have the linux skills and BTRFS knowledge needed to assess the health of the file system with ssh (and resolve any issues), and many of the rest simply aren't going to monitor the file system that closely (until something goes wrong).&nbsp;
&nbsp;
&nbsp;

Forum Discussion

Should I ever run a Defrag operation if I only every insert/append to NAS (with no Snapshots enabled

8 Replies

Related Content

I Don't understand how the RS300 operates.

Snapshots on RN528 OS6

MS510TXM, fan status, operating temperature!

Compatibilité Carte SIM Nano Opérateur Français

High Operating Temp Environment

NETGEAR Academy

ProSupport for Business