I started a defrag on my RN104 about 60h ago and it still hasn't finished. It has so far eaten up about 70GB and I'm getting low on free space, so I want to move files off the NAS to make more room for defrag to do its thing.
The defrag is taking up almost 100% of the cpu so the system is very sluggish and the web interface is not loading fully. I can execute commands via ssh though.
My question is if it's safe to restart the NAS and cancel the defrag? If so, what is the recommended way to do a reboot via ssh; the "reboot" command or "rn_shutdown"?
Does the GUI give a suggestion as to how much longer it has left. I would leave it alone, if you must shut down I'd do it using the GUI. The OS should then prevent a shutdown that would cause a problem. Why do you want to do it using ssh?
My guess is that it's safe to do a reboot while the defrag is in progress. Would be a very dangerous feature if it couldn't handle that… But it would be useful to know if there's a preferred way to do it.
I recall your recent post and this long defrag is probably related to the amount of fragmentation on your system as well as the fact that you have a 104. Personally, I would wait.
Yes I agree that the defrag is taking a long time because of the amount of fragmentation.
My concern is that the process is eating up space and I'm currently below 10GB free. If I let it run it will probably run out of space within 24h, and that can't be good...
Oh I didn't realise you were that close to capacity. There's another reason the defrag is taking so long. I think you are going to have to take a chance and interrupt it. Can you not even see the volume control icon on the GUI to cancel the defrag? 2nd best I think would be to try to power down using the chassis power button. (2 or 3 presses according to hardware manual)
I managed to get the GUI to load far enough so I could use the reboot option. But unfortunately it doesn't seem to work. It's been stuck saying "Rebooting now…" for an hour now.
I'm still connected via ssh, and the free space is now down to 5GB. I think I need to try something harsher, like "reboot -f"
Thanks for the suggestion, but "killall awk btrfs" didn't do it. Still not rebooting and the top process is still flush-btrfs-1 with 99% cpu.
Edit: I spoke too soon. After 15 mins or so it suddenly rebooted. 🙂 It didn't let me connect via afp just now, but I'll let it sit a while and see if it returns to normal.
Edit2: After chugging with cnid_dbd for about 10 minutes it now seems to have returned to normal. Will start moving stuff off the NAS to make room for further defragging.
After freeing up about 250GB I started the defrag again and it completed in 6h this time.
It did however consume 200GB of space. :shock: Is this because COW:ed data is duplicated during defrag? If so, what is the best way to reclaim it? Would running something like bedup be a good idea?
Yes I'd be interested to know about the space consumption during defrag. I had initially thought the space was temporarily used while moving stuff around, but from what you describe it's a permanent increase. I suspect it has something to do with COW.
I don't have any snapshots, plus the space disappeared when doing the defrag. Is this a known side-effect of defragging?
I tried scheduling a balance (is there a way to run one immediately?) but it seems to have failed. readynasd was restarted seconds after starting the balance task and no disk activity occurring after that.
From status.log:
[14/12/19 19:29:54 CET] notice:volume:LOGMSG_SCRUBSTARTED_VOLUME Scrub started for Volume 'data'. [14/12/20 23:24:37 CET] notice:volume:LOGMSG_SCRUBCOMPLETE_VOLUME Scrub completed for Volume 'data'. [14/12/20 23:40:01 CET] notice:volume:LOGMSG_BALANCE_STARTED_VOLUME Balance started for Volume 'data'. [14/12/20 23:40:44 CET] info:system:LOGMSG_READYNASD_ABORTED ReadyNASOS service or process (readynasd) was restarted. [14/12/20 23:40:45 CET] warning:volume:LOGMSG_VOLUME_USAGE_WARN Volume 'data' usage exceeds 95 % of size (5.4 TB). [14/12/20 23:40:46 CET] info:system:LOGMSG_START_READYNASD ReadyNASOS background service started.
The balance completed successfully in just under 9h. There's no big change in available space. I'm still missing about 300GB that disappeared after the defrag.
Googling about this problem led me to this page, saying:
Defragmenting a file or a subvolume that has a copy-on-write copy results breaks the link between the file and its copy. For example, if you defragment a subvolume that has a snapshot, the disk usage by the subvolume and its snapshot will increase because the snapshot is no longer a copy-on-write image of the sub volume.
… not sure if this is accurate for the btrfs version ReadyNAS is using, but it's my understanding that files created with CoW will remain so even if you turn off that feature. So could this explain why the data went missing even though I had disabled CoW before doing the defrag? If so, the missing data would be the files created since OS 6.2 turned on CoW.
Is there a command to list CoW files? All I could find was lsattr which only shows files flagged NOT to use CoW.
6.2 didn't enable CoW. It added the option to disable CoW.
I was under the impression that if you didn't have snapshots enabled, CoW wasn't enabled either? Because the share where I downloaded my torrents had snapshots disabled, and when checking after upgrading to 6.2 the CoW option was on. That's why I'm thinking 6.2 enabled it for me.
One option would be to create new shares, copy your data into those shares and then delete the original shares.
Do I have to create a new share or can I copy the data to an external disk, delete the original files and then copy the files back? This would be preferable since I'm low on space on the NAS but have a 2TB USB disk I could use as temporary storage.
When you run lsattr on a file you will see if CoW is enabled or not.
How would lsattr show files with CoW enabled vs disabled? All the files I have checked show just blanks (no flags), e.g.
No CoW was on regardless of the snapshot setting. On the 104, new shares created from 6.2.0 onwards have CoW disabled by default.
If you are going to copy the files to a USB disk, you may as well do a factory default (wipes all data, settings, everything) and restore your data from backup. This will give you a clean setup on the firmware you currently have installed.
So… if CoW is the cause of the missing space after the defrag, wouldn't that make defragmenting a rather risky option?
If you are going to copy the files to a USB disk, you may as well do a factory default.
I can't do a full backup because I only have 2TB external disk capacity and my NAS is bigger than that.
Would deleting files reclaim the lost space? If so, I could do it in batches (copy 2TB, delete the originals, copy back the 2TB and repeat that until all files have been handled).
