RN516 running 6.2.2 experiencing random hangs. Appears to be induced by usage (either NFS or CIFS) roughly daily. Without usage it can go days without a hang.
I was trying it as a VMFS datastore but it was just temporary.
Space utilization is pretty high (1TB free out of 13TB). There are a lot of snapshots, and I tried removing them, but I haven't found an easy way, other than clicking one by one. After 40 or so I gave up. I have some folders with nothing in them and ~40 snapshots that I just left alone. Is there an easier/quicker way to delete snapshots?
[2013/07/12 15:33:48] Factory default initiated due to new disks (no RAID, no partitions)! [2013/07/12 15:34:19] Defaulting to X-RAID2 mode, RAID level 5 [2013/07/12 15:34:24] Factory default initiated on ReadyNASOS 6.0.8. [2013/09/07 08:56:53] Updated from ReadyNASOS 6.0.8 to 6.1.1. [2013/09/22 08:08:29] Updated from ReadyNASOS 6.1.1 to 6.1.2. [2013/10/29 16:48:16] Updated from ReadyNASOS 6.1.2 to 6.1.4. [2014/01/05 18:00:45] Updated from ReadyNASOS 6.1.4 to 6.1.5. [2014/05/08 07:04:21] Updated from ReadyNASOS 6.1.5 (1386892237) to 6.1.7 (1396977042). [2014/06/12 14:42:25] Updated from ReadyNASOS 6.1.7 (1396977042) to 6.1.8 (1398980083). [2014/10/06 10:17:34] Updated from ReadyNASOS 6.1.8 (1398980083) to 6.1.9 (1409791183). [2015/02/01 11:14:34] Updated from ReadyNASOS 6.1.9 () to 6.2.2 (ReadyNASOS).
We have made some filesystem improvements since 6.0.8, some of which require a factory default to take advantage of. Consequently I would recommend that you backup your data, do a factory default (wipes all data, settings, everything) and restore your data from backup.
Your volume has a huge amount of metadata which suggests that it is likely very fragmented:
Label: '7c6e0b00:root' uuid: db21f750-1a59-4725-a4a7-df7a2301cdd7 Total devices 1 FS bytes used 1.02GiB devid 1 size 4.00GiB used 4.00GiB path /dev/md0
Label: '7c6e0b00:data' uuid: 529ab8c4-9789-4fe9-8ab9-8f85b6f4e2bb Total devices 1 FS bytes used 12.06TiB devid 1 size 13.62TiB used 13.01TiB path /dev/md127
Indeed I can see you have NFS enabled with a thread count of 8 which is very high.
BTRFS is a CoW (Copy on Write) filesystem. The CoW nature of the filesystem is what makes so many of its great features possible. However for some use cases CoW is bad. A NFS datastore for VMs is one of those. This is because when you make a huge number of in place modifications to files using CoW they will get very fragmented very quickly as each write does not overwrite the old data. In 6.2.x we added the option to disable CoW linking disabling this to disabling bitrot protection. Like disabling snapshots this should be done on an empty share. You can't disable CoW on existing files so disabling it on a share which already has files in it would be of little use.
I haven't used this as a VM datastore for 6ish months. I can delete those files if it will help. I am using it as an NFS share for 2 other machines that deal with writing both large and small files (but not any deletes or modifications). What is a better NFS thread setting?
The problem is it is hard to backup when it keeps hanging.
Also, I ran a defrag on the volume last night and it completed successfully. The btrfs cleanup processes that mdgm started last night are still running.
