NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
OS6 stuck at booting / mounting btrfs
I rebooted my NAS today after it became a bit sluggish and prior to wanting to update to 6.9.1 (i'm at the latest 6.8.x version right now).
After rebooting the NAS didnt come up, and the LCD displays "Booting #########> 89%".
I can log in to the nas via SSH and I noticed that [btrfs-transaction] is consuming 99% CPU.
Some further digging shows me that it is seems to be hanging on "/bin/mount /dev/disk/by-label/33ea2307:data /data -t btrfs"
The last bit of dmesg is attached (whichttp://www.maxxmark.com/dropbox/nas.dmesg.out.txt h describes it indeed hanging).
Is there someway to see what (and if) it is still doing something? I was reluctant to just try and reobot it again. Any suggestions?
Edit: after a while it stopped from showing [btrfs-transaction] consuming 100% and went to the /bin/mount as described above to consume 100% cpu.
And the LCD display now showed:
Fail to startup
#################> 94%
Edit 2:
I just noticed that my nas around the time of rebooting mailed "Balance complete for volume data". So my guess it was balancing (which surely would explain the sluggishness of the nas). Is it possible it tried to (or did) cancel the balancing, and restarted the balancing on reboot, and is now continueing the balance?
I'm one small step closer.
I based it off of this post: https://askubuntu.com/questions/157917/how-do-i-recover-a-btrfs-partition-that-will-not-mount
I have done a btrfsck on the system which seems to state that there ís something wrong with my volume:
root@NAS:/var/log/frontview# btrfsck -p /dev/md127 Checking filesystem on /dev/md127 UUID: d3d6c597-3eb2-4923-bb50-ee21e11dcfec checking extents [o] checking free space cache [o] checking fs roots [O] checking csums checking root refs checking quota groups Counts for qgroup id: 0/262 are different our: referenced 12925101330432 referenced compressed 12925101330432 disk: referenced 12849220059136 referenced compressed 12849220059136 diff: referenced 75881271296 referenced compressed 75881271296 our: exclusive 12925101330432 exclusive compressed 12925101330432 disk: exclusive 12849220059136 exclusive compressed 12849220059136 diff: exclusive 75881271296 exclusive compressed 75881271296 Counts for qgroup id: 0/270 are different our: referenced 1132319191040 referenced compressed 1132319191040 disk: referenced 1131488161792 referenced compressed 1131488161792 diff: referenced 831029248 referenced compressed 831029248 our: exclusive 1132319191040 exclusive compressed 1132319191040 disk: exclusive 1131488161792 exclusive compressed 1131488161792 diff: exclusive 831029248 exclusive compressed 831029248 Counts for qgroup id: 0/277 are different our: referenced 20862586880 referenced compressed 20862586880 disk: referenced 20729131008 referenced compressed 20729131008 diff: referenced 133455872 referenced compressed 133455872 our: exclusive 20862586880 exclusive compressed 20862586880 disk: exclusive 20729131008 exclusive compressed 20729131008 diff: exclusive 133455872 exclusive compressed 133455872 Counts for qgroup id: 0/278 are different our: referenced 83479613440 referenced compressed 83479613440 disk: referenced 83248349184 referenced compressed 83248349184 diff: referenced 231264256 referenced compressed 231264256 our: exclusive 83479613440 exclusive compressed 83479613440 disk: exclusive 83248349184 exclusive compressed 83248349184 diff: exclusive 231264256 exclusive compressed 231264256 Counts for qgroup id: 0/279 are different our: referenced 41265213440 referenced compressed 41265213440 disk: referenced 33458343936 referenced compressed 33458343936 diff: referenced 7806869504 referenced compressed 7806869504 our: exclusive 41265213440 exclusive compressed 41265213440 disk: exclusive 33458343936 exclusive compressed 33458343936 diff: exclusive 7806869504 exclusive compressed 7806869504 found 14220525928448 bytes used, no error found total csum bytes: 141941444 total tree bytes: 1367834624 total fs tree bytes: 827392000 total extent tree bytes: 347897856 btree space waste bytes: 251305664 file data blocks allocated: 21737524551680 referenced 14189602930688
Then I followed up with mounting the partition:
root@NAS:/mnt# mount -t btrfs -o recovery,nospace_cache,nospace_cache /dev/md127 /mnt/datadisk/ root@NAS:/mnt# cd datadisk/
root@NAS:/mnt/datadisk# ls -l total 136 drwxr-xr-x 1 root root 230 Nov 23 18:09 . drwxr-xr-x 1 root root 16 Dec 20 18:32 .. drwxrwxrwx+ 1 admin admin 1352 Nov 17 00:56 Backup drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Documents drwxr-xr-x 1 admin admin 94 Aug 20 15:07 home drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Music drwxrwxrwx+ 1 guest guest 38 Sep 15 17:41 Pictures drwxrwxrwx+ 1 502 dialout 170 Dec 18 18:56 Videos root@NAS:/mnt/datadisk# du -sh 13T .
root@NAS:/mnt/datadisk# df -h Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 555M 3.1G 15% / tmpfs 998M 0 998M 0% /dev/shm tmpfs 998M 3.7M 994M 1% /run tmpfs 499M 1.1M 498M 1% /run/lock tmpfs 998M 0 998M 0% /sys/fs/cgroup /dev/md127 14T 13T 701G 95% /mnt/datadiskEdit:
I've unmounted the partition and tried to mount it normally again and that worked like a charm:
root@NAS:/mnt# mount -t btrfs /dev/md127 /mnt/datadisk/ root@NAS:/mnt# cd datadisk/ root@NAS:/mnt/datadisk# ls -l total 136 drwxr-xr-x 1 root root 230 Nov 23 18:09 . drwxr-xr-x 1 root root 16 Dec 20 18:32 .. drwxrwxrwx+ 1 admin admin 1352 Nov 17 00:56 Backup drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Documents drwxr-xr-x 1 admin admin 94 Aug 20 15:07 home drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Music drwxrwxrwx+ 1 guest guest 38 Sep 15 17:41 Pictures drwxrwxrwx+ 1 502 dialout 170 Dec 18 18:56 Videos
root@NAS:/mnt/datadisk# df -h Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 555M 3.1G 15% / tmpfs 998M 0 998M 0% /dev/shm tmpfs 998M 3.1M 995M 1% /run tmpfs 499M 1.1M 498M 1% /run/lock tmpfs 998M 0 998M 0% /sys/fs/cgroup /dev/md127 14T 13Now i'm contemplating on what to do next... I might just do a reboot and hope for the best (as it SHOULD come up again, because there is now nothing that should prevent the mount action from failing). But then again... I have this strange feeling that it might not come up correct again :)
4 Replies
I'm one small step closer.
I based it off of this post: https://askubuntu.com/questions/157917/how-do-i-recover-a-btrfs-partition-that-will-not-mount
I have done a btrfsck on the system which seems to state that there ís something wrong with my volume:
root@NAS:/var/log/frontview# btrfsck -p /dev/md127 Checking filesystem on /dev/md127 UUID: d3d6c597-3eb2-4923-bb50-ee21e11dcfec checking extents [o] checking free space cache [o] checking fs roots [O] checking csums checking root refs checking quota groups Counts for qgroup id: 0/262 are different our: referenced 12925101330432 referenced compressed 12925101330432 disk: referenced 12849220059136 referenced compressed 12849220059136 diff: referenced 75881271296 referenced compressed 75881271296 our: exclusive 12925101330432 exclusive compressed 12925101330432 disk: exclusive 12849220059136 exclusive compressed 12849220059136 diff: exclusive 75881271296 exclusive compressed 75881271296 Counts for qgroup id: 0/270 are different our: referenced 1132319191040 referenced compressed 1132319191040 disk: referenced 1131488161792 referenced compressed 1131488161792 diff: referenced 831029248 referenced compressed 831029248 our: exclusive 1132319191040 exclusive compressed 1132319191040 disk: exclusive 1131488161792 exclusive compressed 1131488161792 diff: exclusive 831029248 exclusive compressed 831029248 Counts for qgroup id: 0/277 are different our: referenced 20862586880 referenced compressed 20862586880 disk: referenced 20729131008 referenced compressed 20729131008 diff: referenced 133455872 referenced compressed 133455872 our: exclusive 20862586880 exclusive compressed 20862586880 disk: exclusive 20729131008 exclusive compressed 20729131008 diff: exclusive 133455872 exclusive compressed 133455872 Counts for qgroup id: 0/278 are different our: referenced 83479613440 referenced compressed 83479613440 disk: referenced 83248349184 referenced compressed 83248349184 diff: referenced 231264256 referenced compressed 231264256 our: exclusive 83479613440 exclusive compressed 83479613440 disk: exclusive 83248349184 exclusive compressed 83248349184 diff: exclusive 231264256 exclusive compressed 231264256 Counts for qgroup id: 0/279 are different our: referenced 41265213440 referenced compressed 41265213440 disk: referenced 33458343936 referenced compressed 33458343936 diff: referenced 7806869504 referenced compressed 7806869504 our: exclusive 41265213440 exclusive compressed 41265213440 disk: exclusive 33458343936 exclusive compressed 33458343936 diff: exclusive 7806869504 exclusive compressed 7806869504 found 14220525928448 bytes used, no error found total csum bytes: 141941444 total tree bytes: 1367834624 total fs tree bytes: 827392000 total extent tree bytes: 347897856 btree space waste bytes: 251305664 file data blocks allocated: 21737524551680 referenced 14189602930688
Then I followed up with mounting the partition:
root@NAS:/mnt# mount -t btrfs -o recovery,nospace_cache,nospace_cache /dev/md127 /mnt/datadisk/ root@NAS:/mnt# cd datadisk/
root@NAS:/mnt/datadisk# ls -l total 136 drwxr-xr-x 1 root root 230 Nov 23 18:09 . drwxr-xr-x 1 root root 16 Dec 20 18:32 .. drwxrwxrwx+ 1 admin admin 1352 Nov 17 00:56 Backup drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Documents drwxr-xr-x 1 admin admin 94 Aug 20 15:07 home drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Music drwxrwxrwx+ 1 guest guest 38 Sep 15 17:41 Pictures drwxrwxrwx+ 1 502 dialout 170 Dec 18 18:56 Videos root@NAS:/mnt/datadisk# du -sh 13T .
root@NAS:/mnt/datadisk# df -h Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 555M 3.1G 15% / tmpfs 998M 0 998M 0% /dev/shm tmpfs 998M 3.7M 994M 1% /run tmpfs 499M 1.1M 498M 1% /run/lock tmpfs 998M 0 998M 0% /sys/fs/cgroup /dev/md127 14T 13T 701G 95% /mnt/datadiskEdit:
I've unmounted the partition and tried to mount it normally again and that worked like a charm:
root@NAS:/mnt# mount -t btrfs /dev/md127 /mnt/datadisk/ root@NAS:/mnt# cd datadisk/ root@NAS:/mnt/datadisk# ls -l total 136 drwxr-xr-x 1 root root 230 Nov 23 18:09 . drwxr-xr-x 1 root root 16 Dec 20 18:32 .. drwxrwxrwx+ 1 admin admin 1352 Nov 17 00:56 Backup drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Documents drwxr-xr-x 1 admin admin 94 Aug 20 15:07 home drwxrwxrwx+ 1 guest guest 20 Aug 20 15:11 Music drwxrwxrwx+ 1 guest guest 38 Sep 15 17:41 Pictures drwxrwxrwx+ 1 502 dialout 170 Dec 18 18:56 Videos
root@NAS:/mnt/datadisk# df -h Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 555M 3.1G 15% / tmpfs 998M 0 998M 0% /dev/shm tmpfs 998M 3.1M 995M 1% /run tmpfs 499M 1.1M 498M 1% /run/lock tmpfs 998M 0 998M 0% /sys/fs/cgroup /dev/md127 14T 13Now i'm contemplating on what to do next... I might just do a reboot and hope for the best (as it SHOULD come up again, because there is now nothing that should prevent the mount action from failing). But then again... I have this strange feeling that it might not come up correct again :)
Another interesting thing is that I noticed the system started mailing me multiple mails that the data partition went from "inactive to redundant" and from "redundant to inactive" a few times after eachother. This started right after I mounted the partition manually (screenshot attached).
Decided to give it a reboot. I couldn't think of anything that I could do (next to doing a full backup from the recoverymount).
Choose shutdown from the webinterface, it rebooted, and came up again as it should!
So for future reference, you could do the same as I described above. Don't just assume all was lost, but check if it is actually gone.
As a warning I should say that the btrfsck output of mine didn't state that there were any problems (except for quota issues). There seemed to be no integrity errors or data errors. If there were, I would not have been comfortable to reboot the unit.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
