My NAS was gracefully rebooted but it was stuck at 97% for a long time so I forced reboot it. Unfortunately my volume is now missing. I can SSH to the NAS and also from GUI I can get to it but I get an error message no volumes are configured.
Searching shows me a few others have faced similar issue but that followed an action like a HDD replacement. In my case however I don't recall any hardware failure and the only known issue was inability to update AV software. My ReadyNAS OS is 6.9.3. SSH'ing into the NAS only shows /, /dev, /dev/shm, /run/ /run/lock, /sys/fs/cgroup filesystem's mounted.
Where can I go about troubleshooting what happened and get the volume back as all the data is in there. I have another NetGear NAS where I sync data weekly which has all my data intact but there is an encrypted volume on the failed NAS that has lots of critical data that isn't the latest. I am probably giving very little information for anyone to be able to assist but if you could guide me I'd be more than happy to furnish the details.
Thank you for your time in reading my post and assisting.
Model: RN524X|ReadyNAS 524X Premium performance Business Data Storage
Thank you for your time RolandWause. Long as in well over a day. Usually the NAS boots up in just under 5 mins but this was extremely long and repeat attempts to reboot did not help.
GM mdgm, thanks for your response. Is there a way I can tar/gzip or just extract the logs from shell and send you. My NetGear NAS web page is again stuck. I went to look at the front of the NAS and I see it's stuck again and in the process of rebooting again. I can SSH fine though and see the following mounted FS
/dev/md0 4.0G 2.7G 4.0K 100% is a bit unusual, saying that you are using 2.7GB out of 4.0GB, but the volume is 100% full. You could balancing the OS partition (using the btrfs command from ssh).
Though 2.7G used is also quite full - you must have installed some packages or apps that use OS partition space. You should probably shift some of those folders to the data volume (symlinks are one way to accomplish this). Of course that will need to wait until the data volume is mounted.
Thank you StephenB for your time. Yes, I agree a FS at 100% is just not right. I am trying to think if I did any package update before that may have taken effect following the reboot but can't recall (likely I am forgetting, after all something changed). Since I have another NAS on network and have connectivity I can temporarily scp files over if that might help and then attempt to do a reboot again to see if NAS can be brought up online
It's not really 100% full; there appear to be unused blocks that can be reclaimed to the free space pool. That is why I suggested the balance. Something like btrfs balance start / should work, though you can get it to run faster if you set some filters. https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balance
My previous response to this message did not get sent. I am sending the message again. please change the extension of the attached file to zip as the same is not permitted for uploading.
I guess I still can't upload getting an error
Correct the highlighted errors and try again.
The attachment's system_log-readynas64-20180720-114325.pdf content type (application/pdf) does not match its file extension and has been removed.
I attempted to send two messages but they failed. Essentially the command failed because there was no space on / file system. It did prompt me in the right direction and I started to look at the files updated recently using find -atime and I noticed this
root@ReadyNAS64:/data/AmazonCloudDrive# pwd
/data/AmazonCloudDrive
This is the app's backup from Cloud down to NAS. I manually deleted all files under the directory but files start copying from cloud down to NAS again. I need to stop the source. Could it be this?
root@ReadyNAS64:/data/AmazonCloudDrive# ps -aux | grep -i ama
I was able to stop the download by turning off the APP from Netgear Ready GUI. So at this time the data is not getting written to root FS. I want to confirm if I can start with btrfs repair flag. Pl. advise and thanks for your time
Somehow I must be techically challenged to be unable to post replies. I have posted my replies that "appear" to have been posted but aren't when page is refreshed. Happened more than 4 times Anyways StephenB I could not balance and continue to get the error no space available on root FS even though there is. I also rebooted the NAS
Maybe there is something in the log that's considered suspicious and deleted from teh post. Tried to attach it as an attachment that failed as well with the error as under:
Correct the highlighted errors and try again.
The file nas2.rtf does not have a valid extension for an attachment and has been removed. jpg,gif,png,pdf are the valid extensions.
mdgm, since the web portal is not accepting the log downloaded from NAS admin page and its getting truncated when I post the log for your review online, do you have a suggestion to send you the logs any other way? I have it; just can't upload to community page thread. Thanks for your time.
Next tried to mount the FS but I guess I was too enthusiastic
Yeah. I think the next step is to gradually increase the -dusage parameter until you reach 100. Doing this gradually keeps the number of blocks that are processed on each pass small.
You should have some free space now (9 blocks worth), but you still have ~460 blocks that could be processed, and that should create more free space.
You can see the free space using btrfs fi df, the usual df command won't show it.
root@ReadyNAS64:~# btrfs balance start -dusage=10 / Done, had to relocate 0 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=20 / Done, had to relocate 0 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=50 / Done, had to relocate 1 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=70 / Done, had to relocate 1 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=100 / ERROR: error during balancing '/': No space left on device There may be more info in syslog - try dmesg | tail root@ReadyNAS64:~# btrfs balance start -dusage=80 / Done, had to relocate 1 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=90 / Done, had to relocate 1 out of 9 chunks root@ReadyNAS64:~# btrfs balance start -dusage=100 / ERROR: error during balancing '/': No space left on device There may be more info in syslog - try dmesg | tail root@ReadyNAS64:~# btrfs balance start -dusage=95 / ERROR: error during balancing '/': No space left on device There may be more info in syslog - try dmesg | tail root@ReadyNAS64:~# btrfs balance start -dusage=90 / Done, had to relocate 1 out of 9 chunks
