NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
ReadyNAS 104 - Volume: Volume data health changed from Degraded to Dead
I have a replacement disk on it's way because the disk in bay 4 has shown an increasing ATA error count. However, whilst the disk is on it's way Disk 2 went offline and, after a reboot of the ReadyN...
Hi Sandshark,
Thanks for the reply and, yes, I didn't put loads of detailed information mainly because I'm not really sure what information is needed.
From loads of Googling and (very carefully) looking around i came across a log file called '1' in the NAS root folder which contains 1 entry:
root@GDN-NAS:/# cat 1
ERROR: not a btrfs filesystem: /data
If I look in the Volumes page of the NAS it shows both a data volume as well as a data-0 volume so I (maybe naively) making the assumption that, because the NAS saw the original data volume as non-btrfs it tried to use the disks to create another one which it dynamically named data-0?
Would it be fair to assume that, because no data has actually been written to the new data-0 volume that the original data is intact, and therefore recoverable?
I do have access to a standalone external hard drive caddy and I natively use Linux Mint which seems positive (if there is anything positive about my situation) because Linux could be useful in trying to recover any data?
Is there any detail I can provide to help troubleshoot? I have ssh root access to the NAS.
Thanks
FWIW, the main do-it-yourself path is to attempt to clone either disk 2 or disk 4 (or both) with a tool that does sector-by-sector cloning. If this is successful, then you could attempt to boot the NAS in read-only mode with disks 1, 3, and the cloned disk(s) in place.
You can alternatively attempt to use a RAID recovery tool that supports BTRFS like ReclaiMe, with the intact and cloned disks.
One takeaway here is that RAID isn't enough to keep data safe. Hopefully you will get it back. Then I recommend putting a backup plan in place for your NAS.
Again, thanks StephenB
Initially I was gutted, but am actually starting to gain a little confidence that maybe there's a slim chance to recover the data.
I've powered off the NAS and powered the disk from bay 1 in a USB attached external drive caddy. Using a Linux PC, and without editing anything instead only looking at directory structures and running cat on various files I can see
[1] /var/backups/shares
contains a bunch of tar files each of which contains the /data/ share folder structure
[2] /var/backups/md
contains a bunch of tar.xz files each of which contains the disk maps and (what I think is) information on the individual disk layouts
What I can't understand is (and is probably where I need help) when I look in the /data/ directory it's completely empty but, when I look at the Admin Web Page there's 2 volumes.
data-0 shows as green, with a red dot saying Inactive or Dead. This volume says 5.44TB data, with 0 Free Space.
data shows as blue, with a red dot which when hovered over is completely blank. This volume says data 0 and Free Space 0
So, how do I tell which is which and where actually the data is supposed to be? I'm reluctant to try to even think about attempting recovery until I least know where things were/are supposed to be.
I can also see from the ssh console:
cat /etc/fstab shows: LABEL=0e34f9fc:data /data btrfs defaults,nodatasum 0 0
Whereas there is a text file '1' in the root which shows:
ERROR: not a btrfs filesystem: /data
MWilkinson wrote:
What I can't understand is (and is probably where I need help) when I look in the /data/ directory it's completely empty but, when I look at the Admin Web Page there's 2 volumes.
The /data you are seeing is a mount point - it's empty because the NAS was unable to mount the volume. The data-0 you are seeing on the volume tab is an artifact of the volume not being mountable. The NAS is just calling the unmountable volume data-0.
The NAS uses mdadm to create the RAID array(s) that are then assembled and mounted as /data by BTRFS.
If the disks in your volume are all the same size and your volume was never vertically expanded (smaller disks upgraded to larger ones), then you only have one RAID array - md127. Otherwise, you would have additional volumes (counting down from, so the next one would be md126).
BTRFS then would mount these arrays onto /data.
In your specific case, two disks are no longer in the array. So mdadm can no longer create md127 from the physical disks that remain. If you can successfully clone disk 2 or disk 4, then mdadm ought to be be able to create md127 (though it would be degraded w/o all 4 disks). That might require some override command options if the array is somewhat out of sync.
Then btrfs could mount the file system if there isn't too much corruption due to sectors that couldn't be copied during the cloning operation.
Thanks for the explanation, and that seems to fit with what I'm seeing.
If I connect disk 2 to a PC running Linux and open the Disks app I see md127 although it won't mount (mostly likely because it needs the other 2 disks in the volume).
StephenB wrote:If the disks in your volume are all the same size and your volume was never vertically expanded (smaller disks upgraded to larger ones), then you only have one RAID array - md127. Otherwise, you would have additional volumes (counting down from, so the next one would be md126).
All 4 disks are all Seagate 2TB 7200 RPM
In your specific case, two disks are no longer in the array. So mdadm can no longer create md127 from the physical disks that remain.
Disk 4 died completely, and I now have a spare replacement Seagate 2 TB disk. Disk 2 went offline and came back online, which is where I think any corruption would have occured. Looking at mdadm -D it shows the following:
Update Time : Sun Jun 5 21:33:46 2022
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0Number Major Minor RaidDevice State
3 8 49 0 active sync /dev/sdd1
4 8 33 1 active sync /dev/sdc1
5 8 17 2 active sync /dev/sdb1If you can successfully clone disk 2 or disk 4, then mdadm ought to be be able to create md127 (though it would be degraded w/o all 4 disks). That might require some override command options if the array is somewhat out of sync.
As disk 4 is dead, I could clone disk 2 but the newly cloned disk 2 would become the new disk 4? Attaching all 4 disks together could recreate a mountable md127 volume?
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
