Need help with XRaid2 problem on ReadyNAS Pro running RAIDiator 4.2.31
It is of course my own fault for not having a backup but the following happened.
I had a 6 bay ReadyNAS pro as above running RAIDiator 4.2.31 with six 4TB drives. Whilst the logs showed an increasing number of ATA errors the system was at that point working. Due to the increasing number of errors I decided to kill two birds with one stone and replace all the drives with 6TB drives by swapping one at a time and allowing re-syncing to occur. I had successfully swapped five of the drives and was doing the final drive and re-syncing again when the fifth drive completely died whilst it was syncing the sixth. This meant effectively two bad drives at the same time.
This resulted in the system failing.
I still had the original six 4TB drives and these seem to still work but it is not mounting the volume although it does boot.
I have been able to boot in to Tech support mode and found that /dev/md0 which is the OS volume and is a RAID1 array was only listing one online member, /dev/md1 which is the swap volume and a RAID6 was only listing four members and /dev/md2 which is the data volume and RAID5 was listing five online members.
This seemed promising.
Note: I also did not have the original bay numbers for the 4TB drives noted so they likely have been inserted in a different order to originally.
I have been able to use mdadm --add /dev/md0 and for md1 as well to add the missing drive partitions and this seemed to work fine. These then seemed to assemble happily as well. md0 in particular shows details that look back to normal. For md1 the picture is slightly different. Whilst blkid shows for md0 an entry like
another identical fully working ReadyNAS shows this as
/dev/md1: TYPE="swap"
(This may mean nothing.)
My big problem is for the /dev/md2 array 😞
As far as mdadm --detail /dev/md2 is concerned it now indicates all six members are now online having used the add command to add the missing member and it synced the single missing member and for the other five members all had identical event numbers and it was happy with the superblock info.
Remember the order is now different so whereas my other box shows as
Number Major Minor RaidDevice State
0 830active sync /dev/sda3
1 8 191active sync /dev/sdb3
2 8 352active sync /dev/sdc3
3 8 513active sync /dev/sdd3
4 8 674active sync /dev/sde3
5 8 835active sync /dev/sdf3
The 'repaired' one shows as
Number Major Minor RaidDevice State
0 830active sync /dev/sda3
1 8 671active sync /dev/sde3
2 8 512active sync /dev/sdd3
3 8 353active sync /dev/sdc3
4 8 834active sync /dev/sdf3
5 8 195active sync /dev/sdb3
Even so, this so far looks good. The problem is a reboot results in the ReadyNAS saying that the volumes and shares cannot be found. 😞
I can also say that via ssh vgscan fails to find any volumes as below
vgscan -vv
Setting global/locking_type to 1
Setting global/locking_dir to /var/lock/lvm
File-based locking enabled.
Wiping cache of LVM-capable devices
/dev/core: stat failed: No such file or directory
Wiping internal VG cache
Reading all physical volumes.This may take a while...
Finding all volume groups
/dev/md2: size is 39022967040 sectors
/dev/md2: lvm2 label detected
It does not then continue with the usual additional lines of
It looks like the RAID(s) are now healthy but the LVM volume may be corrupted beyond use. I am hoping this is not the case and someone can suggest how to fix this.
I had a 6 bay ReadyNAS pro as above running RAIDiator 4.2.31 with six 4TB drives. Whilst the logs showed an increasing number of ATA errors the system was at that point working. Due to the increasing number of errors I decided to kill two birds with one stone and replace all the drives with 6TB drives by swapping one at a time and allowing re-syncing to occur. I had successfully swapped five of the drives and was doing the final drive and re-syncing again when the fifth drive completely died whilst it was syncing the sixth. This meant effectively two bad drives at the same time.
One complication is that 4.2.x systems can't be expanded over 16 TiB.
6x4TB RAID-5 would normally give you a 20 TB volume. If you created the volume by installing all 6 disks at once, I believe you'd get the full 20 TB size (since the all the disks are of equal size, so the volume doesn't actually expand). But the expansion to 6x6TB (30 TB volume) should have failed.
The usual mount procedure in tech support mode is
vgscan
vgchange -a y
mount /dev/c/c /c
Though it might be safer to do the mount read-only
mount -o ro /dev/c/c /c
So you could reboot into tech support mode and try that.
If you can figure out what 4 TB disk was in slot 6, you could also try assembling with the four 6TB drives and the original drive 6.
The other pathway is to use RAID recovery software in a Windows PC. You'd need an enclosure (~$200 USD). R-Studio is one option for recovery software ( https://www.r-studio.com/ )
I am at this stage far less concerned with storage expansion and may indeed look at v6 later for that.
Steve, as I indicated vgscan is not finding a volume i.e. c and hence it is not possible to do vgchange or mounting. Also (not mentioned) likely for the same reason there is no /dev/c/c/ device
You might have been confused by my referring to the fact I have a second identical but working unit and it does list the c volume and have a /dev/c/c/
I am at this stage far less concerned with storage expansion
Understood. I raised the expansion issue in the first post, because the failed expansion might complicate recovery efforts.
You might have better luck with the 4 larger disks + the original disk 6. The volume would be out of sync no matter what you do, so that it is part of the puzzle.
Though honestly I think your best option is recovery software.
