Need help with XRaid2 problem on ReadyNAS Pro running RAIDiator 4.2.31

Question

It is of course my own fault for not having a backup but the following happened.&nbsp;I had a 6 bay ReadyNAS pro as above running RAIDiator 4.2.31 with six 4TB drives. Whilst the logs showed an increasing number of ATA errors the system was at that point working. Due to the increasing number of errors I decided to kill two birds with one stone and replace all the drives with 6TB drives by swapping one at a time and allowing re-syncing to occur. I had successfully swapped five of the drives and was doing the final drive and re-syncing again when the fifth drive completely died whilst it was syncing the sixth. This meant effectively two bad drives at the same time.&nbsp;This resulted in the system failing.&nbsp;I still had the original six 4TB drives and these seem to still work but it is not mounting the volume although it does boot.&nbsp;I have been able to boot in to Tech support mode and found that /dev/md0 which is the OS volume and is a RAID1 array was only listing one online member, /dev/md1 which is the swap volume and a RAID6&nbsp; was only listing four members and /dev/md2 which is the data volume and RAID5 was listing five online members.&nbsp;This seemed promising.&nbsp;Note: I also did not have the original bay numbers for the 4TB drives noted so they likely have been inserted in a different order to originally.&nbsp;I have been able to use mdadm --add /dev/md0 and for md1 as well to add the missing drive partitions and this seemed to work fine. These then seemed to assemble happily as well. md0 in particular shows details that look back to normal. For md1 the picture is slightly different. Whilst blkid shows for md0 an entry like&nbsp;&nbsp;/dev/md0: UUID="c123ab45-xxx-xxxx-806d-c12343c4667b" TYPE="ext3"&nbsp;which is the same format as another working ReadyNAS It is now showing the following for md1&nbsp;/dev/md1: UUID="d123456-xxxx-xxxx-ad50-ec123448612" TYPE="swap"&nbsp;another identical fully working ReadyNAS shows this as&nbsp;/dev/md1: TYPE="swap"&nbsp;(This may mean nothing.)&nbsp;My big problem is for the /dev/md2 array 😞&nbsp;As far as mdadm --detail /dev/md2 is concerned it now indicates all six members are now online having used the add command to add the missing member and it synced the single missing member and for the other five members all had identical event numbers and it was happy with the superblock info.&nbsp;Remember the order is now different so whereas my other box shows as&nbsp;&nbsp;&nbsp; &nbsp; Number &nbsp; Major &nbsp; Minor &nbsp; RaidDevice State&nbsp;&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; &nbsp; 8&nbsp; &nbsp; &nbsp; &nbsp; 3&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sda3&nbsp;&nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 19&nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdb3&nbsp;&nbsp; &nbsp; &nbsp; 2 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 35&nbsp; &nbsp; &nbsp; &nbsp; 2&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdc3&nbsp;&nbsp; &nbsp; &nbsp; 3 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 51&nbsp; &nbsp; &nbsp; &nbsp; 3&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdd3&nbsp;&nbsp; &nbsp; &nbsp; 4 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 67&nbsp; &nbsp; &nbsp; &nbsp; 4&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sde3&nbsp;&nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 83&nbsp; &nbsp; &nbsp; &nbsp; 5&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdf3The 'repaired' one shows as&nbsp;&nbsp; &nbsp; Number &nbsp; Major &nbsp; Minor &nbsp; RaidDevice State&nbsp;&nbsp; &nbsp; &nbsp; 0 &nbsp; &nbsp; &nbsp; 8&nbsp; &nbsp; &nbsp; &nbsp; 3&nbsp; &nbsp; &nbsp; &nbsp; 0&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sda3&nbsp;&nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 67&nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sde3&nbsp;&nbsp; &nbsp; &nbsp; 2 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 51&nbsp; &nbsp; &nbsp; &nbsp; 2&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdd3&nbsp;&nbsp; &nbsp; &nbsp; 3 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 35&nbsp; &nbsp; &nbsp; &nbsp; 3&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdc3&nbsp;&nbsp; &nbsp; &nbsp; 4 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 83&nbsp; &nbsp; &nbsp; &nbsp; 4&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdf3&nbsp;&nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; 8 &nbsp; &nbsp; &nbsp; 19&nbsp; &nbsp; &nbsp; &nbsp; 5&nbsp; &nbsp; &nbsp; active sync &nbsp; /dev/sdb3&nbsp;Even so, this so far looks good. The problem is a reboot results in the ReadyNAS saying that the volumes and shares cannot be found. 😞&nbsp;I can also say that via ssh vgscan fails to find any volumes as below&nbsp;vgscan -vv&nbsp; &nbsp; &nbsp; Setting global/locking_type to 1&nbsp; &nbsp; &nbsp; Setting global/locking_dir to /var/lock/lvm&nbsp; &nbsp; &nbsp; File-based locking enabled.&nbsp; &nbsp; Wiping cache of LVM-capable devices&nbsp; &nbsp; &nbsp; /dev/core: stat failed: No such file or directory&nbsp; &nbsp; Wiping internal VG cache&nbsp; Reading all physical volumes.&nbsp; This may take a while...&nbsp; &nbsp; Finding all volume groups&nbsp; &nbsp; &nbsp; /dev/md2: size is 39022967040 sectors&nbsp; &nbsp; &nbsp; /dev/md2: lvm2 label detected&nbsp;It does not then continue with the usual additional lines of&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; Locking /var/lock/lvm/V_c RB&nbsp; &nbsp; Finding volume group "c"&nbsp; &nbsp; &nbsp; /dev/md2: lvm2 label detected&nbsp; &nbsp; &nbsp; /dev/md2: lvm2 label detected&nbsp; Found volume group "c" using metadata type lvm2&nbsp; &nbsp; &nbsp; Unlocking /var/lock/lvm/V_c&nbsp;lvscan -vv gives a similar picture. &nbsp;For blkid I get the expected line of&nbsp;&nbsp;/dev/md2: UUID="wb1234-xxxx-xxxx-7xXr-6mXc-vQEH-q022yt" TYPE="lvm2pv" &nbsp;but not the next line of&nbsp;&nbsp;/dev/mapper/c-c: UUID="01abcdef12-xxxx-xxxx-bbac-1234567ed1b8" TYPE="ext4" &nbsp;It looks like the RAID(s) are now healthy but the LVM volume may be corrupted beyond use. I am hoping this is not the case and someone can suggest how to fix this.&nbsp;

StephenB · Answer

jelockwood1&nbsp;wrote:
&nbsp;
I had a 6 bay ReadyNAS pro as above running RAIDiator 4.2.31 with six 4TB drives. Whilst the logs showed an increasing number of ATA errors the system was at that point working. Due to the increasing number of errors I decided to kill two birds with one stone and replace all the drives with 6TB drives by swapping one at a time and allowing re-syncing to occur. I had successfully swapped five of the drives and was doing the final drive and re-syncing again when the fifth drive completely died whilst it was syncing the sixth. This meant effectively two bad drives at the same time.
&nbsp;

One complication is that 4.2.x systems can't be expanded over 16 TiB.&nbsp;
&nbsp;
6x4TB RAID-5 would normally give you a 20 TB volume.&nbsp; If you created the volume by installing all 6 disks at once, I believe you'd get the full 20 TB size (since the all the disks are of equal size, so the volume doesn't actually expand).&nbsp; But the expansion to 6x6TB (30 TB volume) should have failed.&nbsp; &nbsp;
&nbsp;
The usual mount procedure in tech support mode is
vgscan
vgchange -a y
mount /dev/c/c /c
Though it might be safer to do the mount read-only
mount -o ro /dev/c/c /c
So you could reboot into tech support mode and try that.
&nbsp;
If you can figure out what 4 TB disk was in slot 6, you could also try assembling with the four 6TB drives and the original drive 6.
&nbsp;
The other pathway is to use RAID recovery software in a Windows PC.&nbsp; You'd need an enclosure (~$200 USD).&nbsp; R-Studio is one option for recovery software (&nbsp;https://www.r-studio.com/&nbsp;)&nbsp;

StephenB · Answer

FYI, after you recover the data, you should consider converting your NAS to run OS-6.&nbsp; That doesn't have the expansion limits that 4.2.x firmware has.

jelockwood1 · Answer

StephenB&nbsp;&nbsp;I am at this stage far less concerned with storage expansion and may indeed look at v6 later for that.&nbsp;Steve, as I indicated vgscan is not finding a volume i.e. c and hence it is not possible to do vgchange or mounting. Also (not mentioned) likely for the same reason there is no /dev/c/c/ device&nbsp;You might have been confused by my referring to the fact I have a second identical but working unit and it does list the c volume and have a /dev/c/c/

StephenB · Answer

jelockwood1&nbsp;wrote:
I am at this stage far less concerned with storage expansion

Understood.&nbsp; I raised the expansion issue in the first post, because the failed expansion might complicate recovery efforts.
&nbsp;
You might have better luck with the 4 larger disks + the original disk 6.&nbsp; The volume would be out of sync no matter what you do, so that it is part of the puzzle.&nbsp;&nbsp;
&nbsp;
Though honestly I think your best option is recovery software.

Forum Discussion

Need help with XRaid2 problem on ReadyNAS Pro running RAIDiator 4.2.31

4 Replies

Related Content

ReadyNas RND2000 v2 & ERR_SSL_VERSION_OR_CIPHER_MISMATCH & RAIDiator 4.1.12 update quest

ReadyNAS RAIDiator 4.1 Anleitung auf deutsch

ReadyNas Duo upgrade RAIDiator 4.1.16

Backup from ReadyNAS OS6 to ReadyNAS Raidiator 4

RN314 problems after Readynas 6.10.7

NETGEAR Academy

ProSupport for Business