NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
ReadyNAS 716X data- Two volumes after Resync
System: Readynas 716X F/W: 6.10.8 Old HDDs: 6TBx6 (WD Red) New HDD: 8TBx1 (WD Red) Raid 5 (X-Raid) I have recently experienced some issues with my Readynas system. I found out that I had to r...
RN716x wrote:
Is it asking for an input here?
root@MaQBaLiNAS:~# cat /etc/mdadm/mdadm.conf
CREATE owner=root group=disk mode=0660 auto=yes
This is the same as the file on my system, so nothing unusual here.
RN716x wrote:
root@MaQBaLiNAS:~# mdadm --assemble --scan --force
mdadm: NOT forcing event count in /dev/sdc3(3) from 11031 up to 19055
mdadm: You can use --really-force to do that (DANGEROUS)
mdadm: /dev/md/data-0 assembled from 4 drives - not enough to start the array.
mdadm: No arrays found in config file or automaticallyThis is telling you that /dev/sdc3 is missing a lot of writes - so it is seriously behind the other drives in the array.
- You could proceed with --really-force instead of force. You could end up with a lot of data loss/corruption if you do that.
- Another other option is go back to SDA. Power down, remove SDC, insert the SDA. Power up, and try the same mdadm command again, and see if the event count gap is smaller. If you go this route, you should definitely mount the volume as read-only - as we know that SDA is failing, and writes will likely only accelerate that. Also they would increase the event gap between sdc and the rest of the array (making going back to SDC even more risky).
This isn't an obvious decision. SDA likely will have a smaller event gap, but we already know it has unreadable sectors. Plus it might fail when you try to offload data. Still, I think it is likely the best path (with read-only mounting).
A variant of (2) is to clone SDA, and insert the clone. The cloning process will skip over any unreadable sectors on the original. The benefit of the variant is that there will be no bad sectors detected on the clone (which is a mixed blessing, as bad sectors do give you some information on file corruption). The risk is that the original will completely fail during the cloning process.
Either way (1, 2, or 2 variant), you'd copy off as much data as you can, do a factory default with the two new drives installed in place of SDA and SDC. You'd reconfigure the NAS at that point, and then restore the files from the backup.
So follwoing what Sandshark has said, I am now able to see that the SSH service is enabled by default. I left it there for now, hoping to get some tips on how to proceed further.
It's not enabled by default, but however it happened, it is enabled.
The log zip is for some reason missing most of the files. It looks like it is from 12/20.
The problem is that the resync of the new 8 TB disk failed due to disk errors on disk sda (in the first slot):
Dec 20 03:36:33 MaQBaLiNAS mdadm[3964]: Fail event detected on md device /dev/md127, component device /dev/sda3Dec 20 03:36:02 MaQBaLiNAS kernel: blk_update_request: I/O error, dev sda, sector 11233252792 Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815544 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815552 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815560 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815568 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815576 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815584 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815592 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815600 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815608 on sda3). Dec 20 03:36:02 MaQBaLiNAS kernel: md/raid:md127: read error not correctable (sector 11223815616 on sda3).There are several more of the uncorrectable errors after this (look in systemd-journal.log).
One option is to try cloning either disk 1 (sda) or the original sdd to a new drive. Not sure which is better. Then you could forcibly assemble the array, and ideally offload data before preceeding. There likely would be some data loss (since there are errors on sda, and sdd also clearly had issues).
You could also try RAID recovery software - though you'd need to be able to connect the drives to a Windows PC (getting a suitable USB enclosure), and software like ReclaiMe is also pricey. Plus you'd need storage to offload the data that is recovered.
- If the original sdd (6TB) is still working in the pre-sync state. i.e. I removed it and replaced it with the new 8TB. Would I still need to clone it to a new drive or can I just connect it and try to assemble?
What difference would cloning make then?
For the sda (6TB), the plan was to replace it when the sdd is resynced successfully with a new 8TB that I've also got ready that I might use for cloning either now.
I'm trying to weigh my options here, would re-connecting the original sdd (6TB) help here while maybe isolating sda just to take a backup first? It should be in the pre-sync state.
Would you elaborate further on this part please, just to make sure I'm following correctly:
"Then you could forcibly assemble the array,"
RN716x wrote:
If the original sdd (6TB) is still working in the pre-sync state. i.e. I removed it and replaced it with the new 8TB. Would I still need to clone it to a new drive or can I just connect it and try to assemble?The two paths are to start with the original sdd, or start with the original sda. Which is best depends on the amount of bad sectors on the two drives.
Cloning just eliminates the chance that the drive will fail during resync. SDA already did, so we know that one would be challenging. If the drive is cloned with errors, then there will probably be corrupt files/folders, but the RAID won't be able to detect that (since the sectors would all be readable).
SDD isn't so clear - you had issues with it before. But you could try putting the drive back into the NAS (with SDA removed), force the array to assemble, and then try to sync with a new 8 TB drive in slot 1. Assuming it does assemble, you could try making a backup first.
RN716x wrote:
Would you elaborate further on this part please, just to make sure I'm following correctly:
"Then you could forcibly assemble the array,"Basically there are write counters on each drive in the RAID array. Those are used by the RAID software to make sure all writes to all drives completed. When you reinsert a drive, in almost all cases the write counters will not be the same (even if you don't think you wrote anything to the array). If the counts aren't very close, the array won't mount - that is one cause of the "inactive volume" problem.
In your case, the count for sda3 is already off, and it was kicked out of the array. Likely that is partly due to the burst of errors at the end of the sync. Since it is off, the original sdd3 is almost certainly also off.
Overall, the process would be to power down the NAS, remove sda, and put back the original sdd. Then power up (perhaps read-only for safety). If that works, then you should back up the data if you can, and hot-insert one of the 8 TB drives.
If it's the one that you already tried, then remove the partitions with Windows Disk Manager before reinserting (or alternatively select it from the NAS web ui, and manually format it).
If that resync completes, then remove sdd, and insert the new 8 TB drive in it's place (NAS running).
However, I think the inactive volume error will persist when you power up the NAS w/o sda and with the original sdd. In that case, you'd need to use some linux commands to force the array to mount. Normally I've done this in tech support mode (not in the normal boot). But the commands should work in a normal boot as well.
Something like
mdadm --force --assemble /dev/md127 btrfs scan device mount /dev/md127 /dataIf you prefer mounting the array read-only then put a "-o ro" after "mount" in the last command.
If you try this, make sure you log in as root (using the NAS admin password) when accessing the NAS with ssh.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
