This topic has been marked solved and closed to new posts due to inactivity. We hope you'll join the conversation by posting to an open topic or starting a new one.
I use Windows and have no experience using Linix commands, however if you can guide me with the force mount command i should be able to work it out. Really appreciate you assistance.
First, you do need to be careful, as mistakes in typing can cause more damage. Note that linux generally uses / for file paths, and not the \ that Windows uses.
If you never changed the admin password of the NAS (default is password), you will need to change it to something else first.
Then log in - you can enter "ssh root@nas-ip-address" in the windows command line if you are using windows 10 or windows 11. Use the real IP address of course. You will get a fingerprint query from Windows (don't recall the exact text), tell it to save the SSH fingerprint. This will open up a black text console window on the PC for command entry. Use the NAS admin password when prompted by the NAS.
Then enter these commands:
mdadm --assemble --really-force /dev/md127 /dev/sd[abcd]3
mount /dev/md127 /data
You might get an error from the first command - I'm specifying all 4 disks, since I am not sure what three device letters are currently assigned to the other three.
If this doesn't work, copy/paste the console output in your reply so I can see what's going on.
If this does remount the data volume, then you should immediately make a backup.
Please also run this command from ssh, and copy/paste the output in a reply:
cat /proc/mdstat
You should see something like this:
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md127 : active raid5 sda3[5] sdd3[7] sdc3[6] sdb3[4]
17567012352 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
md1 : active raid10 sda2[0] sdc2[3] sdd2[2] sdb2[1]
1044480 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
md0 : active raid1 sda1[5] sdd1[7] sdc1[6] sdb1[4]
4190208 blocks super 1.2 [4/4] [UUUU]
I'm interested in md0 and md1 - there was something not right with md0 in particular in your logs. It should be mirrored on all disks, but it seemed to be only installed on one. We should fix that also, but only after you have a backup of the data.
If the NAS doesn't do a clean shutdown, then cached writes are lost. That can result in a volume that is out of sync.
If that is all that happened, then it is possible to forcibly assemble the RAID array from ssh. If you like, you can send me the logs via a download link (dropbox, google drive, etc). There is some privacy leakage, so don't post them publicly. Instead send a private message (PM), using the envelope icon in the upper right of the forum page.
FYI, The best way to prevent this from happening again is to get a UPS that your NAS can monitor. It'd also be good to put a backup plan in place for the NAS. RAID isn't enough to keep it safe.
Drive 4 definitely has problems - 1281 ATA errors in the smart stats, and lots of errors in the log when the NAS tried to reset the SATA link.
There were also some problems with disk 3 on June 2nd - a run of errors like this:
Jun 02 13:25:21 ReadyNAS kernel: ata1.00: configured for UDMA/133
Jun 02 13:25:21 ReadyNAS kernel: sd 0:0:0:0: [sdb] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jun 02 13:25:21 ReadyNAS kernel: sd 0:0:0:0: [sdb] tag#1 Sense Key : Medium Error [current] [descriptor]
Jun 02 13:25:21 ReadyNAS kernel: sd 0:0:0:0: [sdb] tag#1 Add. Sense: Unrecovered read error - auto reallocate failed
Jun 02 13:25:21 ReadyNAS kernel: sd 0:0:0:0: [sdb] tag#1 CDB: Read(16) 88 00 00 00 00 00 00 8e 68 c0 00 00 0a 00 00 00
Jun 02 13:25:21 ReadyNAS kernel: blk_update_request: I/O error, dev sdb, sector 9335416
Jun 02 13:25:21 ReadyNAS kernel: ata1: EH complete
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: irq_stat 0x40000001
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: failed command: READ DMA EXT
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: cmd 25/00:00:c0:72:8e/00:0a:00:00:00/e0 tag 6 dma 1310720 in
res 51/40:00:30:73:8e/00:00:00:00:00/00 Emask 0x9 (media error)
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: status: { DRDY ERR }
Jun 02 13:25:29 ReadyNAS kernel: ata1.00: error: { UNC }
This might be just been a result of the power issue
I am seeing that you replaced one of the Barracudas with an Ironwolf some years back. I suggest future replacements should either be Ironwolf drives (or alternatively WD Red Plus). Lots of desktop class drives in this size range use SMR technology, which is not a good choice for ReadyNAS.
I am seeing one other thing that looks wrong in mdstat (which gives the RAID setup):
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid1 sdb2[0] sdd2[2] sdc2[1]
523264 blocks super 1.2 [3/3] [UUU]
md0 : active raid1 sdd1[2]
4190208 blocks super 1.2 [1/1] [U]
md0 is the NAS OS partition. It should be RAID1 and mirrored on all 4 drives (sda, sdb, sdc, and sdd). Not clear to me why that isn't the case for your NAS. If we can remount the data volume, then we should circle back to this.
Similarly, md1 is a swap partition that is normally on all drives. Yours is not including sda (the disk with the ATA errors). That isn't a big deal, but still odd.
As far as the data volume goes, it is definitely out-of-sync due to lost writes.
Jun 02 16:04:02 ReadyNAS start_raids[1080]: mdadm: NOT forcing event count in /dev/sdd3(3) from 14832 up to 14838
Jun 02 16:04:02 ReadyNAS start_raids[1080]: mdadm: You can use --really-force to do that (DANGEROUS)
You will need to use the --really-force option to mount the volume.
Is the SSH service enabled on the NAS? If not, try enabling it. It might fail (due to the volume problem) - but we can work around that if necessary.
Also, do you have any experience with the linux command line interface?
I have enabled the SSH on the NAS now, initially when i tried to enable it, popup says that it can not be enabled, did it about 3 times and it still contiunued with the popup error. i rebooted the NAS and logged back into the admin UI and SSH shows it is active, however disk #4 now seems to be dead as it is not showing up in the volume page. the slot is just black.
I use Windows and have no experience using Linix commands, however if you can guide me with the force mount command i should be able to work it out. Really appreciate you assistance.
I use Windows and have no experience using Linix commands, however if you can guide me with the force mount command i should be able to work it out. Really appreciate you assistance.
First, you do need to be careful, as mistakes in typing can cause more damage. Note that linux generally uses / for file paths, and not the \ that Windows uses.
If you never changed the admin password of the NAS (default is password), you will need to change it to something else first.
Then log in - you can enter "ssh root@nas-ip-address" in the windows command line if you are using windows 10 or windows 11. Use the real IP address of course. You will get a fingerprint query from Windows (don't recall the exact text), tell it to save the SSH fingerprint. This will open up a black text console window on the PC for command entry. Use the NAS admin password when prompted by the NAS.
Then enter these commands:
mdadm --assemble --really-force /dev/md127 /dev/sd[abcd]3
mount /dev/md127 /data
You might get an error from the first command - I'm specifying all 4 disks, since I am not sure what three device letters are currently assigned to the other three.
If this doesn't work, copy/paste the console output in your reply so I can see what's going on.
If this does remount the data volume, then you should immediately make a backup.
Please also run this command from ssh, and copy/paste the output in a reply:
cat /proc/mdstat
You should see something like this:
Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md127 : active raid5 sda3[5] sdd3[7] sdc3[6] sdb3[4]
17567012352 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
md1 : active raid10 sda2[0] sdc2[3] sdd2[2] sdb2[1]
1044480 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
md0 : active raid1 sda1[5] sdd1[7] sdc1[6] sdb1[4]
4190208 blocks super 1.2 [4/4] [UUUU]
I'm interested in md0 and md1 - there was something not right with md0 in particular in your logs. It should be mirrored on all disks, but it seemed to be only installed on one. We should fix that also, but only after you have a backup of the data.
