Re: ReadyNAS RN 104 volume fails (Firmware 6.10.8)

ftr_nas · ‎2022-12-22

My ReadyNAS RN 104 doesn't work anymore.

First it couldn't start. When booting, it stopped in the middle with booting and a percentage (Every booting try with a other number)

After some tries, i done a disk test from the boot menu.

Now I can start, but I get Problems with inactive Volumes.

Can some one help me?

StephenB · ‎2022-12-22

Do you have a backup of your files?

It would be helpful to download the log zip file from the logs page - it might give more details on what went wrong.

ftr_nas · ‎2023-01-05

mdstat.log

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md125 : active raid0 sdd3[0] sdc3[1]
7804599808 blocks super 1.2 64k chunks

md126 : active raid0 sdb3[0]
3902168832 blocks super 1.2 64k chunks

md127 : active raid0 sda3[0]
3902168832 blocks super 1.2 64k chunks

md0 : active raid1 sdb1[4] sda1[5] sdd1[1]
4190208 blocks super 1.2 [4/3] [UUU_]

unused devices: <none>
/dev/md/0:
Version : 1.2
Creation Time : Tue Dec 29 01:47:36 2015
Raid Level : raid1
Array Size : 4190208 (4.00 GiB 4.29 GB)
Used Dev Size : 4190208 (4.00 GiB 4.29 GB)
Raid Devices : 4
Total Devices : 3
Persistence : Superblock is persistent

Update Time : Tue Dec 27 17:57:24 2022
State : clean, degraded
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

Consistency Policy : unknown

Name : 0e3707ca:0 (local to host 0e3707ca)
UUID : d3387fb8:c59ed4ac:2a5d06dd:e173dd82
Events : 54351

Number Major Minor RaidDevice State
4 8 17 0 active sync /dev/sdb1
1 8 49 1 active sync /dev/sdd1
5 8 1 2 active sync /dev/sda1
- 0 0 3 removed
/dev/md/DVD-0:
Version : 1.2
Creation Time : Mon Dec 28 20:38:15 2015
Raid Level : raid0
Array Size : 7804599808 (7443.05 GiB 7991.91 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent

Update Time : Mon Dec 28 20:38:15 2015
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Chunk Size : 64K

Consistency Policy : unknown

Name : 0e3707ca:DVD-0 (local to host 0e3707ca)
UUID : bdd9dd6e:460b18e1:76bfafd8:e827ca5a
Events : 0

Number Major Minor RaidDevice State
0 8 51 0 active sync /dev/sdd3
1 8 35 1 active sync /dev/sdc3
/dev/md/DVD-1:
Version : 1.2
Creation Time : Sat Dec 24 12:41:16 2016
Raid Level : raid0
Array Size : 3902168832 (3721.40 GiB 3995.82 GB)
Raid Devices : 1
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Sat Dec 24 12:41:16 2016
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Chunk Size : 64K

Consistency Policy : unknown

Name : 0e3707ca:DVD-1 (local to host 0e3707ca)
UUID : fb057a48:933d8bff:b3bb6d40:b45c8091
Events : 0

Number Major Minor RaidDevice State
0 8 19 0 active sync /dev/sdb3
/dev/md/DVD-2:
Version : 1.2
Creation Time : Sat Dec 24 12:51:33 2016
Raid Level : raid0
Array Size : 3902168832 (3721.40 GiB 3995.82 GB)
Raid Devices : 1
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Sat Dec 24 12:51:33 2016
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Chunk Size : 64K

Consistency Policy : unknown

Name : 0e3707ca:DVD-2 (local to host 0e3707ca)
UUID : b4de1ab1:b55a9bfb:5d6852a7:c46237a7
Events : 0

Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3

ftr_nas · ‎2023-01-05

How can I post longer text here?

I tried several times with 6, 4, 2 and 1 log-files.

StephenB · ‎2023-01-05

@ftr_nas wrote:

How can I post longer text here?

I tried several times with 6, 4, 2 and 1 log-files.

It's not the length - they are getting caught by the automatic spam filter.

I released the one from earlier today - wasn't sure if releasing the older ones made sense or not.

StephenB · ‎2023-01-05

@ftr_nas wrote:

mdstat.log

Personalities : [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md125 : active raid0 sdd3[0] sdc3[1]
7804599808 blocks super 1.2 64k chunks

md126 : active raid0 sdb3[0]
3902168832 blocks super 1.2 64k chunks

md127 : active raid0 sda3[0]
3902168832 blocks super 1.2 64k chunks

This suggests you had three volumes - two JBOD (sda and sdb), and one RAID-0 (spanning sdc and sdd).

Looks like DVD-0 is the RAID-0 volume, with the two jbod volumes being DVD-1 and DVD-2.

Is this correct? (not sure it is)?

@ftr_nas wrote:

md0 : active raid1 sdb1[4] sda1[5] sdd1[1]
4190208 blocks super 1.2 [4/3] [UUU_]

This suggests something is wrong with sdd. Though it does still seem to be in DVD-0.

Are you seeing any disk errors in system.log, kernel.log, or systemd-journal.log? Or btrfs errors?

ftr_nas · ‎2023-01-09

No, I had only one volume with the name dvd.

First a RAID0 with two disks and later with four disks.

system.log errors (several times):

Dec 21 14:57:37 DVDs wsdd2[2199]: error: wsdd-mcast-v4: wsd_send_soap_msg: send
Dec 21 14:57:37 DVDs wsdd2[2199]: error: wsdd-mcast-v6: wsd_send_soap_msg: send

kernel.log errors (several times):
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#3 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#3 CDB: Read(16) 88 00 00 00 00 00 00 8f ff c0 00 00 00 08 00 00
Dec 26 20:01:27 DVDs kernel: blk_update_request: I/O error, dev sdc, sector 9437120
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#4 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#4 CDB: Read(16) 88 00 00 00 00 00 00 8f ff c0 00 00 00 08 00 00
Dec 26 20:01:27 DVDs kernel: blk_update_request: I/O error, dev sdc, sector 9437120
Dec 26 20:01:27 DVDs kernel: Buffer I/O error on dev sdc2, logical block 131056, async page read
Dec 26 20:01:27 DVDs kernel: Buffer I/O error on dev md125, logical block 1951149936, async page read
Dec 26 20:01:27 DVDs kernel: Buffer I/O error on dev md125, logical block 1951149936, async page read
Dec 26 20:01:27 DVDs kernel: Buffer I/O error on dev md125, logical block 1951149936, async page read

In the systemd-journal.log the smae errors like errors in kernel.log.

I can't find errors in btrfs.log

StephenB · ‎2023-01-09

@ftr_nas wrote:

No, I had only one volume with the name dvd.

First a RAID0 with two disks and later with four disks.

Did you concatenate the disks 3 and 4 with the first two?

@ftr_nas wrote:

kernel.log errors (several times):
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#3 FAILED Result: hostbyte=DID_BAD_TARGET
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#3 CDB: Read(16) 88 00 00 00 00 00 00 8f ff c0 00 00 00 08 00 00
Dec 26 20:01:27 DVDs kernel: blk_update_request: I/O error, dev sdc, sector 9437120
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#4 FAILED Result: hostbyte=DID_BAD_TARGET
Dec 26 20:01:27 DVDs kernel: sd 2:0:0:0: [sdc] tag#4 CDB: Read(16) 88 00 00 00 00 00 00 8f ff c0 00 00 00 08 00 00
Dec 26 20:01:27 DVDs kernel: Buffer I/O error on dev sdc2, logical block 131056, async page read

So all of these are sdc.

Do you have a backup of the files?

Sandshark · ‎2023-01-10

Having chosen JBOD, you chose to have no redundancy. Worse yet, you chose a format where the loss of one drive results in total loss of the entire volume. That choice appears to have now bitten you. With a different RAID scheme, you may still have had an issue, but it's less likely, or it could have only been a partial loss. It may be recoverable, or at least not be a total loss, but it's going to take some sorting out.

It looks to me like you have this:

You started with sdc and sdd, which were made into an MDADM RAID0 volume named DVD-0 and a BTRFS volume named DVD. You then added either sda or sdb, which was made into an MDADM "single drive RAID0" (an odd way to do it, but the ReadyNAS does stuff like that for constancy and to aid expandability), named DVD-1, and then concatenated with BTRFS volume DVD. Lastly, you added the last drive and the same sort of thing happened with an MDADM name of DVD-2 and also concatenated with BTRFS volume DVD.

All of the MDADM volumes seem to be OK, though it's a bit confusing because the partition on sdc that should be a part of the OS partition (md0) is missing. So maybe sdc really has some issues that MDADM isn't recognizing but BTRFS is. The BTRFS volume is definitely broken.

I'm not the best one to help with recovery steps, but I hope that my deciphering of the construct of your volume will help @StephenB and/or the mods help you there.