× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Readynas 316 Volume is degraded (but all disks green)

armybill
Aspirant

Readynas 316 Volume is degraded (but all disks green)

So I have my system setup for X-Raid 5, Everythings been going good, but all of a sudden I have this volume degraded error. I have lost a disk before and it was pretty obvious that the disk had failed. But for this I am at a loss.

 

I am no programmer, but I did look at the logs, in the mdstat log it seems as though the system thinks one of the harddrives is removed, when its clearly not. Below I pasted the part Im refereincing.

/dev/md/server-0:
        Version : 1.2
  Creation Time : Sat Mar 24 08:34:42 2018
     Raid Level : raid5
     Array Size : 19510844160 (18606.99 GiB 19979.10 GB)
  Used Dev Size : 3902168832 (3721.40 GiB 3995.82 GB)
   Raid Devices : 6
  Total Devices : 5
    Persistence : Superblock is persistent

    Update Time : Thu Apr 19 15:18:46 2018
          State : clean, degraded
 Active Devices : 5
Working Devices : 5
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           Name : 43f65fee:server-0  (local to host 43f65fee)
           UUID : 36881ec7:fe01ca74:cf066349:4dccda40
         Events : 906

    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       -       0        0        1      removed
       2       8       35        2      active sync   /dev/sdc3
       3       8       51        3      active sync   /dev/sdd3
       4       8       67        4      active sync   /dev/sde3
       5       8       83        5      active sync   /dev/sdf3

Anyone have any idea about this? Any smarter people wanna look at my logs and set me straight??

Model: RN31661D|ReadyNAS 316 6-Bay
Message 1 of 10

Accepted Solutions
StephenB
Guru

Re: Readynas 316 Volume is degraded (but all disks green)


@armybill wrote:

Can we add this disk back into the volume and rsync it somehow?

 

 


You can try clicking on the drive, and then clicking on "format".  After the format, it should be automatically added an XRAID volume.  With FlexRaid, you'd need to add it to the volume manually.

View solution in original post

Message 8 of 10

All Replies
mdgm-ntgr
NETGEAR Employee Retired

Re: Readynas 316 Volume is degraded (but all disks green)

Do you see recent error count increases in smart_history.log?

 

Especially for the disk that's /dev/sdb (the disk may/may not show in disk_info.log)

Message 2 of 10
armybill
Aspirant

Re: Readynas 316 Volume is degraded (but all disks green)

In the smart history log I can actually see the old HD that failed and all the errors it had under uncorrectable errors and ATA errors

 

Under the HD that is shown as removed by the system, it has -1 for ioedc and 1 for cmd timeout, other than that its all 0s

 

In the Volume log all the errors are also at 0

Message 3 of 10
mdgm-ntgr
NETGEAR Employee Retired

Re: Readynas 316 Volume is degraded (but all disks green)

Check e.g. kernel.log or systemd-journal.log for messages about the disk that is marked as failed.

Message 4 of 10
armybill
Aspirant

Re: Readynas 316 Volume is degraded (but all disks green)

Sorry I should be clear, the failed disk I referenced in the previous message is a disk that failed had been replaced about a month ago.

 

As for the disk that is currently in the machine, but the mdstat log is showing as removed, there is no reference to it in the system journal and the kernal log is completely empty.

 

Its been running fine for about a month or so since I replaced the failed drive, than pow, this message comes up. The drive that failed was 4TB, I replaced it with a 12TB, and that is the drive that the system is showing as removed. 

Message 5 of 10
StephenB
Guru

Re: Readynas 316 Volume is degraded (but all disks green)


@armybill wrote:

... there is no reference to it in the system journal and the kernal log is completely empty. ...

I replaced it with a 12TB, and that is the drive that the system is showing as removed. 


 

If you hot-inserted the drive, you might try a reboot.  Occasionally the linux disk drivers will mark the SATA port as down when a disk fails, and in those cases the system won't see the new drive when it is hot-inserted.  Alternatively, remove the drive, reboot, and then hot-insert it again.

 

Note you won't get the full space benefit of the 12 TB unless you already have a 12 TB in the system (the XRAID capacity rule is "sum the drives and subtract the largest").

Message 6 of 10
armybill
Aspirant

Re: Readynas 316 Volume is degraded (but all disks green)

Ya I know I wont get to use the most of my 12TB. My thinking was (and my thinking could be right out of her), once one drive failed, the others were sure to follow. Ive had this NAS for a number of years and I went through several configurations before going with X-Raid 5, so I feel Ive been pretty hard on the drives and wasn't super surprised when one blew. Assuming the rest will blow soon, I plan on replacing all of them with 12TB as they go. 

 

I did not hot swap the drives, Im not sure if its because I learned to use computers in the 90s, but I am distrustful of all automated processes on computers lol, Even though it should be hot swapable, I never trust it. But in the interst of troubleshooting I did as you described and booted it with the 12TB out and got swapped it in once booted. 

 

Curiously the drive now shows as black rather than blue, indicating it is not part of the raid volume anymore. And of course the volume is still degraded. Can we add this disk back into the volume and rsync it somehow?

 

Thanks for all the help btw, this community is awesome for answering questions 

 

 

Message 7 of 10
StephenB
Guru

Re: Readynas 316 Volume is degraded (but all disks green)


@armybill wrote:

Can we add this disk back into the volume and rsync it somehow?

 

 


You can try clicking on the drive, and then clicking on "format".  After the format, it should be automatically added an XRAID volume.  With FlexRaid, you'd need to add it to the volume manually.

Message 8 of 10
armybill
Aspirant

Re: Readynas 316 Volume is degraded (but all disks green)

Stephen B, you are the man!

 

After I read your message and refreshed the page, it actually was showing as red, and gave me a message saying the drive was part of another volume and needed to be formatted, and once I did it started to resync. Crazy, I wonder how it got confused?

 

Anyways, thank you so much!

Message 9 of 10
StephenB
Guru

Re: Readynas 316 Volume is degraded (but all disks green)

I'm glad it's working again - let us know if there are any more issues after the resync.

Message 10 of 10
Top Contributors
Discussion stats
  • 9 replies
  • 1328 views
  • 0 kudos
  • 3 in conversation
Announcements