- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Readynas 316 Volume is degraded (but all disks green)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So I have my system setup for X-Raid 5, Everythings been going good, but all of a sudden I have this volume degraded error. I have lost a disk before and it was pretty obvious that the disk had failed. But for this I am at a loss.
I am no programmer, but I did look at the logs, in the mdstat log it seems as though the system thinks one of the harddrives is removed, when its clearly not. Below I pasted the part Im refereincing.
/dev/md/server-0: Version : 1.2 Creation Time : Sat Mar 24 08:34:42 2018 Raid Level : raid5 Array Size : 19510844160 (18606.99 GiB 19979.10 GB) Used Dev Size : 3902168832 (3721.40 GiB 3995.82 GB) Raid Devices : 6 Total Devices : 5 Persistence : Superblock is persistent Update Time : Thu Apr 19 15:18:46 2018 State : clean, degraded Active Devices : 5 Working Devices : 5 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K Name : 43f65fee:server-0 (local to host 43f65fee) UUID : 36881ec7:fe01ca74:cf066349:4dccda40 Events : 906 Number Major Minor RaidDevice State 0 8 3 0 active sync /dev/sda3 - 0 0 1 removed 2 8 35 2 active sync /dev/sdc3 3 8 51 3 active sync /dev/sdd3 4 8 67 4 active sync /dev/sde3 5 8 83 5 active sync /dev/sdf3
Anyone have any idea about this? Any smarter people wanna look at my logs and set me straight??
Solved! Go to Solution.
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@armybill wrote:
Can we add this disk back into the volume and rsync it somehow?
You can try clicking on the drive, and then clicking on "format". After the format, it should be automatically added an XRAID volume. With FlexRaid, you'd need to add it to the volume manually.
All Replies
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
Do you see recent error count increases in smart_history.log?
Especially for the disk that's /dev/sdb (the disk may/may not show in disk_info.log)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
In the smart history log I can actually see the old HD that failed and all the errors it had under uncorrectable errors and ATA errors
Under the HD that is shown as removed by the system, it has -1 for ioedc and 1 for cmd timeout, other than that its all 0s
In the Volume log all the errors are also at 0
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
Check e.g. kernel.log or systemd-journal.log for messages about the disk that is marked as failed.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
Sorry I should be clear, the failed disk I referenced in the previous message is a disk that failed had been replaced about a month ago.
As for the disk that is currently in the machine, but the mdstat log is showing as removed, there is no reference to it in the system journal and the kernal log is completely empty.
Its been running fine for about a month or so since I replaced the failed drive, than pow, this message comes up. The drive that failed was 4TB, I replaced it with a 12TB, and that is the drive that the system is showing as removed.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
@armybill wrote:
... there is no reference to it in the system journal and the kernal log is completely empty. ...
I replaced it with a 12TB, and that is the drive that the system is showing as removed.
If you hot-inserted the drive, you might try a reboot. Occasionally the linux disk drivers will mark the SATA port as down when a disk fails, and in those cases the system won't see the new drive when it is hot-inserted. Alternatively, remove the drive, reboot, and then hot-insert it again.
Note you won't get the full space benefit of the 12 TB unless you already have a 12 TB in the system (the XRAID capacity rule is "sum the drives and subtract the largest").
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
Ya I know I wont get to use the most of my 12TB. My thinking was (and my thinking could be right out of her), once one drive failed, the others were sure to follow. Ive had this NAS for a number of years and I went through several configurations before going with X-Raid 5, so I feel Ive been pretty hard on the drives and wasn't super surprised when one blew. Assuming the rest will blow soon, I plan on replacing all of them with 12TB as they go.
I did not hot swap the drives, Im not sure if its because I learned to use computers in the 90s, but I am distrustful of all automated processes on computers lol, Even though it should be hot swapable, I never trust it. But in the interst of troubleshooting I did as you described and booted it with the 12TB out and got swapped it in once booted.
Curiously the drive now shows as black rather than blue, indicating it is not part of the raid volume anymore. And of course the volume is still degraded. Can we add this disk back into the volume and rsync it somehow?
Thanks for all the help btw, this community is awesome for answering questions
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@armybill wrote:
Can we add this disk back into the volume and rsync it somehow?
You can try clicking on the drive, and then clicking on "format". After the format, it should be automatically added an XRAID volume. With FlexRaid, you'd need to add it to the volume manually.
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
Stephen B, you are the man!
After I read your message and refreshed the page, it actually was showing as red, and gave me a message saying the drive was part of another volume and needed to be formatted, and once I did it started to resync. Crazy, I wonder how it got confused?
Anyways, thank you so much!
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Re: Readynas 316 Volume is degraded (but all disks green)
I'm glad it's working again - let us know if there are any more issues after the resync.