× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

AndyWheatcroft
Aspirant

ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

ReadyNAS Pro 6 - Running RAIDiator 4.2.31 - X-RAID2 

5 x 3TB - Seagate ST33000650NS - Dual Redundancy

Disks installed in bays 1 thru 5

Whilst running my monthly data scrubbing, disk in Bay 1 died and data scrubbing stopped.  Front panel showed "X" for bay1.

I replaced with another ST33000650NS which had passed Seagate drive tests but RNP did not acknowledge and showed drive as "Dead".

I moved the new drive to Bay 6 and system recognized and started a re-sync but still showed a drive in Bay 1 as dead. even though the bay is now empty.

After the re-sync had completed, I received the following:

"RAID sync finished on volume C.  The array is still in degraded mode".

I have tried re-installing 4.2.31.

I have tried turning RNP off and leaving unplugged for 2 hours.

Now, front panel shows no disk in Bay 1 but "Health" tab still says "Seagate ST33000650NS 2794 GB" - "Dead".

I doubt that I am really in degraded mode and I have dual back-ups on 2 of the three USB ports. (UPS on third USB.)

All 5 drives have ZERO Rellocated Sector Counts and ZERO ATA Error Counts.

So, I am not concerned about losing data but I would like to clear the error.

Why does it think there is still a drive in the empty bay?  I thought the total power down would cause the BIOS to lose it's memory.

Model: RNDP6000|ReadyNAS Pro 6 Chassis only
Message 1 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty


@AndyWheatcroft wrote:

I doubt that I am really in degraded mode


Not sure myself.

 

Can you download the log zip file, and post mdstat.log and mdconfig.log?

Message 2 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Had to covert files to PDF.  Looking at this file, it is obvious that the RNP thinks I only have 4 working disks yet Frontview shows different. FrontView_Health.jpegmdstat.jpg

Message 3 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

You can just copy/paste text snippets from the logs.

 

mdstat shows the system is somewhat confused.

  • md1 and md4 are using all disks (including the missing/dead one)
  • md0 and md3 are using the missing/dead disk, but not the new one
  • Only md2 is correct.

This could probably fixed with ssh (or by paid support).  Another approach is just do a factory reset, and restore the NAS from backup.

Message 4 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Hi Stephen,

I am happy to try fixing with SSH.  If that fails I can always do a factory reset but it will take a long time to resore everything due to only having USB2 speeds.

Can you tell me what I should modify?

File "/proc/mdstat" is a zero length file but when I type "cat mdstat" it shows what is in the log file. And if I try "vi mdstat", i get a new file.

Thanks,

Andy.

Message 5 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty


@AndyWheatcroft wrote:

it will take a long time to resore everything due to only having USB2 speeds.

 


It's quite a bit faster to connect the USB drive to a PC and transfer over the network.

 


@AndyWheatcroft wrote:

Can you tell me what I should modify?

 


You'd need to run mdadm to add drive 5 to the RAID groups that are missing it, and then remove drive 1 one from the RAID groups that still have it..  I'm not certain of exactly what the commands would be.  @Sandshark probably has a better idea, since he has recently been playing with mdadm on OS-6.

Message 6 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

You seem to have too many mdadm arrays, and some have 5 drives and others only 4.  But maybe they were created because of incremental upgrades.  I don't really know much about how the 4.2.x version of XRAID did its magic, as I knew a lot less Linux when I still had 4.x based systems (I upgraded my Pros to OS6 early on).   AFAIK, the EXT file system can't span volumes, so something like LVM (Logical Volume Manager) has to be in the middle.  I have no experience with that.

 

I suspect those arrays with only 4 elements are what's causing it to say you are degraded, and I think you very well may still be.  And it may also be that those arrays never having re-sync'ed with the new drive 6 and still showing drive 1 as the "missing" one is what's causing the phantom "dead" drive.

 

You can gain some additional insight using:

 

fdisk -l

mdadm  --detail  /dev/md0

mdadm  --detail  /dev/md1

mdadm  --detail  /dev/md2

mdadm  --detail  /dev/md3

mdadm  --detail  /dev/md4

 

But it's good your backup is in order.  A factory default may be the only solution.  Consider moving to OS6 if you do need to default.

Message 7 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Thank you to Sandshark and StephenB for your suggestions.

 

I agree that if I am to do a factory reset, I might as well go to OS6.

As to why there are so many mdadm arrays, I suspect it is because I started 10 years ago with three 500MB disks, upgraded in small steps to 4, then 5 x 1TB disks and last year upgraded to 5 x 3TB drives.

One hope I have with OS6 is to stop high temp warnings.  The Seagate ST33000650NS drives run hot - their operational range is up to 60C but the RNP freaks out when they get above 55C.  Would OS6 help here? (Don't worry if you don't know the answer.)

Thanks,

Andy.

Message 8 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

OS6 has fan options of "quiet", "balanced", or "cool".  I think running it with the "cool"" settings should keep you from getting that hot unless the NAS is in a very warm spot.   I recently did some experiments requiring resyncs using some 1TB NS grade drives in an Ultra4 with OS6 and they never got that hot.  My NAS is in a room at 75°F (24°C).

Message 9 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

I recommend you use the commands listed above to verify the configuration first.  Based on the assumption that the phantom in slot one has no assigned sd number, I believe the correct commands to add in the missing partitions are:

mdadm  /dev/md1  --add /dev/sde2

mdadm  /dev/md2  --add /dev/sde5

mdadm  /dev/md3  --add /dev/sde6

 

Of course, if the partitions are missing, you'll have to create them and re-boot to have them recognized.

 

I have no idea how to get rid of the phantom drive, since it does not appear to have an sd device assignment (your 5 active ones are sda through sde).  Maybe put a drive in that slot that's too small for expansion.

 

Of couse, maybe it's the slot and not the drive that went bad.

Message 10 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Yes, I am concerned that Slot 1 got fried when the disk failed.  I know the disk is very dead - I tested using SeaTools on a Win10 PC and it wouldn't even spin up.

"fdisk -l" shows that devices /dev/sda through /dev/sde look good.

There is also a /dev/sdf which I am pretty sure is an external 2TB USB drive. (used for limited backups)

There is also a /dev/sdg which I am pretty sure is an external 8TB USB drive. (used for complete backups)

And here is the rest of fdisk that I cannot explain:

Disk /dev/md0: 4294 MB, 4294955008 bytes

2 heads, 4 sectors/track, 1048573 cylinders

Units = cylinders of 8 * 512 = 4096 bytes

 

Disk /dev/md0 doesn't contain a valid partition table

fdisk: device has more than 2^32 sectors, can't use all of them

 

Disk /dev/dm-0: 2199.0 GB, 2199023255040 bytes

255 heads, 63 sectors/track, 267349 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/dm-0 doesn't contain a valid partition table

 

As for being redundant, I am now concerned after looking at the my "Raid Settings" in FrontView. (See screenshot)

I would be happy to replace the MB if Slot 1 is dead but I don't really need all 6 slots.  And I suspect it will be quite hard to find a replacement MB anyway.  But if Slot 1 is dead, upgrading to OS6 may not resolve my phantom issue.

 

 

Message 11 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Hi Sandshark,

 

Sorry - I got sidetracked on a more serious production issue.

I tried a couple of the commands that you suggested but got nowhere.

mdadm --detail /dev/md2

shows:

  Raid Devices : 5

  Total Devices : 4

    Persistence : Superblock is persistent

 

    Update Time : Tue Apr 30 14:53:12 2019

          State : clean, degraded

Active Devices : 4

Working Devices : 4

Failed Devices : 0

  Spare Devices : 0

 

    Number   Major   Minor   RaidDevice State

       0       0        0        0      removed

       7       8        5        1      active sync   /dev/sda5

       6       8       21        2      active sync   /dev/sdb5

       9       8       37        3      active sync   /dev/sdc5

       8       8       53        4      active sync   /dev/sdd5

When I tried:

mdadm /dev/md2 --add /dev/sde5

I get the message:

mdadm: /dev/sde5 reports being an active member for /dev/md2, but a --re-add fails.

mdadm: not performing --add as that would convert /dev/sde5 in to a spare.

mdadm: To make this a spare, use "mdadm --zero-superblock /dev/sde5" first.

 

Time to abandon ship and re-install the OS?  Go to OS6?

 

Thanks,

Andy.

Message 12 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Zeroing the superblock removes the "markers" that it belongs to that RAID.  So, you should then be able to re-add it to the array or make it a spare, after which it shoud add itself in place of the missing one.  You may have to re-boot after zeroing and/or after making it a spare.  There is likely a better way than re-booting, but I've not discovered it.

 

Some things I read regarding MDADM indicated a drive should be made a spare instead of directly adding it.  That didn't seem to be necessary in OS6, but might be in older Debian distros.

Message 13 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Well, I think I have "bricked" my RNP!

I decided to bite the bullet and install OS6 but it failed with some error message and stopped at 92%.

Now, nothing I do will get the RNP to boot.  I can get to the BOOT menu, but that is about it.

One concern I had before starting the upgrade was memory - I only have 1GB installed.  I tried to find out if OS6 had a minimum but I couldn't find an answer.

I did read that one user was able to recover from BOOT failure by wiping the root volume and putting a fresh copy of the OS on it.

 

Can anyone tell me how to do this?

Message 14 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Can you try

  • zeroing (or unformatting) a disk
  • put just that disk in the NAS
  • Power it up again

See if that completes the install.  If it does you can then add the other disks.

 


@AndyWheatcroft wrote:

I tried to find out if OS6 had a minimum but I couldn't find an answer.

 


You can only go by the existing platforms.  The RN100 has 512 MB.  The other platforms have two GB or more.

 

It's pretty clear that the RN100 doesn't really have enough to run all the services well.  Many folks are fine with 2 GB (RN200, RN300, some RN400 models).

 

Based on posts here, I'd say the 1 GB will do, upgrading the memory to 2 GB is a good idea.

Message 15 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Hi Stephen,

You guys are brilliant!  I removed all the disks and inserted a blank, unformatted disk and it booted into 6.9.5!!

So I plan to wipe all the original disks and re-insert them before I do any configuration.

By the way, I inserted the blank drive into Slot 1 and it is working, so my 4.2.31 configuration just got messed up when the old drive died.

Hopefully, OS6 is more robust.

I did some reseach on memory and my concern is that all the DDR2 that is available is dual channel but I believe the RNP requires single channel memory which appears impossible to obtain.  Am I correct?

Thanks again for monitoring my painful crash.

Andy.

 

Message 16 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty


@AndyWheatcroft wrote:

I did some reseach on memory and my concern is that all the DDR2 that is available is dual channel but I believe the RNP requires single channel memory which appears impossible to obtain.  Am I correct?

 


@Sandshark is a better resource on hardware stuff than me, but I believe the dual channel/single channel configuration is more about the CPU chipset and system board than it is about the memory itself.   The Pro-6 does have two memory slots, but I haven't researched whether it the system board accesses it in dual channel mode or not.

 

You're looking for 240 pin DDR2 800 PC2 6400 Ram.

 

This should work I think, and would get you 4 GB of Ram for about $16.  https://www.amazon.com/gp/offer-listing/B0050D6KXI/ref=olp_f_new?ie=UTF8&f_new=true

Message 17 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Dual channel is a system board configuration.  Memory comes as "dual channel" only in that it is sold as two matched sticks.  The Pro6 motherboard is not dual channel.

Message 18 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Thanks Sandshark for the clarification - I have ordered two 2GB sticks.

My RNP is back in operation although it took 18 hours for it to build out my 5 x 3TB drives as a RAID6 volume.

When I started the re-sync, I had the side covers off and the drives over-heated in approximately 40 minutes, causing an automatic shutdown.  After putting the sides back on and setting the fan to "Cool", I haven't had any overheating issues.  I had replaced the main fan some time ago with a Noctura NF-S12A FLX 120mm Case Fan. It is so quiet, even in "cool" mode, and that seems to be keeping the disk temps down to around 50C.

Last question (hopefully) - I have read that some folks have upgraded their CPUs. I have no idea what is on my MB.  How can I deteremine what a good replacement CPU is for my board?

Many thanks,

Andy.

Message 19 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

If you have a "Pro6", as opposed to the earlier Pro Business Edition or Pro Pioneer, you really won't get a lot more CPU bang for the buck unless you go to a quad core.  Not only do I think that's unnecesaary, the TDP is such that you will likely have overheating problems.

 

You can check which CPU you have in dmesg.log in the log .zip file.  And while you are there, check the BIOS version in bios_ver.log.

 

If you have the newer Pro6, you have a Pentium E5300 @ 2.60GHz.  IMHO, an upgrade just isn't worthwhile.  But if you have one of the older ones, you have a Pentium E2160 @ 1.80GHz, and can gain a lot from an upgrade.  A Core2 Duo E7500 @ 2.93 is a good choice and cheap on eBay these days.  But, you need to have the latest BIOS (07/26/2010 FLAME6-MB V2.0) to go that far.  I don't recall how far up the list of Wolfdale family processors you can go without it, but I do know the E6600 does not require it because I had a unit in that configuration.  You must update the BIOS before you swap in an otherwise incompatible CPU, or it won't boot.  Don't try to go to a chip that has a max FSB greater than 1066MHz, as they won't work.  And, BTW, the board will stay at 800MHz FSB regardless.

 

Updating the BIOS in OS6 is a bit trickier than in OS4.2.x, as there was an add-on that did it under 4.2.x.

Message 20 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Wonderful information!

My CPU is:

Intel Pentium(R) Dual-Core CPU E5300 @ 2.60GHz (family: 0x6, model: 0x17, stepping: 0xa)

And my BIOS is:

07/26/2010 FLAME6-MB V2.0

 

So, I will not touch the CPU - I have read several posts about heat issues when folks have tried the quad cores.

Memory arrived today and now I have 4GB installed - thank you Amazon - $15.98.

 

I am a happy camper now with my well used ReadyNAS Pro6 running OS 6.9.5.  (I will wait a few weeks before upgrading to 6.10 in case there are issues with the older legacy machines.)

 

And after a ton of research, I have managed to restore (4) TimeMachine sparse bundles and got them "inherited" by the correct Mac.  That was no fun.

I'm wondering if a post would be in order to document the exact steps required.  It is not trivial.  I could not find a single post that walked me through the process.

 

Thanks again to you and Stephen B for your insight and help.

Andy.

Message 21 of 25
Sandshark
Sensei

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty


@AndyWheatcroft wrote:

 

I'm wondering if a post would be in order to document the exact steps required.  It is not trivial.  I could not find a single post that walked me through the process.

 


I'm sure that would be welcomed by the Mac users of the forum.

Message 22 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty


@AndyWheatcroft wrote:

 

I'm wondering if a post would be in order to document the exact steps required.  It is not trivial.  I could not find a single post that walked me through the process.

Posting the process would be helpful.

Message 23 of 25
AndyWheatcroft
Aspirant

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

I have added a lengthy post with the subject - "Restoring a Time Machine Sparse Bundle to a new NAS".

 

It is quite verbose since I wanted any user with any amount of knowledge to be able to follow.

Message 24 of 25
StephenB
Guru

Re: ReadyNAS Pro 6 - Disk showing "dead" even though bay is empty

Thanks Smiley Happy

Message 25 of 25
Top Contributors
Discussion stats
  • 24 replies
  • 3125 views
  • 0 kudos
  • 3 in conversation
Announcements