Re: RN316 soaped one drive and all volume is unreadable and HDD's are in red

ovidiu · ‎2019-02-20

I wanted to exchange the HDD's, because it showed degraded, but I had access to all the data.

In theprocess, I put a brand new WD RED 8TB, that prooved to be defective and I RMA-ed it.

I put it first in slot 1 , it started making some whirr noises (well , maybe that's what they do now I thought when the initialize) and after formating it and started resyncing it. Then it just plain dissapearead. I assumed (wrongly apparently) that HDD! has some initialization info, and the I tried slot 5 (HDD5) that was a WD red 3TB.

Before doing it I did add back the old HDD1 (The WD RE 2TB) , resyned and then only swaped to slot 5.

Same clicking noises, and at that point I realized the HDD is defective, put back the old HDD5 (a WD RED 3TB) ans start ed noticing that though all HDD were "healthy" the volume is degraded. Hooveringover the HDD's in the Web UI I could see that HDD1 and 5 had that error 14 and Hdd5 an error 4.

Data was accesible.

6.10-RC1 (I lurk here for more than 10 years it was Irfant, I always used the beta versions with no ptoblem, this is my 4th unit)

That's when the nigthtmare started. I was noticing tha HDD1 (old WD RE 2TB) was showing ATA error 14 ( no ideea what that means, all the smart tables I found go to 13 the jump in the hundreds, so I had two Seagate desktops 4TB laying around, and I thoought I will change with them

So I scrubbed the volume and proceededvwith changes.

After the 1st swap everything was OK, it was HDD1 (old WD RE 2TB).

I thought I will change the one with same error 14 in slot 5 *old WDred 3TB), out the other Seagate desktop, and that's when the fecal matter hit the fan.

Now I have no access to data, all disks are color red and from degraded it became dead.

I have the original HDD1 and HDD5 (the old WD's),

PLEASE let me know how to proceed.

I have (yes I know, I should have) no backup, though I felt I should back it up, I said , nah, worse case I will go degraded untill the new 3 disks arrive.

I am attaching the logs (downloaded through Raidar)

Sandshark · ‎2019-02-20

A degraded volkume with all "healthy" drives usually means the volume is out of sync. Until it's back to healthy, replacing a drive is not a good idea and replacing anything except the one that's out of sync is a disaster.

I'm not sure if you did actually replace the wrong drive at one point or if the (quite drive intensive) sync process pushed another drive over the cliff, but I agree it sounds like you are in file recovery territory.

View solution in original post

ovidiu · ‎2019-02-20

soaped=swaped...

long sleepless night..

Do I need to activate remote access to the server for troubleshooying?

StephenB · ‎2019-02-20

You might want to adjust the size of your signature.

You can try cloning the two suspect drives to new ones (using sector by sector cloning).

From there you can recover data by

using Netgear's data recovery service: https://kb.netgear.com/69/ReadyNAS-Data-Recovery-Diagnostics-Scope-of-Service
connecting all the hard drives to a Windows PC (likely using a USB adapter/dock) and then use ReclaiMe software. https://www.reclaime.com/
attempting to recover the data yourself (if you have the linux skills).

@ovidiu wrote:

I was noticing tha HDD1 (old WD RE 2TB) was showing ATA error 14 ( no ideea what that means, all the smart tables I found go to 13 the jump in the hundreds, so I had two Seagate desktops 4TB laying around, and I thoought I will change with them

First of all, you can't replace disks when the volume is degraded. That's the step that did the damage. When you replace a disk, the system uses RAID parity to reconstruct the data on the new drive from the others. That requires that the RAID array be redundant. If it's not the volume fails (as you found).

As far as ATA errors go, these aren't really SMART errors. The NAS itself reports them when the NAS SATA controller can't communicate with the disks. https://kb.netgear.com/19392/ATA-errors-increasing-on-disk-s-in-ReadyNAS

ovidiu · ‎2019-02-20

Thank you Stephan for your input.

Just before you replyed I panicked, and put back th original HDD1 (that were before the mishap with the defective 8TB). it's resyncing now, and what is annoying is that the system is described as Healthy but dead.

My Linux skills are abysmal at best, I can follow instructions like a monkey maybe, because I was used to DOS, but not familiar to Linux).

I can make a live Linux USB, and I do have 1 or 2 SATA internal cables on my Win machine.

Nothing was really written on the Hdd's ecept the re-syncing steps. Are there any chances of recovering the data?

StephenB · ‎2019-02-20

@ovidiu wrote:
Just before you replyed I panicked, and put back th original HDD1 (that were before the mishap with the defective 8TB).

This could have done more damage - potentially destroying your ability to recover the data. Hopefully it didn't. Don't do this with the other hard drive you removed.

@ovidiu wrote:

Nothing was really written on the Hdd's ecept the re-syncing steps. Are there any chances of recovering the data?

Re-syncing a newly inserted drive writes something to every sector on it. That destroys everything that was on it before.

ovidiu · ‎2019-02-20

I see, I might have diminshed the chances. Well, regarding the other hard-drive, that one was first put back at 10PM last night, when they all became color red, so basically it's in the initial "degraded" working configuration.

So I might have messed out more... hmm..

The "Status Healthy" message was confusing, and all HDD show nicely green dots and temperatures. So I assummed just that the degraded was just a SMART code for some minimal [roblem.

What was interesting, was that I was able to swap drives that resynced in that degraded state. SoI got encouraged and to change the disks with the smart ATA error code (showing only byhoovering over the bih NAS icon in WebUI.

Because the Deagates Showed no ATA eror I was expecting to fix the "degraded" message, Tha apppearead after I tried that unfortunatelly defective new WD Red.

In 6 hours thr resyncyng will be done.

What would be the best course of action.?

StephenB · ‎2019-02-20

@ovidiu wrote:

What would be the best course of action.?

Well, stop manipulating disks and look into data recovery.

Netgear offers a service: https://kb.netgear.com/69/ReadyNAS-Data-Recovery-Diagnostics-Scope-of-Service If you can connect all the disks to a Windows PC (USB adapters/dock is ok), then you could try ReclaiMe: https://www.reclaime.com/

Sandshark · ‎2019-02-20

A degraded volkume with all "healthy" drives usually means the volume is out of sync. Until it's back to healthy, replacing a drive is not a good idea and replacing anything except the one that's out of sync is a disaster.

I'm not sure if you did actually replace the wrong drive at one point or if the (quite drive intensive) sync process pushed another drive over the cliff, but I agree it sounds like you are in file recovery territory.

ovidiu · ‎2019-02-20

Thanks Sandshark,

I did swap the right drives, but made (now in hindsight) many mistakes. Like not backing up, swaping disks while the unit was running, got too confident that's OK after the first swap worked and checked and after, rebootng twice, was fine. I assume that's what get's us as a specie in trouble - overconfidence in things that we don't really understand

All I wanted was to get rid of that "degraded" thing that appearead after I tried a brand new new 8TB Western Red (that was unknowingly to me bad - again, I should have checked on my PC, but did not.

Followed Stephan's instructioI, bought an icident and a year of call support, because after this I ploan to change to the 10TB Seagate4 enterprise that are now very affordable and put the unit in a raid 6 this time.

So right now I just put the machine in tech suport mode, and opened a case support.

The confusing thing was "healthy disks" (which aparrently on hoovering in the WebUI NAS icon were not). So by trying to fix the "degraded" state i got in "dead" state, emboldened by appaent initial success.