RN204 - an early tale of woe but with a happy ending (shaggy dog story with a strong message)

Ok, bit of a story but hopefully it's a heads up to others.

I had some spare drives lying around so thought i'd pop those into an RN204 bought for the purpose. 3x 1TB drives and a single 500GB. Configured as X-RAID, the ReadyNAS creates two RAID5 volumes, a 4 disk one of 500GB and a 3 disk one of the remaining 500GB on the 1TB drives. It then presents the sum of that as a single data volume so quite painless and neat.

During some configuration, the unit didn't respond and I had to pull the power plug, at this point, it appeared to have lost the configuration and told me that I had to remove inactive disks to use the volumes data and data-0 (the above RAID sets) but the disks weren't inactive. Anyway, since I was just setting up, I put it down to one of those random IT things and carried on.

Having set up the volumes, I set to work shoving stuff from a plethora of disks onto this, you know the sort of thing, the family photos, the family videos, all that important stuff. Good plan, after all, this is going onto a RAID store...(you can see where this is going already right?)

Next, I thought i'd get smart. Rather than copy from a USB external drive on my PC, across the network to the NAS, why not just connect the USB 3 drive direct to the back of the NAS, surely the copy would be faster on the same hardware than introduce the network.

That all went smimmingly...until the last and yes really it was the last file, 7GB, not that that's of specific relevance but as my luck would have it, the NAS fell off the network, it would respond to ping but not the power button. Only one thing for it, pull the power plug.  (Searching forums, i'm not the only person who has had this mode of failure and requirement to pull the power)

Upon re-inserting the power plug and watching it boot, it flashed to say that drive 3 had failed (well I didn't think it had and it hadn't, it has been quite happy since) but I thought "Uh oh...at least it's RAID!" (are you falling off your chair laughing yet?)

So I went into the NAS web admin page to be told... "Remove inactive disks #1,#2#,#3,#4 in order to access the volume data" and as previously, it was showing me the raw data and data-0 RAID 5 sets. Going to the share config in the admin told me that I needed to create a volume first. WTF!

You might think, oh well, it was only copying to it but in classic screw up fashion, i'd been deleting as I went from my source disks *bangs head VERY hard*.

Now, just to put this into some perspective, there are many things that I'd be very ashamed to have to admit to my wife that i'd done. Telling her how I, an IT guy for 30+ years had just monumentally "lost" all our family pictures and videos was right there at the top of the shame list.

Being a holiday weekend, I figured now was the time to turn it off and just leave it before waiting 2 days to get in touch with Netgear support. I rationalised that the data must be there, some part of the config was missing but nonetheless, spent money on volume recovery/undelete software and went at the lengthy process of scouring my original disks to recover what I could.

I also had 2 days in which to read/research and see what other experiences people had with these units. Some even report the same issues (or very similar) in Amazon reviews.

When I did get through to Netgear the first level support girl was very understanding, had a look at the logs and said "one of the disks isn't on the supported HCL. We had a discussion on this point, I accept the HCL, non compliant statement but pressed my point that the disk that "failed" wasn't the non HCL one and that upon reboot again, it was not showing an error so I believed the error at the time to be a timing related issue or similar.

Anyway, she was courteous and escalated me to second level. "Ian" went through his routine (equally competently I might add) and conceded that this was going to be escalated and he wanted to go through the option to enable SSH...except you can't do that if you don't have a volume mounted. Never mind, paperclip in the rear and bring it up in the "Support" mode where it listens on a port. He confirmed that it was accessible and that was the end of his involvement.  (Slight confusion over what a # is called as part of the password.  You see, being English, we call that 'hash', it's not 'square' and definitely not 'pound', THIS --> £ <--- that's a pound sign! :)  We got there in the end though.

There the unit sat for a day... patiently waiting to be stroked and teased by a level 3 tech.

I did expect a phone call first but I happened to be sitting next to it and noticed it was active, disk activity, then it shutdown and restarted into the running mode. Immediately then came an email from level 3 tech to say "he had recovered the volume, take a look"

Immediate first task, do that backup of the RN204 to another disk, as was the original plan!

The unit has a choice of several apps which connect to various cloud storage providers for automated backup, one of those will be going on later and I shall also be writing out some Blu Ray discs for backup and looking at a fireproof safe.

I will replace the non HCL drive tomorrow, even though I don't see that as a factor here, Netgear wouldn't be unreasonable for denying support as they state they may. I've spent my working career for IT hardware and software manufacturers so I know how that works, the risk was mine.

Credit to Netgear support, very professional, courteous, understanding and with a fabulous outcome - HUGE THANK YOU! (I really appreciate the attendance and not an immediate brick wall of 'sorry, that drive is not on the HCL <call terminated>'

Now that the content is secure, i'm going to bully it a bit more and see if I can understand the failure mode, especially when I have 4 HCL compliant drives, then that might be a different discussion but for now, if you too want to avoid utter shame and a divorce lawyer...

- This is NOT your only back up solution
- Do make regular copies
- Don't delete your only original copy
- Establish an alternate back up
- DO use supported hardware, it avoids those awkward discussions about "but yeah it *should* be ok"
- Netgear Support may be able to help if you don't **bleep** around with it and make things worse

(All of the above being blindingly obvious to me throughout my career but I thought I was going to be ok just this once yeah yeah, laugh at me now but learn vicariously and don't be so foolish.  I've educated more than enough people over the years over the importance of backups etc.  geez, someone shoot me)

About the device itself, i'm sure it'll be a wonderful addition to the family IT infrastructure, once i've got past the stage of treating it like a cat that has just scratched my eyes out.

I like the iSCSI support, just need bigger disks to make more use of that to create volumes on others machines in the house, plus I need to have more confidence that the thing isn't going to drop the config on me again.

The ability to dynamically add bigger drives (one at a time to allow rebuild) and not have to fully backup (hahahaha) and reconfig the RAID sets seems like it should be a nice feature.

So at the moment, I'm not unhappy, just bruised confidence and after a few days of bullying it, if I can reproduce it dropping the volume configuration then i'll likely be winging its way back to Amazon.

Now,  to Netgear or anyone else, any thoughts on the failure where a power off/on without proper shutdown should cause the loss of the config data structure that maps the 'data' volume to the disk config?  This seems a bit catastrophic for an end user especially as i've now encountered it twice within 2 days and at this point, I still don't see it being related to a disk not being on the HCL.  I'd be interested to know what's involved at the btrfs or whatever side of things in terms of what would have been done to recover this as the web UI presents alarmingly dangerous options of 'remove all disks'.

Final words in case they haven't sunk in... you cannot replace those years of lost family pictures and videos, however you can replace the wife but that tends to be expensive (apparently).

(To the Netgear support staff who helped on case #26813237 - a seriously HUGE THANK YOU.  Support can be a thankless task, all you ever hear all day is problem, problem, problem but when the outcome is so successful, you should feel warm and fuzzy at the result).