ReadyNAS 316 - file system read only? Unusable #27793592

So I can't create one big 6 x 3TB RAID array and then break that down into different volumes? Effectively partition it up.

I'm looking for protection against a corrupt file system which appears to be the case here.

Failing that I guess two 3 x 3TB RAID 5 volumes would have the same capacity as RAID6 but also protection against a single volume corruption?

Answered some of my question by reminding myself of the volume configuration screen.

http://kb.netgear.com/23131/How-do-I-create-a-volume-on-my-ReadyNAS-OS-6-storage-system?cid=wmt_netgear_organic

So I have to pick whole drives to add to a volume, I can't have separate partitions in the same array. Shame.

Yes, a drive can't be split and put into two volumes.

Also, two 3x3 TB volumes would have the same storage overhead has RAID-6.  It would protect against some combinations of 2-drive failures, but not all of them. Performance would be faster than RAID-6. 

I can put a drive in two volumes?

With 6 x 3TB drives, can I have 5 x 3TB volumes? Effectively equivalent to five external 3TB drives but with parity to protect against a single drive loss.

---

ro53ben wrote:

I can put a drive in two volumes?

 

 

OK, thanks for the feedback, much appreciated.

I guess what I should really wait for, before rebuilding, is an OS update.

---

ro53ben wrote:

 

I guess what I should really wait for, before rebuilding, is an OS update.

---

What firmware are you running?

The latest 6.6.0 but, looking at other threads, it looks like there may be a bug.

---

ro53ben wrote:

The latest 6.6.0 but, looking at other threads, it looks like there may be a bug.

---

There can be (and will) always be bugs, but I'm not seeing any smoking gun on this.  There are always some people dealing with data corruption on all of the ReadyNAS platforms - both btrfs and legacy ext volumes.  And I'm sure it's the same in competitor forums.  FWIW I'm running 4 OS6 systems at the moment, and haven't ever had this issue. 

Never has a loss myself before either, even on preceeding NV+ kit, but something caused that crash last weekend and I'm quite sure the end result was this terminal corruption.

Interesting time trying to archive off the TBs of data here. Usually getting the expected ~100MB/s but after a while it just starts going really slow, almost to a complete halt...like 200KB/s at most. Sometimes it times out completely. After a reboot I get full throughput again.

Had to reboot twice in the past couple of hours. At this rate it's going to take me days to get the data off.

Woke up to a dead NAS and this amazing error this morning:

https://goo.gl/photos/veVNrFiJRR36PGhT6

Bit rot has detected an error within //usr/lib/x86_64-linux-gnu/libndr-standard.so.0.0.1 and cannot correct the error.

Not a good thing, right?

Hi ro53ben,

Seems that no one in the community had the same experience. Would you be able to send the logs? I will try and have your case be checked further.

How do I send all logs to ReadyNAS Community moderators?

Regards,

Device is currently in tech support recovery mode whilst one of your L3 guys tries to recover my lost LUNs. There are log copies attached to that ticket.

When the volume got corrupted, it went into read-only mode. This still allowed me to easily copy off the data from the shares on the device, but it's seemingly impossible to access any iSCSI LUNs.

At this point I'm going to recommend to anybody reading this to never use the iSCSI function on ReadyNAS with any live data. In the case of a corrupt file system on the NAS, you are no longer able to mount the LUNs in any usable way. Throw Windows deduplication into the mix and you're in a real mess.

I've had a ticket open since Saturday morning and we're now at Tuesday morning and struggling to get any usable data.

Looking at the case it sounds like copying the data off to USB disks is working well.

iSCSI LUNs working normally does require the filesystem to be mounted read/write.

In any case if the data's important it is important to have backups in place.

The challenge here is that the ReadyNAS doesn't provide any capability to back-up iSCSI LUNs. The built-in backup tool will allow you to schedule external copies of any shares, but not of the LUNs.

The copying data off to USB disks has had no success at all yet. The guy has mounted a deduplicated Windows 2016 LUN in an incompatible Linux NTFS reader and is taking copies of files that can't work as they need to be rehydrated before they will be usable.

The only way I can get to this data is to mount it in Windows 2016, that's not happened yet and we've lost a few days copying data that won't work by internally mounting an unsupported volume.

Hi ro53ben,

I will be keeping the thread open to give for other community members to share or read your current concern.

Regards,

Thanks.

Still working hard here to try and get some data back.

It's worth pointing out that the Microsoft Windows 2012/2016 Deduplication is VERY proprietary. If you need to somehow recover a LUN in this format, you've really got your work ahead of you.

The support engineer mounted the LUNs internally and copied the contents but all we've ended up with is an entirely empty file structure except for files less than 32KB in size which are smaller than the deduplication cut off.

We're now going to attempt to unmount and then copy the entire raw LUN to the external USB drive and try and mount it from there as it will then be hosted on a read/write volume.

---

ro53ben wrote:

 

It's worth pointing out that the Microsoft Windows 2012/2016 Deduplication is VERY proprietary. If you need to somehow recover a LUN in this format, you've really got your work ahead of you.

 

The new ReadyDR backup will back up changes to LUNs very efficiently (thanks to CoW in the underlying btrfs file system).  I suspect that you can't create a ReadyDR share on a USB backup drive though - if i am right on that, you'd need to backup to another business ReadyNAS.

Yes, as StephenB mentioned you can use the new ReadyDR feature to backup LUNs from one x86 ReadyNAS to another. They must both be x86 ReadyNAS.

In 6-bay that would mean the RN316, RN516 or RN716X or the new RN526X and RN626X

At this point I'm not entirely convinced that the backup solution for a £1000 ReadyNAS set-up that randomly corrupts its own file system is another £1000 ReadyNAS set-up.

I've still yet to see a single byte of restored data from my premium data restore ticket, which isn't really giving me the assurance in the brand I need.

Needless to say, I won't be using iSCSI LUNs on this device in future - I wouldn't even trust them for test data. Also, if I hear another support engineer suck air through his teeth regarding snapshots, I shall probably abandon the use of those too.

As far as I can tell so far, ReadyNAS OS6 is perfect as a long as you:

- Buy two devices, not one.
- Don't use iSCSI LUNs
- Don't fill your volume beyond 80%

- Don't use bitrot on data that changes (about the only files it's of use on)

- Don't use snapshots (the hourly setting is just for fun, you aren't supposed to actually use it)

- Don't connect modern (post 2011) Windows OS to it.

Another day or so later, still nothing restored.

Actually got no idea if the L3 guy is working today, he never responded when asked about it yesterday. I needed to get this device back online for Xmas, it's my media server and we planned to watch a lot of TV/Films over Christmas but right now I've got nothing. Nothing at all. It wouldn't be so bad if this was free support, but it's not - I've paid circa $200 for this.

I may sound a bit grouchy, I probably am. I've not slept all week trying to stay up to align my time zone with an L3 analyst based in California. It hasn't worked well. I spend hours after midnight waiting for replies which don't really answer any of my questions and tend to arrive after I fall asleep.

All I wanted when I opened this ticket on the morning of the 17th was to make some progress, I feel like I've made none.

It's becoming clear this is no longer the artist formally known as ReadyNAS - it's all Netgear these days.

Merry Christmas