Checksum capabilities with btrfs on top of mdraid?

[Andlier]

As I understand, readynas OS6 implements btrfs on top of an mdraid volume because native raid-support in btrfs is still a bit experimental (introduced 2013). How does this affect the checksum capabilities of btrfs?
1: Does btrfs on top of mdraid have any clue what physical disk the data belongs to? So a checksum error can be narrowed down to a specific disk.
2: Is btrfs capable of correcting checksum errors at all? Or does it only detect them, and then you have to hope for a hard-drive reporting an issue to have mdraid resolve it correctly?
3: Has anyone tried to take out a disk of the nas, dd some zeroes to it and plug it back in and check what happened? did a scrub detect it? Did it get fixed or only flagged?
4: Any plans from Netgear to implement native btrfs raid support? I suspect this is the only way to get all those nice zfs features, such as proper error correction and data-aware scrubbing.

Any other comments to the questions/issues above? What is the benefit of having btrfs on top of mdraid compared to ext4 on mdraid?

[StephenB]

btrfs checksums apply to metadata and data blocks. OS6 lets you turn them off if you like. They are crc32 checksums - and therefore in themselves are only useful for error detection, not recovery.

Classic RAID protection only kicks in when sectors cannot be read. Then redundancy is used to reconstruct them. Re-writing the data blocks at that point should result in reallocated sectors (which then can be read, at least for a while). So I don't think your scenario in (3) will be corrected. btrfs itself can store redundant metadata, so that is perhaps an exception.

btrfs snapshots are more useful than anything you can get in ext4, and there is no need for journalling. It also does not have the RAID expansion limits that ext4 implementations in the 4.2.x NAS have.

Personally I have found btrfs in OS6 to be stable, and I do like the snapshot feature. But I am also perfectly fine with ext4+journalling, and have no immediate plans to install OS6 on my pro-6.

I don't know what plans Netgear has in the future. Integrating a variant of RAID into the filesystem (which is basically what btrfs is doing) is a good idea in principle. You'd need a factory reset to migrate to that though, and obviously it will need a lot of testing. Keeping RAID as a strictly separate lower layer is definitely less complex.

[Scram]

I've been poking around my own RN104 ...

It might be possible to get Checksum-autohealing with ReadOS6 without loosing too much.

Readnas creates raid devices for every hard disk. If you put in multiple hard disks, and select "expand volume", it will create one md device (raid0 or raid1 depending on several facts i've not clarified yet) per hard disk

over all those devices it creates ONE btrfs filesystem (by using btrfs device add or something similar).

When logging in to ssh and using "btrfs balance -mconvert=raid1 -dconvert=raid1 /data" you can convert the volume to raid-1.

Raid-1 in btrfs interpretation is: Every chunk of data is to be written to two different devices to be redudantly available. So raid-1 in btrfs will be able to use 4 disks of say 2TB each, and will have 4TB available.


BTRFS and its raid-implementation is highly experemential, so if you want to try this out (I'm in progress of doing so, still copying my 5TB of data from my two old, slow NAS to the RN104...) - do it at your own risk.

I won't take any guarantees, and netgear won't , i'm sure.

[StephenB]

Scram wrote:
...BTRFS and its raid-implementation is highly experemential, so if you want to try this out (I'm in progress of doing so, still copying my 5TB of data from my two old, slow NAS to the RN104...) - do it at your own risk...

So why exactly do you want to convert your everyday file system to a mode that is experimental and unstable?

I get the self-healing idea, and see value in integrating raid-type modes into the file system itself. But I don't get why you'd switch to this now, with all the warnings that it isn't ready for prime time yet.

[drewttt]

So you don't get healing with checksums enabled on the RN104? Darn, that's why I bought this btrfs NAS.

Is it safe to rebalance manually to a -d raid1 to get redundant data copies for recovery after a checksum error during scrub? Would it completely kill performance?

[xeltros]

RN104 is pretty weak (it doesn't even handle gigabit ethernet speed while copying...), so any kind of checksum, virus scan, scrub, rebuild, etc. will surely decrease the perf.
When you go to system/volumes/clic on the space graph/settings you have an option called checksum. Not sure about what it's actually doing, and I'm pretty sure performance will decrease (not sure how much though) but if your NAS is still in test mode (means you have a duplicate of all data in it and time to restore everything of needed) you can give it a try and factory default if needed.

I don't really see what you are willing to do there. The BTRFS raid 1 options sounds close (not identical though) to raid 10 which I think is supported by RN104 and if you want a full redondant thing, why not going the raid1 with 4 disks ? As for the checksum, I believe that's the rebuild role to do that when a new disk is inserted. I see no other purpose than detect incoherences for the rebuilding process, it won't surely take 2 days to add a disk to a raid without any kind of verification.
and isn't that the raid job to make sure everything is well copied ? I believe after a copy there is a read to check that or at least an "ack", am I wrong ?

So can you be more specific as to why you want this ? it would be a better way to help you find an alternative I think.

[drewttt]

I want this feature after reading through this article, which includes a description of bit-rot, silent data corruption that traditional raid could miss - http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/#image-2

I figured that a btrfs volume with its snapshots and data protection would be a good target for backup. The "raid1" in the btrfs filesystem comes into play, because if the filesystem detects a checksum mismatch, during a normal read or a scrub, it can attempt to get a good copy of the data from the redundant store.

I'm pretty happy with the speed so far. Doing a large copy from the RN104 to a local client, getting ~70MB/s, so not full gigabit, but acceptable. I may go ahead and just mess around with it as you suggest, the RN104 is being added as a new device for secondary backup, so I can burn it down to test things out, before it is relied upon.

[xeltros]

Bit rot is indeed a concern. Never seen one though. Never seen any countermeasure in up to mid-sized enterprise either. Maybe it's more seen with high speed transactional data or with long term storage (tapes).
What I'm saying is that you may end up with more trouble messing with the NAS than you could actually experience with bit rot. Messing with the raid can actually get you boot problem, disk adding errors, disk replacing problems, scrub/defrag buttons broken. If you really want to go the full BTRFS way i would advise to get a fresh Debian and do it all by yourself but a NAS doesn't seem a good place to experiment to me.
The way I see things, you may ask Netgear what they have in place to deal with it, use what you can that's already integrated on the NAS, maybe do a little tweaking but rebuilding a raid is definitely more than just that.

You can always try with a VM though, seek "OS6 tips and tricks" in "user submitted tips" forums, I think I posted the link in it.