NAS104 Raid decision? Xraid vs Raid 10 etc #24296869

Thanks. I just took a look at the issue raised with product engineering seeking an explanation for this. Here's the explanation they gave:

That's right in the expected range for RAID6 on a RN10x. RAID6 parity calculation is a very computationally intensive operation, and the CPU on the RN104 is not capable of doing RAID6 without a significant performance drop vs. other RAID levels. It's similar to using encryption on RN10x. RAID10 may be a better option if faster performance is desired. If you choose to continue to use RAID6, you can safely use the system normally with full data redundancy during the resync. So it shouldn't really matter how long the resync takes.

mdgm wrote:

Thanks. I just took a look at the issue raised with product engineering seeking an explanation for this. Here's the explanation they gave: That's right in the expected range for RAID6 on a RN10x. RAID6 parity calculation is a very computationally intensive operation, and the CPU on the RN104 is not capable of doing RAID6 without a significant performance drop vs. other RAID levels. It's similar to using encryption on RN10x. RAID10 may be a better option if faster performance is desired. If you choose to continue to use RAID6, you can safely use the system normally with full data redundancy during the resync. So it shouldn't really matter how long the resync takes.

Thanks for looking because they haven't told me anything. On the phone, the escalation level support people I spoke with said that it absolutely shouldn't take that long and the engineering team believe there is a software bug. It just finished building, it ended up taking about 48 hours.

The support agent handling your case probably hasn't had a chance to look at the update from product engineering yet.

StephenB wrote:
I haven't initialized with RAID-6 - in principle the RAID mode shouldn't matter much, since the two parity blocks are always the same, so they don't need to be recomputed. Though if they are computed over and over, it likely will take somewhat longer.

I'm not sure what you meant by this, but just so that it's clear, RAID 6 is not simply two copies of the XOR data. It's described nicely on Wikipedia.

Skywalker wrote:

StephenB wrote: I haven't initialized with RAID-6 - in principle the RAID mode shouldn't matter much, since the two parity blocks are always the same, so they don't need to be recomputed. Though if they are computed over and over, it likely will take somewhat longer. I'm not sure what you meant by this, but just so that it's clear, RAID 6 is not simply two copies of the XOR data. It's described nicely on Wikipedia.

I realize it is not two copies of the XOR (I've implemented Forward Erasure Correction using the Galois field techniques).

I was just trying to point out that if you are zeroing all the data blocks in the stripes when you create the array (or if you know all the data blocks are zeroed), then P and Q always end up the same (since the data blocks that feed the computations are identical). When the array is first being created you can potentially take advantage of that (and save time by skipping repetitive computations).

This wouldn't work for resync, and it wouldn't work in regions where you have non-zero metadata. But on initial creation of the array, it could save quite a bit of time (and result in all RAID variants taking about the same amount of time to create).

If I understand the btrfs "raid" concepts correctly, they are going one better. By integrating RAID into the filesystem, they only need to do the parity block computations on blocks that are actually used by the file system, so initial creation would be extremely fast.

Moving back on topic, the software RAID packages used in OS6 clearly don't have optimizations like that, since it took 28 hours to create the array.

Thanks for all the continued input everyone.

StephenB wrote:
Skywalker wrote: StephenB wrote: Moving back on topic, the software RAID packages used in OS6 clearly don't have optimizations like that, since it took 28 hours to create the array.

This seems to be at the crux of my next step. 48 hours actually was the time to build the raid 6. This concerns me very much as well as the support people telling me different things. Some have said this is absolutely not normal and the latest says it is. I don't get the sense that the raid implementations other than xraid have been optimized. It also makes me wonder how tested they are. Not many people seem to be using them on the 104 at least.

After the build I did another benchmark:

============================RAID 6
NAS performance tester 1.7 http://www.808.dk/?nastester
Running warmup...
Running a 100MB file write on Z: 3 times...
Iteration 1: 11.44 MB/sec
Iteration 2: 11.78 MB/sec
Iteration 3: 11.78 MB/sec
-----------------------------
Average (W): 11.66 MB/sec
-----------------------------
Running a 100MB file read on Z: 3 times...
Iteration 1: 10.77 MB/sec
Iteration 2: 11.73 MB/sec
Iteration 3: 11.11 MB/sec
-----------------------------
Average (R): 11.20 MB/sec
-----------------------------
Running warmup...
Running a 400MB file write on Z: 3 times...
Iteration 1: 11.77 MB/sec
Iteration 2: 11.78 MB/sec
Iteration 3: 11.79 MB/sec
-----------------------------
Average (W): 11.78 MB/sec
-----------------------------
Running a 400MB file read on Z: 3 times...
Iteration 1: 11.64 MB/sec
Iteration 2: 11.70 MB/sec
Iteration 3: 11.67 MB/sec
-----------------------------
Average (R): 11.67 MB/sec

Running a 2000MB file write on Z: 3 times...
Iteration 1: 11.81 MB/sec
Iteration 2: 11.81 MB/sec
Iteration 3: 11.81 MB/sec
-----------------------------
Average (W): 11.81 MB/sec
-----------------------------
Running a 2000MB file read on Z: 3 times...
Iteration 1: 11.79 MB/sec
Iteration 2: 11.41 MB/sec
Iteration 3: 11.38 MB/sec
-----------------------------
Average (R): 11.53 MB/sec
-----------------------------

As you can see the results are only marginally slower than the raid 10 implementation. Per your earlier point one of the limiting factors seem to be the network. Frankly though I would imagine most users of the NAS104 will be using it via a wireless network. This isn't the device you buy for a robust business environment.

My original goals were to have a storage device that heavily protected me against disk failure that also performed well. The results seem to depreciate the value of Raid 10. So I would think I either want to go with Raid 6, Raid 1 or Xraid. Obviously, Xraid has the benefit of giving me more space. In the end the raid 10/raid 6 only give me 5.45TB of usable space. The problem is that now my confidence in the Raid6/Raid10 implementation is not high. I would assume the simplicity of Raid 1 makes it effective. But I am not sure what to do now. Should I just go with Xraid and cross my fingers there is no failure or that the failure never exceeds a disk? Or should I trust one of these other implementations and sacrifice the disk space? Decisions decisions...

StephenB wrote:
I realize it is not two copies of the XOR (I've implemented Forward Erasure Correction using the Galois field techniques).

I'm confident you know how it works. It just sounded unclear when I read it, so I figured it may confuse others.

BTRFS RAID 5/6 isn't robust enough for us to use yet. If we could be certain that all the disks contained all zeros when the system was factory defaulted, we could skip the initial sync. But the initial sync really shouldn't be too bothersome, since the system if fully usable and fully redundant during the entire sync. A rebuild is a different story, for obvious reasons.

offbyone wrote:

This seems to be at the crux of my next step. 48 hours actually was the time to build the raid 6. This concerns me very much as well as the support people telling me different things. Some have said this is absolutely not normal and the latest says it is. I don't get the sense that the raid implementations other than xraid have been optimized. It also makes me wonder how tested they are. Not many people seem to be using them on the 104 at least.

The support people aren't really familiar with people running RAID 6 on RN104, because, well, almost nobody does. In the market segment where the RN104 resides, capacity per dollar is of more interest than maximum redundancy.

The RAID implementation on the RN104 is exactly the same as the one used in every other ReadyNASOS-running system uses. There is no "optimizing" to do. Since you're capped at 100Mbit by your network equipment, RAID 6 isn't going to slow you down anyway.

Skywalker wrote:
BTRFS RAID 5/6 isn't robust enough for us to use yet. If we could be certain that all the disks contained all zeros when the system was factory defaulted, we could skip the initial sync. But the initial sync really shouldn't be too bothersome, since the system if fully usable and fully redundant during the entire sync. A rebuild is a different story, for obvious reasons.

Agreed.