Reply

Re: X-RAID 2 Operation and Downsizing

Learning2NAS
Tutor

X-RAID 2 Operation and Downsizing

I've tested answers in other posts on my NAS and found some of them to be incorrect, so please cite when you can to support your response.

 

Questions:

(1) If I lose a disk in a RAID-5 and don't have an equal capacity replacement on hand, can I use a smaller disk to fill in the dead disk's place until the new disk arrives? (This would downgrade the RAID volume to a lower capacity for the interim).

 

(2) Since X-RAID 2 works by creating layers, does the answer to (1) change if my array was originally created on smaller disks, and my temporary disk is the size of the original disks that my volume used to be made of? Or do those layers disappear each time the volume expands?

 

(3) I currently have disks of varying sizes in my array and the ReadyNAS is utilizing 100% of their capacity. If it can do this as they're added to the array, regardless of what order their added, why can't it downsize if there is free space to do so? Please explain this in detail, rather than saying that XRAID is meant to be expandable only (a common answer I've found to similar questions).

 

(4) If anyone knows where there is a full technical specification breakdown for X-RAID 2, please post it. It would be helpful to have an X-RAID bible around for future reference.

 

Thanks for your time!

 

NOTE: For others who are curious the best discussion I've found on X-RAID is at the link below, but it doesn't directly address the questions posted above.

https://community.netgear.com/t5/ReadyNAS-in-Business/Expansion-with-X-Raid2-resulting-sizes-and-dif...

Message 1 of 11

Accepted Solutions
StephenB
Guru

Re: X-RAID 2 Operation and Downsizing


@Learning2NAS wrote:

Hey Stephen,


Thanks for answering my questions. You addressed everything in my original post, but I have a clarification to your response to #2 (quoted below). Using your hypo, if I have a base layer of 1TB that is always there and my current array is 1x1TB (original disk that hasn't died yet) and 2x3TB, can I replace the 1x1TB with another 1TB? Additionally, can I replace the 3TB with a 1TB if it fails? It sounds like the answer to that second question is no, because there might be data stored somewhere in the 2TB layer, but I just want to verify this.

 


You can replace the 1 TB drive with another 1 TB drive.  But you can't replace the 3 TB drive with a 1 TB drive.

 


@Learning2NAS wrote:

 

 

If that's true, that brings me full circle back to what I'm having such a hard time understanding: when you have a failure in RAID 5 there is enough redundancy to rebuild an effected file from the remaining drives (that's the point, right?). With this being so, I don't understand why the RAID can't rebuild the missing files on any size drive I stick in, so long as it is at least large enough to support the amount of data that needs to be striped to the new drive. 



The RAID software sits between the physical disks and the file system - and has no idea about how the file system is organized (e.g., where files are stored, and where there is free space; or even what the file system is).  When you replace a drive, the RAID software reconstructs all the disk blocks on the drive from the blocks on the other drives (whether there is data in them or not).  Likewise, a resync always takes the same amount of time, no matter if the file system is full or empty.  So RAID treats all data blocks on the drives identically (used or not).

 

 

There potentially are some advantages to fully integrating RAID-like redundancy into the file system itself - then features like the ones you are envisioning would be easier to do.  The BTRFS team is working on a design like that, but it is still experimental - so Netgear (wisely) isn't using it.  If they were to switch to it, it would likely require a factory reset.

View solution in original post

Message 4 of 11

All Replies
StephenB
Guru

Re: X-RAID 2 Operation and Downsizing


@Learning2NAS wrote:

 

 

(1) If I lose a disk in a RAID-5 and don't have an equal capacity replacement on hand, can I use a smaller disk to fill in the dead disk's place until the new disk arrives? (This would downgrade the RAID volume to a lower capacity for the interim).

No.  One exception might be if some of the disk wasn't actually in the array.  For instance if you had 3x2TB+3TB, you might be able to replace the 3 TB drive with a 2 TB drive.  I haven't tried this, so I don't know for sure what would happen.


 

(2) Since X-RAID 2 works by creating layers, does the answer to (1) change if my array was originally created on smaller disks, and my temporary disk is the size of the original disks that my volume used to be made of? Or do those layers disappear each time the volume expands?

 

The layers never disappear.  So if you start with 3x1TB, the array will always have a 1 TB base layer.  If you then expand to 1x1TB+2x3TB, the array will aways have a second 2 TB layer.  It doesn't change the answer to (1).



 

 

(3) I currently have disks of varying sizes in my array and the ReadyNAS is utilizing 100% of their capacity. If it can do this as they're added to the array, regardless of what order their added, why can't it downsize if there is free space to do so? Please explain this in detail, rather than saying that XRAID is meant to be expandable only (a common answer I've found to similar questions).

 

I think it is theoretically possible to downsize.  Basically you need to make sure that any file data that is in the section you want to remove is moved to a new safe area before you destroy it.   In principle, this could be done.  In practice I think it would be hard to make it reliable.  In any event, Netgear hasn't chosen to invest the amount of work needed to do it.  

 

If they did do it, it would almost certainly need to be done off-line (with the data not available).

 

Expanding is much simpler, since you know for certain that there is nothing in the newly added space.



 

 

(4) If anyone knows where there is a full technical specification breakdown for X-RAID 2, please post it. It would be helpful to have an X-RAID bible around for future reference.

 


I haven't seen a full specification. I think that it is better to think of it as a collection of expansion features, which are implemented in somewhat different ways on different platforms.

 

For instance, XRAID-2 in NAS running 4.2.x uses a combination of lvm and mdadm and the ext file system.  In NAS running 6.x.x, it uses mdadm and btrfs (and no longer uses lvm).   

 

So I don't think there is a single technical specification, rather it is a set of behaviors that various generations of ReadyNAS implement in somewhat different ways.

Message 2 of 11
Learning2NAS
Tutor

Re: X-RAID 2 Operation and Downsizing

Hey Stephen,


Thanks for answering my questions. You addressed everything in my original post, but I have a clarification to your response to #2 (quoted below). Using your hypo, if I have a base layer of 1TB that is always there and my current array is 1x1TB (original disk that hasn't died yet) and 2x3TB, can I replace the 1x1TB with another 1TB? Additionally, can I replace the 3TB with a 1TB if it fails? It sounds like the answer to that second question is no, because there might be data stored somewhere in the 2TB layer, but I just want to verify this.

 

If that's true, that brings me full circle back to what I'm having such a hard time understanding: when you have a failure in RAID 5 there is enough redundancy to rebuild an effected file from the remaining drives (that's the point, right?). With this being so, I don't understand why the RAID can't rebuild the missing files on any size drive I stick in, so long as it is at least large enough to support the amount of data that needs to be striped to the new drive. Ex: Assume the array above only has 200gb of data on it, for whatever reason. Sticking in a 1TB replacement for a 3TB drive shouldn't cause a problem, as no data in the "new" 2TB layer is missing (yet) because it can be rebuilt from the other drives, and whatever the rebuilt stripes look like will easily fit on a 1TB replacement disk. So why wouldn't this be possible? A rebuild is a rebuild, right?


@StephenB wrote:

 

(2) Since X-RAID 2 works by creating layers, does the answer to (1) change if my array was originally created on smaller disks, and my temporary disk is the size of the original disks that my volume used to be made of? Or do those layers disappear each time the volume expands?

 

The layers never disappear.  So if you start with 3x1TB, the array will always have a 1 TB base layer.  If you then expand to 1x1TB+2x3TB, the array will aways have a second 2 TB layer.  It doesn't change the answer to (1).

Message 3 of 11
StephenB
Guru

Re: X-RAID 2 Operation and Downsizing


@Learning2NAS wrote:

Hey Stephen,


Thanks for answering my questions. You addressed everything in my original post, but I have a clarification to your response to #2 (quoted below). Using your hypo, if I have a base layer of 1TB that is always there and my current array is 1x1TB (original disk that hasn't died yet) and 2x3TB, can I replace the 1x1TB with another 1TB? Additionally, can I replace the 3TB with a 1TB if it fails? It sounds like the answer to that second question is no, because there might be data stored somewhere in the 2TB layer, but I just want to verify this.

 


You can replace the 1 TB drive with another 1 TB drive.  But you can't replace the 3 TB drive with a 1 TB drive.

 


@Learning2NAS wrote:

 

 

If that's true, that brings me full circle back to what I'm having such a hard time understanding: when you have a failure in RAID 5 there is enough redundancy to rebuild an effected file from the remaining drives (that's the point, right?). With this being so, I don't understand why the RAID can't rebuild the missing files on any size drive I stick in, so long as it is at least large enough to support the amount of data that needs to be striped to the new drive. 



The RAID software sits between the physical disks and the file system - and has no idea about how the file system is organized (e.g., where files are stored, and where there is free space; or even what the file system is).  When you replace a drive, the RAID software reconstructs all the disk blocks on the drive from the blocks on the other drives (whether there is data in them or not).  Likewise, a resync always takes the same amount of time, no matter if the file system is full or empty.  So RAID treats all data blocks on the drives identically (used or not).

 

 

There potentially are some advantages to fully integrating RAID-like redundancy into the file system itself - then features like the ones you are envisioning would be easier to do.  The BTRFS team is working on a design like that, but it is still experimental - so Netgear (wisely) isn't using it.  If they were to switch to it, it would likely require a factory reset.

Message 4 of 11
Learning2NAS
Tutor

Re: X-RAID 2 Operation and Downsizing

I'm following you now. It seems like when XRAID 2 initializes a disk for the first time it creates the RAID "container" (I don't know the technical term for this proprietary RAID, but I'm thinking LVM like functionality in BTFRS) so that it fills the disk. Since RAID operates at the block level, the next disk must be able to hold at least the current container. The RAID controller isn't able to shrink the container to fit the new drive because the controller doesn't know where the data is inside the container, it just copies/rebuilds block by block. So install too small of a drive and system will probably just ignore it.


That's pretty cool, and very helpful to understand.


I did some searching before writing this reply and I found several products that are playing with file system RAID. As far as Netgear is concerned, only some of their enterprise products advertise block-level replication (Netgear ReadyDATA in particular). How do you know that "prosumer" products running OS6 are using block level RAID, rather than a software based file system RAID?

I think you're right, I just like to have test results or a source to cite. I intend to test this sometime soon. I will post results in the next few days for all who are/may be curious in the future.

 

I will mark your answer as the solution once I confirm its accuracy. Thanks for your time 😃

Message 5 of 11
StephenB
Guru

Re: X-RAID 2 Operation and Downsizing


@Learning2NAS wrote:

How do you know that "prosumer" products running OS6 are using block level RAID, rather than a software based file system RAID?

Software raid on linux is block-level.  ZFS and BTRFS are the only file systems I know of that have attempted to integrate raid features into the file system itself (there could be others, but I don't know of them).  

 

BTRFS's RAID integration isn't mature/stable.  ReadyNAS definitely runs mdadm (https://en.wikipedia.org/wiki/Mdadm).  You can see the layering partitions in the downloaded logs.

 

 

Message 6 of 11
Learning2NAS
Tutor

Re: X-RAID 2 Operation and Downsizing

Test results are in. I found a useful blog post about shrinking linux software RAID sizes here (http://www.starcoder.com/wordpress/2011/08/shrinking-a-linux-software-raid-volume/). Those instructions allow you to easily shrink a partition, to remove a disk from the array, or both. However, that process only works smoothly with a manually configured RAID array. Since Netgear devices default to XRAID2 and create multiple layers on each drive, it is more complicated than those instructions. It is still theoretically possible to reduce the size of an XRAID2 and you would do it by repeating the steps in that blog post, however it is a time intensive process and there is no guarantee that it will work (in my case, I had a small stripe that was created by XRAID2 that was 100% full, and spread across several drives, so I could not resize that small stripe whatsoever. This proved fatal to my shrink operation, and made it fail.).

 

In the end, I was not able to resize my XRAID2 array without destroying the volume and having the device rebuild it. If this function is important to users, they should be mindful to build a Flex-RAID array when commissioning their devices (however, this is more complicated and should be reserved for somewhat experienced users).

 

I also tested the automatic functions of XRAID to confirm that it was unable to shrink a drive on its own. When you insert a drive smaller than the one that was there previously, the system just throws and error and continues to operate in degraded mode. It will not shrink a volume, automatically or otherwise.

 

I appreciate your time, Stephen. Hopefully this is helpful to someone in the future.

 

Message 7 of 11
IcyK
Tutor

Re: X-RAID 2 Operation and Downsizing

Regarding Flexraid: on OS6 you can change from xraid to flexraid and back. So that may be a route to test something.

However it seems that a volume that has been expanded using flexraid can NOT be switched back to xraid.

 

Thanks for the link. If I can find some time, I'd like to try to change an OS6 raid6 volume 'back' to raid5 and use the remaining drive for space.

With bitrot protection enabled, raid6 maybe is overkill.

Message 8 of 11
StephenB
Guru

Re: X-RAID 2 Operation and Downsizing


@IcyK wrote:
 

With bitrot protection enabled, raid6 maybe is overkill.


RAID-6 gives you more availability on drive replacements (e.g., the volume stays up even if you get two disk failures).

 

But you still need backups, which is what keeps the data safe.   I've chosen RAID-5 for the main NAS.  If I were running a business using it, I might use RAID-6.  But with my current usage, losing access to the volume for a couple days would be an inconvenience, not a disaster.

 

Bitrot protection doesn't provide the same protection features as RAID-6 (and vice versa).  It can't protect against a double-disk failure.  It is designed to recover a file that gets corrupted somehow on the volume. 

Message 9 of 11
Learning2NAS
Tutor

Re: X-RAID 2 Operation and Downsizing

I tried another experiment and got a positive result. This one is not as interesting as the original, but it is worth letting the community know about in case this ever helps someone in the future.

 

I created an XRAID2 volume using wildly different sized drives (40GB, 300GB, and 500GB). XRAID, predictably, decided to create several different layers. It ended up doing the following:

1 40GB layer that ran across all three drives (RAID 5)

1 260GB layer shared between the two largest drives (RAID 1)


That decision left 200GB of unused space on the largest disk in the array. I have been repeatedly told that you cannot replace a disk with a smaller disk, but in this situation it would seem silly not to allow a smaller disk to fill in for the 1/3 empty 500GB drive.

 

I pulled out the 500 after the array finished bulding and hot-swapped for a 320 disk (which was 20GB larger than it needed to be to accommodate all of the data on the 500). Although I had just swapped a larger disk for a smaller disk, the ReadyNAS unit accepted it and rebuilt the array onto the smaller disk. I don't know how often this will be helpful, but it is good to know that the drive looks for the necessary capacity, rather than an exact replacement. Makes sense, but it's good to have confirmation.

Message 10 of 11
Learning2NAS
Tutor

Re: X-RAID 2 Operation and Downsizing

IcyK,

 

I agree with Stephen, RAID 6 and Bitrot protection are for different kinds of failures. There are certainly situations where it makes sense to have a RAID 6 with Bitrot protection turned on, but it depends on your use case. Unless you're taxing your unit heavily, I would advise you to leave Bitrot protection on because it's a good insurance policy for your data.

 

However, with regard to your question about RAID 5 or RAID 6, I have another link to give you. Take a look at the "Ideal Uses" section under each RAID level. http://www.prepressure.com/library/technology/raid If you're trying to warehouse data for as little money as possible and don't have an enormous array (10+ disks) or high capacity disks that take a long time to rebuild (3TB +), RAID 5 is probably the right choice.

 

However, if you're trying to store mission critical information where downtime is unacceptable or where data loss is unacceptable, RAID 6 is the best option. I've seen a lot of people use RAID 6 for volumes that have their operating system installed on them, for example, and RAID 5 for their storage volumes because it provides a more efficient use of their disk space. That said, you just have to determine how much risk you're willing to accept. Less risk? RAID 6. Some risk? RAID 5 and get a bit of bonus storage for your trouble.

 

Note that this discussion is off topic for this thread, and you should post elsewhere if you have additional questions.

 

Take care

Message 11 of 11
Top Contributors
Discussion stats
  • 10 replies
  • 3963 views
  • 1 kudo
  • 3 in conversation
Announcements