Hoping someone can lend some help and maybe provide a solution.
I am helping a friend out with her ReadyNAS. She's not big on being on forums or social media so relaying things for her.
She has a ReadyNAS 428 running 6.10.9, it had eight 10TB drives in one volume. With five 8TB drives in an EDA500
She wanted to upgrade the first volume with 16TB to get more storage. As a cautionary measure due to my own experience with ReadyNAS. I told her to turn off X-raid before putting in the last drive to upgrade.
In the past, every time I've done an upgrade of drives, ReadyNAS has changed from Raid5 to Raid6 on me without asking if I wanted to do that.
So once the last drive sync'd she tried to turn x-raid back on. Unfortunately she now gets this message
"You are attempting to to switch from Flex-RAID to X-Raid. This is not possible because you have expanded volumes"
From what I've been told by someone who looked at the logs. It seems that it tried to change the raid level when drive two finished syncing. But it never did and the volume is not expanded.
From what I've been told by someone who looked at the logs. It seems that it tried to change the raid level when drive two finished syncing. But it never did and the volume is not expanded.
Yeah I'll say that's not true. My old 428 had eight 6TB drives and when I went to all 10TB drives it flipped from raid5 to raid6 on me and I got stuck having to live with that for a few years.
She had eight 10TB drives in it and now all eight 16TB drives in volume 1
I'd be willing to take a look at the log zip - if she is ok with that, you should send them with a PM (private message). You do that using the envelope link in the upper right of the forum page.
The logs would need to be put into cloud storage (dropbox, google drive, etc), with a download link in the PM.
There is some privacy leakage - if email alerts are configured, I'd see the email address, for example.
I'm not sure what you mean by "the volume is not expanded". Do you mean that the volume didn't expand at all, that the volume didn't expand to the additional space on the last drive, or what? The NAS certainly believes your volume is expanded, and I suspect it did expand when the first few drives were swapped with XRAID enabled.. By "the last drive", do you mean the last of all 8, or some number less? If something less, that's the expected behavior of FlexRAID.
As for that message, unfortunately, you gave bad advice. What she is seeing is simply "the way it works". It wasn't always the case, but it's been the case since at least 6.9.something when I ran into this same situation. If you have a multi-level FlexRAID volume, switching (back) to XRAID is not allowed. I don't understand why this is the case since, AFAIK, all FlexRAID expansion is exactly as XRAID would have done it, there are just fewer options. It's clearly not a bug since the message correctly tells you what's going on, so programmer laziness seems most likely. While I also had an EDA500 at the time I discovered this, I don't believe that's a part of the issue. I recall I ran some experiments proving otherwise.
If it really didn't expand (completely) and all drives were replaced, then perhaps @StephenB can help with that.
Unless somebody at Netgear who is familiar with what the OS is looking at to prevent the switch and how to work around it gives some advice here (and they didn't a few years ago when I first reported this behavior), her only solution to get back to XRAID is to destroy and re-create the volume. There may be no need to do that now, but it will be necessary if she ever wants to increase the size of less than the full complement of drives. That she has an EDA500 may create some issues even when doing that, as the NAS may want to convert the EDA volume to the primary volume (which is a really bad idea). When I took the plunge and did this, I first exported the EDA volume, then destroyed and re-created the main volume, then imported the EDA.
One aspect here - the system has been expanded several times. The original disks were 4 TB, with later expansions to 6 TB, 10 TB, and now 16 TB. As a result, the system has 4 different RAID groups in the data volume - md124, md125, md126, and md127. These are all concatenated together to form the data volume. When you replace a disk, all of these groups need to be resynced.
Physically, each of these groups has a corresponding partition on each disk. Looking at disks 1-5 (sda-sde), these partitions look like this:
Since the system is running FlexRAID (RAID-5), all the RAID groups need to include all the disks in order for the system to expand.
There's not enough info in the log zip to see why this partition wasn't created, and then added to md127. But the disk does appear healthy.
I suggest removing the disk, and connecting it to a PC. From there, you can either unformat the disk (deleting all the "volumes" on it using the Windows disk manager), or just format it. Then you can add it back to the NAS. If you choose to reformat it, then you will then have to format it again in the NAS before the system will add it back to the data volume.
Hmm, that only accounts for 6 of the 8 drives, so something more is wrong. Are you sure it's 8 drives and was 8 before swapping any? If it were 8, it should already have been RAID6 under XRAID.
I think I can understand why the last drive (which from the information given should be the 8th, not the 6th) didn't get the last partition and didn't get added to the RAID. At that point, the unit was in FlexRAID. And FlexRAID expects to operate on all drives at the same time, not incrementally as with XRAID. So, it didn't know what to do with that lone drive. But all but that last one should have been added to the volume as the drives were replaced in XRAID. The information @StephenB posted seems to indicate that they were. But maybe the OS is confused since that last expansion didn't take place.
As @StephenB said, the volume is expanded, it just didn't (fully) expand this time. So the message that you can't go from FlexRAID to XRAID with an expanded volume is appropriate (to the extent that's a limitation of the OS).
So did the volume size increase with the second through fifth drive swap while the unit was still in XRAID? The data @StephenB posted seems to indicate it did, and that would be expected. Unfortunatly, the inability to switch back to XRAID is completely normal, and I suspect the fact that the last drive isn't expand is also directly related to that.
If it did not expand at all, I cannot explain why. There may be a complex way to get it to do so by deleting all the "unused" partitions and the letting FlexRAID expand to all of them, but I've never tried that and destroying and re-creating may be best, especially since it can restore XRAID mode.
OK larger drive with partitions wiped and allowed to re-sync did not work.
still can't turn on xraid
I think she is stuck w/o XRAID unless she does a factory default.
Can you send a fresh set of logs? Please don't redact anything, just provide the full log zip you download.
Also, syncing a 16 TB drive will take a while, so it would be good to leave it in the NAS, just in case it is still syncing (but not showing up in the web ui for some reason).
If the volume really did never expand (which disagrees with the information @StephenB posted from the log), that might work. If it did, the expansion would occur when the last one was inserted. If the volume did partly expand with the first 5 drives, removal of the second one would kill the volume.
(which disagrees with the information @StephenB posted from the log)
From the 28 May log, the new RAID group was created, but that the final disk wasn't added. BTRFS also wasn't expanded to use the new RAID group.
I expected that wiping disk 6 and trying again would work - obviously it didn't. Note that an important step was to delete all partitions (or just reformat in windows).
Hopefully fresh logs would tell us more.
Does she have experience with the linux command line?
From the 28 May log, the new RAID group was created, but that the final disk wasn't added. BTRFS also wasn't expanded to use the new RAID group.
OK, I missed out on that last part, and it's the one thing I can't understand at all (assuming I really do understand why the last wasn't added). As each drive was added and the unit was still in XRAID mode, the additional RAID should have been added when the second drive was inserted and then expanded with drives 3-5.
I'm not surprised that didn't work, and it adds to my belief that it can't expand because the first 5 drives already have the last partition and they are a part of an existing RAID, even though it was not added to the volume. I suspect that FlexRAID expects to operate on all the drives only when the last is inserted, creating the partitions and RAIDing them all then. The existing RAID (and maybe partitions) are preventing that.
Since md127 was never added to the volume, deleting it might do the trick. Or, you may need to delete the last partition on the first 5 drives, too. That's a big may, so the best way may be the manual expansion I pointed to above. There is a ReadyNAS-specifc command (volume_util) that might also help, but I'm in uncharted territory with it, relying only on it's --help. But before you try any of this, it would be best if the backup is up to date in case something goes sideways.
One confounding issue for the manual expansion is that I believe you are going to need to install a partitioning utility (I suggest parted) that can replicate the non-aligned partitioning on drives 1-5 on drive 6, so you have to deal with the Debian Jessie repository being archived. I've read that perhaps using dos mode of the already-installed fdisk may also allow it, but I've never tried.
But this still just completes the expansion. All of my experiments and real use say there is simply no way to go back to XRAID mode no matter what you do short of destroying and re-creating the volume.
Thanks for the fresh logs. It is clear that the resync didn't result in adding disk 6 to md127.
Though I cannot tell if it created the partition, since she replaced the WDC disk with an HGST 10 TB model before she got the logs.
May 30 16:10:05 Datastore2 readynasd[3766]: Disk Model:WDC WUH721816ALE6L4 Serial:2BKLVLET was removed from Channel 6 of the head unit.
May 30 16:10:10 Datastore2 readynasd[3766]: Volume data health changed from Redundant to Degraded.
May 30 16:16:19 Datastore2 readynasd[3766]: Disk Model:WDC WUH721816ALE6L4 Serial:2BKLVLET was added to Channel 6 of the head unit.
May 30 16:16:46 Datastore2 readynasd[3766]: Resyncing started for Volume data.
May 31 01:00:47 Datastore2 readynasd[3766]: Volume data is Degraded.
Jun 01 01:00:23 Datastore2 readynasd[3766]: Volume data is Degraded.
Jun 01 04:08:20 Datastore2 readynasd[3766]: Volume data is resynced.
Jun 01 04:08:20 Datastore2 readynasd[3766]: Volume data health changed from Degraded to Redundant.
Jun 01 04:08:23 Datastore2 readynasd[3766]: Disk in channel 6 (Internal) changed state from RESYNC to ONLINE.
Jun 01 15:45:34 Datastore2 readynasd[3766]: Disk Model:WDC WUH721816ALE6L4 Serial:2BKLVLET was removed from Channel 6 of the head unit.
Jun 01 15:45:40 Datastore2 readynasd[3766]: Volume data health changed from Redundant to Degraded.
Jun 01 15:47:05 Datastore2 readynasd[3766]: Disk Model:HGST HUH721010ALE600 Serial:7JJZZSMG was added to Channel 6 of the head unit.
Jun 01 15:47:13 Datastore2 readynasd[3766]: Resyncing started for Volume data.
As a result, the mdstat info is just showing the resync of the HGST drive (still underway when the logs were taken), and the partition/smart info for the WDC drive isn't there anymore.
I don't have any good ideas on how to kick-start the expansion at this point.
From a practical point of view, the options are
reinsert the WDC drive, and wait for resync. Then create the missing partition from ssh, and then add it to md127 (also using ssh). This requires some linux skills, and she'd still likely be locked into FlexRAID.
Start over, with a fresh volume, and rebuilding the NAS. Painful, and time consuming - but doesn't require use of ssh. The result would be a completely clean volume, with XRAID.
We can provide more guidance on (2) if she needs it. We'd need to know if she has a backup of the files - and if not, if she has another 6-8 bay ReadyNAS.
An FYI for future reference: The conversion from single-redundancy (RAID-5) to dual-redundancy (RAID-6) only happens when you install a 7th drive to a desktop system. So it is not a concern for a 6-bay ReadyNAS like the RN316.
So she decided to just wipe her readynas and start all over.
She didn't want to drag things out any longer. She's also got a big 24 bay dell server that she's going to get setup with Windows server and build that out with ten 8TB drives and six 10TB drives.
This is on top of the 516 with EDA500 which is her off sight cold backup. Two 314's she has laying around and a 628 that's her main NAS
She's got the 516 rebuilt and firmware updated, with eleven 16TB drives syncing now. Backup jobs from all the other NAS's will be starting up on the weekend
