Issue with hard drive upgrade and xraid

Hmm, that only accounts for 6 of the 8 drives, so something more is wrong.  Are you sure it's 8 drives and was 8 before swapping any?  If it were 8, it should already have been RAID6 under XRAID.

I think I can understand why the last drive (which from the information given should be the 8th, not the 6th) didn't get the last partition and didn't get added to the RAID.  At that point, the unit was in FlexRAID.  And FlexRAID expects to operate on all drives at the same time, not incrementally as with XRAID.  So, it didn't know what to do with that lone drive.  But all but that last one should have been added to the volume as the drives were replaced in XRAID.  The information StephenB posted seems to indicate that they were.  But maybe the OS is confused since that last expansion didn't take place.

As StephenB said, the volume is expanded, it just didn't (fully) expand this time.  So the message that you can't go from FlexRAID to XRAID with an expanded volume is appropriate (to the extent that's a limitation of the OS).

---

Sandshark wrote:

Hmm, that only accounts for 6 of the 8 drives, so something more is wrong. 

leo037:  The log zip you sent me says this is an RN316, not an RN428.  So only 6 internal disks. Plus 5 more in the EDA volume.

boot_info.log:

vendor:	NETGEAR
model:	ReadyNAS 316
serial:	
sku:	RN316
loader:	4.6.5
board_rev:	
system_rev:	
reason:	Normal

Something is odd here, as the serial, board_rev, and system_rev fields should be populated.

Yeah my fault on which model it is. She has at least four, I thought she was talking about another one. 

I can confirm it is is a 316

Yeah sorry it is actually a 316. 

Serial number I removed

Not sure why board and system rev aren't there

I don't recall where she got this one from. Could have been a craigslist or ebay find

So did the volume size increase with the second through fifth drive swap while the unit was still in XRAID?  The data StephenB posted seems to indicate it did, and that would be expected.  Unfortunatly, the inability to switch back to XRAID is completely normal, and I suspect the fact that the last drive isn't expand is also directly related to that.

If it did not expand at all, I cannot explain why.  There may be a complex way to get it to do so by deleting all the "unused" partitions and the letting FlexRAID expand to all of them, but I've never tried that and destroying and re-creating may be best, especially since it can restore XRAID mode.

There is a complex method of manually doing the expansion via SSH.  See How-to-do-incremental-vertical-expansion-in-FlexRAID-mode.  But the unit will remain unable to go back to XRAID.

---

Sandshark wrote:

So did the volume size increase with the second through fifth drive swap while the unit was still in XRAID? 

---

No.  The RAID group exists (with one missing disk), but BTRFS did not expand to use it.

I think unformating and then re-inserting disk 6 will likely fix it.

OK larger drive with partitions wiped and allowed to re-sync did not work. 

still can't turn on xraid

---

leo037 wrote:

OK larger drive with partitions wiped and allowed to re-sync did not work. 

still can't turn on xraid

---

I think she is stuck w/o XRAID unless she does a factory default.

Can you send a fresh set of logs?  Please don't redact anything, just provide the full log zip you download.

Also, syncing a 16 TB drive will take a while, so it would be good to leave it in the NAS, just in case it is still syncing (but not showing up in the web ui for some reason).

She asks about wiping the other drives one by one and having them re-sync one at a time? 

I don't think that would work though

If the volume really did never expand (which disagrees with the information StephenB posted from the log), that might work.  If it did, the expansion would occur when the last one was inserted.  If the volume did partly expand with the first 5 drives, removal of the second one would kill the volume.

What volume size is being reported?

---

Sandshark wrote:

(which disagrees with the information StephenB posted from the log)

---

From the 28 May log, the new RAID group was created, but that the final disk wasn't added.  BTRFS also wasn't expanded to use the new RAID group.

I expected that wiping disk 6 and trying again would work - obviously it didn't.  Note that an important step was to delete all partitions (or just reformat in windows). 

Hopefully fresh logs would tell us more.

Does she have experience with the linux command line?

Also, does she have a backup of the data?

---

StephenB wrote:

---

---

From the 28 May log, the new RAID group was created, but that the final disk wasn't added.  BTRFS also wasn't expanded to use the new RAID group.

OK, I missed out on that last part, and it's the one thing I can't understand at all (assuming I really do understand why the last wasn't added).  As each drive was added and the unit was still in XRAID mode, the additional RAID should have been added when the second drive was inserted and then expanded with drives 3-5.

I'm not surprised that didn't work, and it adds to my belief that it can't expand because the first 5 drives already have the last partition and they are a part of an existing RAID, even though it was not added to the volume.  I suspect that FlexRAID expects to operate on all the drives only when the last is inserted, creating the partitions and RAIDing them all then.  The existing RAID (and maybe partitions) are preventing that.

Since md127 was never added to the volume, deleting it might do the trick.  Or, you may need to delete the last partition on the first 5 drives, too.  That's a big may, so the best way may be the manual expansion I pointed to above.  There is a ReadyNAS-specifc command (volume_util)  that might also help, but I'm in uncharted territory with it, relying only on it's --help.  But before you try any of this, it would be best if the backup is up to date in case something goes sideways.

One confounding issue for the manual expansion is that I believe you are going to need to install a partitioning utility (I suggest parted) that can replicate the non-aligned partitioning on drives 1-5 on drive 6, so you have to deal with the Debian Jessie repository being archived.  I've read that perhaps using dos mode of the already-installed fdisk may also allow it, but I've never tried.

But this still just completes the expansion.  All of my experiments and real use say there is simply no way to go back to XRAID mode no matter what you do short of destroying and re-creating the volume.

leo037:

Thanks for the fresh logs.  It is clear that the resync didn't result in adding disk 6 to md127.

Though I cannot tell if it created the partition, since she replaced the WDC disk with an HGST 10 TB model before she got the logs.

May 30 16:10:05 Datastore2 readynasd[3766]: Disk Model:WDC  WUH721816ALE6L4 Serial:2BKLVLET was removed from Channel 6 of the head unit.
May 30 16:10:10 Datastore2 readynasd[3766]: Volume data health changed from Redundant to Degraded.
May 30 16:16:19 Datastore2 readynasd[3766]: Disk Model:WDC  WUH721816ALE6L4 Serial:2BKLVLET was added to Channel 6 of the head unit.
May 30 16:16:46 Datastore2 readynasd[3766]: Resyncing started for Volume data.
May 31 01:00:47 Datastore2 readynasd[3766]: Volume data is Degraded.
Jun 01 01:00:23 Datastore2 readynasd[3766]: Volume data is Degraded.
Jun 01 04:08:20 Datastore2 readynasd[3766]: Volume data is resynced.
Jun 01 04:08:20 Datastore2 readynasd[3766]: Volume data health changed from Degraded to Redundant.
Jun 01 04:08:23 Datastore2 readynasd[3766]: Disk in channel 6 (Internal) changed state from RESYNC to ONLINE.
Jun 01 15:45:34 Datastore2 readynasd[3766]: Disk Model:WDC  WUH721816ALE6L4 Serial:2BKLVLET was removed from Channel 6 of the head unit.
Jun 01 15:45:40 Datastore2 readynasd[3766]: Volume data health changed from Redundant to Degraded.
Jun 01 15:47:05 Datastore2 readynasd[3766]: Disk Model:HGST HUH721010ALE600 Serial:7JJZZSMG was added to Channel 6 of the head unit.
Jun 01 15:47:13 Datastore2 readynasd[3766]: Resyncing started for Volume data.

 As a result, the mdstat info is just showing the resync of the HGST drive (still underway when the logs were taken), and the partition/smart info for the WDC drive isn't there anymore.  

I don't have any good ideas on how to kick-start the expansion at this point.

From a practical point of view, the options are

1. reinsert the WDC drive, and wait for resync.  Then create the missing partition from ssh, and then add it to md127 (also using ssh).  This requires some linux skills, and she'd still likely be locked into FlexRAID.
2. Start over, with a fresh volume, and rebuilding the NAS.  Painful, and time consuming - but doesn't require use of ssh.  The result would be a completely clean volume, with XRAID.

We can provide more guidance on (2) if she needs it.  We'd need to know if she has a backup of the files - and if not, if she has another 6-8 bay ReadyNAS.  

An FYI for future reference:  The conversion from single-redundancy (RAID-5) to dual-redundancy (RAID-6) only happens when you install a 7th drive to a desktop system.  So it is not a concern for a 6-bay ReadyNAS like the RN316.

So she decided to just wipe her readynas and start all over. 

She didn't want to drag things out any longer. She's also got a big 24 bay dell server that she's going to get setup with Windows server and build that out with ten 8TB drives and six 10TB drives. 

This is on top of the 516 with EDA500 which is her off sight cold backup. Two 314's she has laying around and a 628 that's her main NAS

She's got the 516 rebuilt and firmware updated, with eleven 16TB drives syncing now. Backup jobs from all the other NAS's will be starting up on the weekend

Thank you all for your input