Reply
Highlighted
Aspirant

Recovering data - Display code

In the process of doing a volume expansion, replacing my 3TB drives w/8TB drives by pulling them 1 by 1, letting it resync and doing another. First drive swap went without issue - waited a few days and did the 2nd one last night at midnight as it takes easily 6-8 hours to resync. Before I went to bed, the display said Resync Data and percentage. I wake up this morning, i cannot log into the admin console, I cannot access any of the SMB shares and I also cannot reset the password. The display says Recovering Data, a percentage. Once that was done, it then switched to Resync Data and has been doing there ever since, been about 3 hours now and its almost complete. 

 

Has anyone run into this? Any idea if the data is gone? Why i can't even log into the admin console or reset the password using the reset tool? I'm at work right now so I'll probably have to reset the password using the pin hole in the back but this is just so weird to me. 

Model: RN51663E|ReadyNAS 516 6-Bay 6x3TB Enterprise Drive
Message 1 of 7
Highlighted
Guru

Re: Recovering data - Display code

What firmware are you running?

 

With the second drive there are two steps.  One is recovering the original 3 TB of data blocks that were on the original disk.  The second is to expand the RAID array (in your case adding 5 TB more space).  I think those two steps will give different messages on the LCD display.

 

FWIW, the resync time will increase as you continue to add more disks.  Every resync will require either reading or writing every block in the data volume - which of course is growing as you continue the process.  6x8TB will take about 3x longer than the initial 6x3TB.

 

Rebuilding the RAID does take quite a bit of work, but it shouldn't lock up an RN516.  I think your data is likely ok, but I recommend that you update (or create) a backup before you insert the next disk.  I'd also download the log zip file, and look at the SMART information in disk_info.log.  Also look in kernel.log for disk-related errors.  If one of the disks is generating errors, make sure you replace that one next.

 

 

Message 2 of 7
Highlighted
Aspirant

Re: Recovering data - Display code

Firmware v6.10.1.

 

So the first drive being replaced was basically the parity drive swap. Every drive from now on out is like replacing a data drive and rebuilding the parity? If im understanding that correctly. Stil very concerning on why I cannot log into the admin portal..hopefully that changes once it fully resyncs

Message 3 of 7
Highlighted
Guru

Re: Recovering data - Display code


@syphonct wrote:

So the first drive being replaced was basically the parity drive swap. Every drive from now on out is like replacing a data drive and rebuilding the parity? If im understanding that correctly. 


There is no parity drive with RAID-5 - data and parity blocks are spread over all the drives.

 

If you originally started with 6x3TB, then you have one RAID group.  Replacing the first drive just reconstructs the 3 TB partition from the remaining drives.  The remaining 5 TB is ignored.

 

When you replaced the second drive, a second RAID group is created that fills the remaining space - 2x5TB.  The 3 TB parition on the second drive needs to be reconstructed.  Then the new RAID-1 group is constructed and both groups are concatenated into one volume.

 

When you do the third drive, the second RAID group is converted to RAID-5 and expanded.  The remaining three drives will also expand (which doesn't change the RAID mode, but it does require that all the data blocks be moved around, and the parity blocks recomputed).

 

When done, you'll have the original 6x3TB RAID-5 group and an 6x5TB RAID group.

 


@syphonct wrote:

Still very concerning on why I cannot log into the admin portal..hopefully that changes once it fully resyncs


Definitely it is concerning.  The admin web ui does run at a low priority.  But the RN516 has a fast processor, so there should be enough reserves to all you to log in.  I expanded my RN526x a couple months ago, and I was able to log into the web UI during the expansion.

 

One possibility is that one of the disks is struggling - which is why I suggested checking the disk health when it completes.  You might also enable ssh (or if it is enabled, see if you can log in with ssh).  Disk failures during resync often do result in data loss, so we do recommend making sure you have an up to date backup before you begin - especially when you are replacing all the disks.

 

What disk drives are you using?

 

Message 4 of 7
Highlighted
Aspirant

Re: Recovering data - Display code

So little update from last night. After it the data recovery and resync I was forced to do an OS reinstall which was pretty straight forward. It was the only way I was able to get back into the admin console. After installing the 2nd drive my free space increased, all my data is still there and the drives are reporting as green health with no issues. Absolutely no idea why I got locked out completely but lets hope for the best going forward. I was able to stay logged in during the first drive swap and also during the beginning of the second one. Since I went to bed shortly after doing the 2nd drive swap I dont have any idea of when it kicked me out of it. Going to give it a few more days before I swap another drive. 

 

For reference I'm using Seagate 8TB IronWolf drives from the list of approved drives. (ST8000VN0022) to be exact.

Message 5 of 7
Highlighted
Master

Re: Recovering data - Display code

I have had a couple similar lock-outs on my 516 when doing any kind of maintenance, especially on the EDA500.  I've come to be in the habit of leaving an SSH shell open when I do them, and I can see that the readynasd process hits 100% CPU usage, and then I am locked out of everything but SSH (and then, only because it is already open). Ultimately, it does normally clear up when the maintenance task completes.  But without the SSH shell open, you have no way to know when that will be.  I'm not sure why you needed to do an OS re-install to fix it.

 

This issue has been reported by others as well, but Netgear thus far has not responded.  I assume they are unable to re-create it, so can't fix what they can't see. 

Message 6 of 7
Highlighted
Aspirant

Re: Recovering data - Display code

very intesting, i had heard of doing the SSH route but who knows if that will lock me out as well. the OS reinstall takes 30 seconds so seems like a solid backup plan just in case. I'll make sure to have SSH open next time I pull a drive and see what happens, thanks for the feedback.

 

Message 7 of 7
Top Contributors
Discussion stats
  • 6 replies
  • 545 views
  • 0 kudos
  • 3 in conversation
Announcements