× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

magwems
Tutor

Remove inactive volumes to use the disk. Disk #1,2,3,4.

Dear Netgear,

 

I'm getting the following error message on my NAS since yesterday evening:

Remove inactive volumes to use the disk. Disk #1,2,3,4.

See attached screenshot, all 4 drives have status online but red, I don't believe the second RAID configuration existed before this error.

 

The NAS has been on the same version for quite a while (New update available today, but hoping to apply update once above issue is fixed only!) Currently on 6.10.4 Hotfix 1, offered to deploy 6.10.5re

 

I believe my 6 y old has 'opened' the NAS recently, removed the disks, and put them back inside the NAS, but I don't expect he would have put them in the appropriate slot afterwards, Can this be related? How can I figure out the original position of the drives? Or does the position not matter? 

 

The device has 'green' status, but all drives together and independently have a red status in the Volumes page,

I have tried to put each disk on it's own into bay 1, and rebooted the device, every time the volume has red status.

 

I don't expect all 4 disk have broken at the same time, but you never know...

 

Please let me know how to fix this error?

 

Thanks for your assistance!

 

Magwems

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 1 of 15

Accepted Solutions
StephenB
Guru

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Just to clarify - this is a user community, so you aren't engaged with Netgear.  Netgear does moderate, but doesn't respond very often.

 


@magwems wrote:

 

See attached screenshot, all 4 drives have status online but red, I don't believe the second RAID configuration existed before this error.

 


It didn't.  

 

The red status is because the NAS cannot mount your RAID array.  There are several possible causes - one common one is that there were some cached writes that didn't get written to the disks.  That will create an out-of-sync array.  It's possible that is linked to your 6 year old removing the disks, but there are other possible causes.

 


@magwems wrote:

How can I figure out the original position of the drives? Or does the position not matter? 

 


The NAS will boot with the drives out of order.  Keeping them in order makes some troubleshooting easier, but it isn't absolutely essential.  It is possible to figure out the original drive order from the full log zip file (which can be downloaded from the logs page).  But you likely will need some help in figuring that out.

 


@magwems wrote:

 

I have tried to put each disk on it's own into bay 1, and rebooted the device, every time the volume has red status.

That's a bit risky, and it wouldn't give much information.  XRAID has single redundancy - so you need at least three working disks to mount the array.  Although all the disks do seem to be working, that's not the only thing that's needed.

 


@magwems wrote:

 

Please let me know how to fix this error?

 


Do you have a backup of the data?  If you do, then one option is to do a factory default, reconfigure the NAS, and restore the data from the backup.

 

If not, you should put a backup plan in place for the NAS after you get this sorted out.  RAID isn't enough to keep your data safe.

 

Assuming no backup:  

 

Do you have any experience with linux command line interface ("ssh")?  If not, have you ever used the PC command line interface (CMD)?  If yes, then it is possible that you can get this resolved without paid tech support.  There are a few folks here who can look through your logs, and give you some guidance on the specific commands.

 

If you don't have that experience, then this really isn't the time to get it.  You'd be better off using either paid Netgear support, or use RAID recovery software like ReclaiMe.  Unfortunately, both are expensive.

 

Netgear's terms are here: https://kb.netgear.com/69/ReadyNAS-Data-Recovery-Diagnostics-Scope-of-Service  My guess is that the initial assessment would be sufficient (but that is just a guess).

 

RAID recovery software that supports BTRFS (like ReclaiMe) is also pricey: ~$200 for the version you need, plus you likely need to purchase USB docks that will let you connect the disks to a Windows PC (~ $100-$200).

 

Let us know what path you'd like to take.

 

 

 

View solution in original post

Message 2 of 15

All Replies
StephenB
Guru

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Just to clarify - this is a user community, so you aren't engaged with Netgear.  Netgear does moderate, but doesn't respond very often.

 


@magwems wrote:

 

See attached screenshot, all 4 drives have status online but red, I don't believe the second RAID configuration existed before this error.

 


It didn't.  

 

The red status is because the NAS cannot mount your RAID array.  There are several possible causes - one common one is that there were some cached writes that didn't get written to the disks.  That will create an out-of-sync array.  It's possible that is linked to your 6 year old removing the disks, but there are other possible causes.

 


@magwems wrote:

How can I figure out the original position of the drives? Or does the position not matter? 

 


The NAS will boot with the drives out of order.  Keeping them in order makes some troubleshooting easier, but it isn't absolutely essential.  It is possible to figure out the original drive order from the full log zip file (which can be downloaded from the logs page).  But you likely will need some help in figuring that out.

 


@magwems wrote:

 

I have tried to put each disk on it's own into bay 1, and rebooted the device, every time the volume has red status.

That's a bit risky, and it wouldn't give much information.  XRAID has single redundancy - so you need at least three working disks to mount the array.  Although all the disks do seem to be working, that's not the only thing that's needed.

 


@magwems wrote:

 

Please let me know how to fix this error?

 


Do you have a backup of the data?  If you do, then one option is to do a factory default, reconfigure the NAS, and restore the data from the backup.

 

If not, you should put a backup plan in place for the NAS after you get this sorted out.  RAID isn't enough to keep your data safe.

 

Assuming no backup:  

 

Do you have any experience with linux command line interface ("ssh")?  If not, have you ever used the PC command line interface (CMD)?  If yes, then it is possible that you can get this resolved without paid tech support.  There are a few folks here who can look through your logs, and give you some guidance on the specific commands.

 

If you don't have that experience, then this really isn't the time to get it.  You'd be better off using either paid Netgear support, or use RAID recovery software like ReclaiMe.  Unfortunately, both are expensive.

 

Netgear's terms are here: https://kb.netgear.com/69/ReadyNAS-Data-Recovery-Diagnostics-Scope-of-Service  My guess is that the initial assessment would be sufficient (but that is just a guess).

 

RAID recovery software that supports BTRFS (like ReclaiMe) is also pricey: ~$200 for the version you need, plus you likely need to purchase USB docks that will let you connect the disks to a Windows PC (~ $100-$200).

 

Let us know what path you'd like to take.

 

 

 

Message 2 of 15
magwems
Tutor

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

ouch looks harder than I hoped 😉

Yes, I know cmd, ssh, I'm a software developer, 

Most of the data on the nas I have backup, but family photos are there, so would really like to get it back, even including fee 😉

It's down for 24h only, so I'm not factory resetting just yet,

My 10y old tells me the 7y old played with the nas 3 weeks ago, so not sure it's his fault in the end...

 

I believe I can connect all drives to home pc without external boxes, but not keen if I can avoid, cause I will need system disk + 4 NAS disks and then some more for backup ...

 

happy to look into your option or netgear support (prefer paying netgear to fix than operating ReclaiMe myself, but I prefer operating reclaiMe than shipping the nas to netgear)

 

Thanks

Message 3 of 15
rn_enthusiast
Virtuoso

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Hi @magwems 

I can take a look at the logs for you to see what is going on exactly. When you access the NAS Web Interface, go to: System > Logs and here you will see a button called "Download Logs" on the right-hand side. Click this and it will download a zip file with all the NAS logs inside.


Take this zip file and upload to your Dropbox, Google Drive or similar and then make a link where I can download the log zip file. PM me this link and then I will have a look to see what is going on.

 

Cheers

Message 4 of 15
Sandshark
Sensei

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

There is not, BTW, an actual "second RAID".  The NAS is seeing two parts of the Main array that it cannot mount as one.  It then shows both broken pieces.  But since there can't be two volumes of the same name, it uses the creation name of the RAID -- Main-0 for the first (or only) RAID in the Main volume -- to display one.  Both are broken beyond autromatic repair, as it should start a re-sync if one had enough working parts.  But you absolutely don't want to delete either one -- they both contain parts of your data volume, and deleting one will make recovery impossible.

Message 5 of 15
rn_enthusiast
Virtuoso

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Hi @magwems 

 

Thanks for the logs.

 

Your have one disk (Disk bay 1) that is dodgy with 76 ATA errors. However, the other disks appear healthy. No sign in dmesg logs that the NAS is having issues talking to the disks.

Device:             sdd
Controller:         0
Channel:            0  <<== Disk bay 1
Model:              ST3000VN000-1HJ166
Serial:             W7307G2T
Firmware:           SC60
Class:              SATA
RPM:                5900
Sectors:            5860533168
Pool:               Main-0
PoolType:           RAID 5
PoolState:          5
PoolHostId:         2fe63a64
Health data 
  ATA Error Count:                76
  Reallocated Sectors:            0
  Reallocation Events:            0
  Spin Retry Count:               0
  End-to-End Errors:              0
  Command Timeouts:               0
  Current Pending Sector Count:   0
  Uncorrectable Sector Count:     0
  Temperature:                    32
  Start/Stop Count:               44250
  Power-On Hours:                 41098
  Power Cycle Count:              62
  Load Cycle Count:               4425

While that disk do need to be replaced at some point in the future, that is not the cause of the issue. Instead it looks like all 4 disks dropped out of the chassis and reconnected later on. If you did not remove any disks, then the disks must have lost connectivity with the SATA bay in the chassis. It starts with disk 4, getting "disconnected" and then reconnects moments later. This prompts the NAS do a RAID resync.

[21/07/22 15:47:12 AEST] warning:volume:LOGMSG_HEALTH_VOLUME Volume Main health changed from Redundant to Degraded.
[21/07/22 15:47:17 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was removed from Channel 4 of the head unit.
[21/07/22 15:48:17 AEST] notice:disk:LOGMSG_ADD_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was added to Channel 4 of the head unit.
[21/07/22 15:48:28 AEST] notice:volume:LOGMSG_RESILVERSTARTED_VOLUME Resyncing started for Volume Main.

But just a minute later disk 2 and 3 also disconnects along with disk 4 again disconnecting. At this point the RAID stops working and the RAID is declared "dead".

[21/07/22 15:49:31 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: ST3000VN000-1HJ166 Serial:W7307G2T was removed from Channel 2 of the head unit.
[21/07/22 15:49:33 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: ST3000VN000-1HJ166 Serial:W7306SYV was removed from Channel 3 of the head unit.
[21/07/22 15:49:35 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was removed from Channel 4 of the head unit.
[21/07/22 15:49:41 AEST] warning:volume:LOGMSG_HEALTH_VOLUME Volume Main health changed from Degraded to Dead.

And later we see further evidence of disks disconnecting and reconnecting.

[21/07/22 15:50:44 AEST] notice:disk:LOGMSG_ADD_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was added to Channel 4 of the head unit.
[21/07/24 12:02:00 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was removed from Channel 4 of the head unit.
[21/07/24 12:02:39 AEST] notice:disk:LOGMSG_ADD_DISK Disk Model: WDC WD60EFAX-68SHWN0 Serial:WD-WX21DC91CSVU was added to Channel 4 of the head unit.
[21/07/24 12:03:21 AEST] warning:disk:LOGMSG_DELETE_DISK Disk Model: ST3000VN000-1HJ166 Serial:W7306SYV was removed from Channel 3 of the head unit.
[21/07/24 19:07:30 AEST] notice:disk:LOGMSG_ADD_DISK Disk Model: ST3000VN000-1HJ166 Serial:W7307G2T was added to Channel 1 of the head unit.
[21/07/24 19:07:50 AEST] notice:disk:LOGMSG_ADD_DISK Disk Model: ST3000VN000-1HJ166 Serial:W7307TF7 was added to Channel 3 of the head unit.

So, the question is. Did you remove and re-insert disks like this? If you didn't, then you probably have a faulty chassis or a faulty power supply. I tried to find evidence of PSU issues but I don't see any thus I am thinking that a bad chassis is likely the cause here.
Saving the RAID and the data should be a very straight forward task. The RAID will be marked as "dirty" as this point due it being out of sync but that can manually be reassembled. It should not be a problem at all, but safest to let Netgear Level 3 do this, though that might require a "per incident" case fee which shouldn't be too expensive. The question is whether you need a new chassis (given you didn't manually pull out those disks). You can show Netgear support the logs and ask what they reckon.

 

Cheers

Message 6 of 15
Sandshark
Sensei

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Assuming your child did the drive removal/insertion with the power on, that is almost certainly the issue.  If you have an old copy of the log .zip, you can find what order the drives belong in in disk_info.log.  If it's not too old, take a look at it and see if the ATA errors have been there a while, as those can also be caused by messing with the drives (esp. if not fully seating them).  Just getting them back into the right order won't fix the problem, but it would be good to have them in the right order before trying to manually mount the volume.  I'm not the one to give you that set of commands, but @rn_enthusiast or @StephenB can.  You'll still need a place to offload the data since the volume isn't going to mount clean or be cleanable, so you'll eventually be deleting and re-creating the volume.  And once you have that place to put the files, I recommend you utilize it as the destination for a permanent scheduled backup.  Of course, you may want to put the backup somewhere other than on the NAS via USB -- somewhere where a child can't get to it.  It'll take longer for the backup to complete over the network, but a backup that's as vulnerable as the primary has more limited usefulness, especially in your kind of situation.

 

If it looks like the ATA errors are recent, then I'd ignore them for now and assume it was the removal/insertion that created them.  Even if not, it probably won't matter if you ignore them until you've re-created the volume and find out if that causes more.  ATA errors can be caused by a lot of things, including things not in the drive itself.  If you do get more, especially if during the volume re-creation (which may be noticeable by it taking way too long), it could be the chassis instead of the drive, so come back and give details for advice on handling them.

 

The caddies do have a slide lock, were they engaged?  Of course, a child may figure them out.  A zip tie around the unit may look funky, but should keep a child from getting to the drives.

Message 7 of 15
rn_enthusiast
Virtuoso

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

This should "just" be an excercise in assembling the raid again. After that, all should be good (given the child pulled the disks and it isn't the chassis that is faulty). I don't see why he would need to offload the data and recreate the volume.

Message 8 of 15
rn_enthusiast
Virtuoso

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

And I missed the obvious statement... I believe my 6 y old has 'opened' the NAS recently, removed the disks, and put them back inside the NAS

😄

 

So yeah, then the logs make sense. Not a faulty chassis, but a curious child 🙂

A manual assembly of the raid is all that is needed here and you will be back in business.

Netgear support (per incident fee) would be my advise for this.

Message 9 of 15
Sandshark
Sensei

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

If Netgear does it for you, then I'm sure they'll let you know if the volume is back to 100& health.  If I were doing it myself, I wouldn't risk that I missed something and it would be in bad shape again down the road.  I'd re-create what I would then know is a fully healthy volume.

 

If you don't end up re-creating the volume, then running the disk test and a scrub will help you figure out if the drive with the ATA errors really needs to be replaced.  If it makes it through those without any additional errors, I think you can to continue using it without concern, though you should also probably check in on it occasionally since the SMART limits Netgear assigns to email and log alerts are a bit large.

Message 10 of 15
magwems
Tutor

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Thanks everyone for your help and detailed analysis of the log,

I will contact the support to see if/how to rebuild the disks in order and also bought 2 new NAS for backup so next time, I can factory reset and restore from the second NAS.

 

PS: Love the Zip tie idea!

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 11 of 15
magwems
Tutor

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Also, do you know if Netgear gave up on home NAS? Can't seem to find any of these products available....

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 12 of 15
Sandshark
Sensei

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Some have reported that Netgear support has said that they have, but Netgear has made no such announcement.  But stock is hard to find.  For a while, the rack-mount units remained available, but they have also dried up.

 

COVID certainly had a double-whammy on the NAS market.  There were a lot of companies suddenly in need of a work-from-home solution, where a NAS with remote access can be a big part, so that lowered stock.  Then there was a downturn in parts availability, manufacturing capability, and transportation (both because of the reduced work force and because priorities often went elsewhere) that made recovey difficult.  And some companies were just better prepared for something like this than others.

 

So is this just a result of COVID, or is something else going on?  Anyone who knows for sure isn't saying.

Message 13 of 15
magwems
Tutor

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Netgear support did the magic remotely, resynced the drives, all back to normal now!

 

Happy!

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 14 of 15
rn_enthusiast
Virtuoso

Re: Remove inactive volumes to use the disk. Disk #1,2,3,4.

Good to hear 🙂

 

Message 15 of 15
Top Contributors
Discussion stats
  • 14 replies
  • 5221 views
  • 8 kudos
  • 4 in conversation
Announcements