× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Numb3r6
Apprentice

ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

This is a ReadyNAS Duo v1


-Bootup via Power Timer, as usual, no errors reported or emailed to me
-First use of NAS was to stream music to a mobile device. Done it many times, no issues. This time though the streamer was clearly struggling finally reporting it could not connect to device (i.e. the NAS)
-Visually checking it the blue LED was flashing at a steady rate but the unit was not finishing boot.
-No errors previously noted
-Held the power button down to force shutdown, which it did, and I removed power briefly
-Restarted, same
-Repeated this and eventually the blue LED went steady, but system still inaccessible
-Looking at the green LEDs, the top one (disk activity) was flashing rapidly and the 2 beneath it (disk 1 and 2) were steady.
-The IP address was then visible on the network, but could not connect to Frontview via a browser
-Left it a while longer and could access content via network, as well as using Frontview
-Checking Frontview logs etc, there's nothing worrying about disk health
-Only clue is in logs, just saying "Volume scan found and corrected errors"

 

Now I can access the file system I'm doing a (slightly overdue) backup to a standalone disk. It'll take a while....

 

With that in mind, I am puzzled as to what has happened. I'm pretty sure the timing coincided with my use of Stratospherix MusicStreamer.

 

So are there any known triggers that could provoke the symptoms I experienced, or is there anywhere else I can look at within the NAS for more detailed error info?

 

Storage is at 76% BTW
RAIDar shows no issues

 

thanks

Dave

Model: ReadyNAS RND2000|ReadyNAS Duo Chassis only
Message 1 of 18

Accepted Solutions
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

I wouldn't be concerned about lp stat events.

 

A volume scan is a reasonable idea, though I'd suggest making a backup (or updating it if needed) first.

View solution in original post

Message 7 of 18

All Replies
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Did you check system.log and kernel.log for details on what the volume scan fixed?  Also, did you look for any disk errors in those logs?

Message 2 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Thanks for reply StephenB. OK I'd been looking in the file system rather than using the Frontview option to download them. I hadn't realised that option would reveal more detail than is shown in the Frontview summary. That said I don't know what I would be looking for that relates to the event that took place. System has not been rebooted since yesterday, as I am offloading content to another drive. Mind you it has shown no signs of misbehaviour nor anything bad in the Frontview health summary since.

 

Whilst my system is far from new, I have replaced disks for higher capacity in its life, and regularly give it a blowthrough to shift dust. Mind you heat has never been an issue, and the only times I've had problems before (maybe twice, before I got a UPS) was unplanned power interruption which it recovered from anyway, with no signs afterwards.

 

Message 3 of 18
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

It's hard to say if the file system corruption led to the streaming problem, or if the forced shutdown did.

 

The forced shutdown would trigger a volume scan on startup, and possibly a RAID resync.  There could have been some lost (cached) writes, which would account for some filesystem corruption.

 

Generally - look in system.log and kernel.log for any evidence of system crashes or disk-related errors between the time you had the streaming problem and the time the system was fully recovered.

 

Also look in disk_usage.log at the fullness of the 2 GB OS partition (/dev/hdc1 if you are running XRAID).  Normally the OS partition on the Suo is about 25% full.  

 

disk_smart.log is a good place to look for information on disk health - you can also see the SMART+ stats in frontview (though they will come up blank if you use Chome).

 

 

 

Message 4 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Thanks for the extra information StephenB

Here's part of the disk_usage.log:

Filesystem Size Used Avail Use% Mounted on
/dev/hdc1 2.0G 781M 1.2G 40% /


and regarding disk_smart.log, the most recent instance is dated a few days before the event, there's a lot in there that does not grab my attention, mostly as I'm not familiar with entries that could be a concern. I can see however references to SMART health tests stating
"Passed" and "no errors logged"

I guess it'd be a good idea to look at the same logs for newer entries once I get round to restarting the system. As I said, I was attempting to stream audio from a device on wifi, which I've done many times before, but this time the software didn't begin streaming as normal and showed a generic error about the device not being available, and it was then I noticed the reboot taking place, and taking longer than usual.

When I had a couple of power interrupts ages ago, something more meaningful showed even in the Frontview Log summary, but this time it merely says "Volume scan found and corrected errors" (severity shown as green) which must have been created on reboot. There is nothing in the Frontview logs immediately before the episode, the sequence showing:

1 System is up (from scheduled start)
2 System powering off...
3 System powering off...
4 Volume scan found and corrected errors.
5 System is up.

I'm assuming entries 2&3 result from my mashing of the power button to restart it (after not getting any error warnings via email - I've also just tested the alert function, working ok)

What I have noticed in looking at the content of SMART+ via Frontview is that for Disk1, the Lp stat events count seems higher than I recall. Disk 1=156 whereas Disk2=4. BTW both disks were installed at the same time. Not understanding that I looked it up and the suggestion is that relates to IO taking longer than expected (a symptom of the failed streaming "calls"?)


For ref the thread I'm looking at is
https://community.netgear.com/t5/Using-your-ReadyNAS-in-Business/Lp-stat-events-What-does-it-mean/td...

so following that I checked
Reallocated sector count
Current Pending Sector
Offline uncorrectable
ATA errors

All of the above have a zero count, both disks

I'm still copying data to other drives, not the fastest connections, but whilst doing that there has been no untoward behaviour at all

It does seem highly coincidental that the system auto started without any errors and the issue started when the media streamer misbehaved. Is it possible this somehow upset the NAS, and without an error alert, my manual interruption recommenced any recovery that was ongoing? It'd be nice if I've dodged a bullet on this one....

Message 5 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

OK I just restarted the system and it's up as normal. Nothing untoward showing in Frontview at all.

Checking that Lp stat events counter again, both disks now show 6 and 5, hugely different from before.

 

Tempted to restart with a volume scan at next boot to see what happens....

 

Message 6 of 18
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

I wouldn't be concerned about lp stat events.

 

A volume scan is a reasonable idea, though I'd suggest making a backup (or updating it if needed) first.

Message 7 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Hi StephenB - following a couple of system reboots, the NAS appears to be back to its normal self. On the last one I opted to include a volume scan, which didn't take long really, and right now, all health indicators are good.


I'm going to do some data management anyways, and will be watching for any oddities during that - but right now I think the issue was catalysed by some glitch between the media streaming software on another device, upsetting the NAS, and with a lack of alerts being sent to me, perhaps some impatience on my part by interrupting a reboot, purely as I had no clue it may be mid-volume scan. Clearly if the system had issued an email to advise of this, I'd have just left it alone. Quite odd really but hoping it has dealt with it all now!

Many thanks for your guidance.

Message 8 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Things have moved on, in the midst of moving data around a couple of errors have been reported:

-A SATA reset has been performed on one or more disks....RAID resync recommended

-Reallocated sector count increased on disk2, previous count 0, current count 70

Going to order a couple of replacement disks....

Model: EDA500|ReadyNAS Expansion Chassis 5-Bay
Message 9 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Oops I should have added that looking via the SMART+ button, all still shows as green, despite the 2x yellow entries in the logs that I mentioned in previous post. Enough to be concerned about though I think.

Message 10 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

After completing the ongoing data shift, I did as the system suggested by selecting Resync. Within that it says the system should stay usable. Right now it pings, and via PC it tries to load Frontview, but there are no buttons showing on the GUI. Only indicators are ble LED, steady, and green LEDs x2 also steady. I can’t browse or connect to shares at all.

 

is this just a case of waiting and hoping now?

Message 11 of 18
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

I'd replace the drive with ~70 reallocated sectors.

Message 12 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Thanks StephenB, that's my thinking too. I have 2 replacements due tomorrow, same spec as before. For info I have been using WD Red 2TB WD20EFRX, which I know quite a few owners have used. They've been in the enclosure approx 38 months without a hint of a problem. I'm still puzzled as to why the SMART/Health indicators on Frontview  showed green until things seemed to escalate today.

The email alert it sent regarding "Reallocated sector count increase" happened during some data shifting that was already ongoing, a few hours after the SATA reset alert (also received during data management) - so rather than interrupt that task I let it finish then went for the recommended RAID volume resync.

 

That was initiated almost 4 hours ago. Apart from blue LED and disk1/2 green LEDs (all steady) there is no sign of activity at all.

What I am unsure of is if that is normal during a resync, is that something you can advise on?

I think my previous note that Frontview was trying to load in a browser was a mistake, and the partial page I was looking at was perhaps cached on the PC. If I use Browser/Frontview, RAIDar on PC, or other file utils and the little NAS applet on an iPad, nothing at all. The only clue it is not fully dead is a returned Ping.

Any thoughts on those symptoms would be welcomed just so I have a better grasp of events.

It was Disk2 with the increased reallocated sector count so I assume that's the right hand disk in the enclosure.

Should there be no progress by tomorrow AM, I'm thinking of the following steps:
-Do whatever is needed to power the NAS down, and remove Disk2
-Try powering up with just Disk1 in place and see if I can connect to shares/data (if so thinking that drive is good, just not redundant)
-If the above works, insert one of the awaited new drives into slot2, and let the NAS sort things. Undecided if I should do that hot or not.

At some point when I hope to have seen progress I may even hook the suspect drive up to an external adapter and see if I can use any software tools to investigate further. I may even hook the "good" (I hope) disk up too, just to see what I can learn from that.

Assuming I do see some progress with a new Disk2, I may then give it a couple of days and swap out Disk1 for a newer instance also.

One aspect I don't follow at all, regarding shares, users etc is where that information is held. Is it somehow held in the file structure, or is it stored some other way, protected unless a system reset is performed?

A lot of questions I know, and I hope you don't mind too much Smiley Wink


Message 13 of 18
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"


@Numb3r6 wrote:

I have been using WD Red 2TB WD20EFRX, which I know quite a few owners have used.

I have one installed in my Duo v1 (the other disk is an old Seagate).

 


@Numb3r6 wrote:

Apart from blue LED and disk1/2 green LEDs (all steady) there is no sign of activity at all.

What I am unsure of is if that is normal during a resync, is that something you can advise on?

It's been a while since I've done a resync in a v1 NAS, and I don't recall if there are blinking LEDs. Normally you do see progress in RAIDar or the web ui when you hover over the volume or disk icons.

 


@Numb3r6 wrote:


-Try powering up with just Disk1 in place and see if I can connect to shares/data (if so thinking that drive is good, just not redundant)

That is worth a try. You can also connect disk 1 to a Windows PC (USB is ok) and see if you can offload data with R-linux for Windows https://www.r-studio.com/free-linux-recovery/Download.shtml#win

 


@Numb3r6 wrote:

Undecided if I should do that hot or not.

I recommend hot insertion,


@Numb3r6 wrote:


One aspect I don't follow at all, regarding shares, users etc is where that information is held. Is it somehow held in the file structure, or is it stored some other way, protected unless a system reset is performed?


The disks have a 2 GB OS partition that holds Linux, the ReadyNAS application, and the system configuration.  This information is mirrored on all disks (two in your case), allowing the system to boot from any (healthy) disk.

 

Message 14 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Thanks once more StephenB.

 

With no front LED action, nor the ability to launch Frontview, I suspect it’s hung right now.

I might give it a little longer this evening, then pull Disk2 after a power down. Wondering if that disk has suddenly reached a stage where the sync cannot work, and the system has responded badly to that perhaps.

Message 15 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

This just in...its alive, well half of it says so. 

With no hints that anything was happening I managed to power down via front panel.

Pulled Disk2, restarted with an immediate change in LED behaviour.

Power LED, steady blinking, Disk activity LED fast flashing. Disk1 LED on, Disk2 LED slow flashing. 

After about 45 mins, system up and accessible.

 

Disk1 showing no reallocations.

 

Here’s hoping a new Disk2 moves things along tomorrow....Smiley Happy

 

Message 16 of 18
StephenB
Guru

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"


@Numb3r6 wrote:

This just in...its alive, well half of it says so. 

 

After about 45 mins, system up and accessible.

 


That's good news.  I suggest backing up the files before you insert the new disk.

 

Message 17 of 18
Numb3r6
Apprentice

Re: ReadyNAS Duo v1 boot fail into "Volume scan found and corrected errors"

Hopefully concluding this one now. Replacement disk synced up with no errors. No adverse log/SMART issues I can see.
Interestingly I hooked up the drive to a PC and tried the WD tools. It could see the drive, picked up on the allocation errors, but when I tried the repair option it was no go.

I've decided to leave the original Disk1 in place, if it's not broke etc. Also got a new spare 2TB WD Red, just in case.

Thanks for all your help StephenB

Message 18 of 18
Top Contributors
Discussion stats
  • 17 replies
  • 2831 views
  • 1 kudo
  • 2 in conversation
Announcements