× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: RN 104 failed to start with "Boot from USB" - no FW update

berillio
Aspirant

RN 104 failed to start with "Boot from USB" - no FW update

RN104, 4x4TB, 6.2.5 Firmware

 

2020_03_15 - Sunday morning at 3am, the NAS disappeared from the network, as expected, as I have a weekly "reboot".

It should have come back online at 4am.

But when I tried to access it next day, it wasn't available, I checked it and found it stuck on the "Boot from USB" message.

 

Than message was unknown to me, apparently it comes after a failed Firmware upgrade.

That in itself is strange, Automatic upgrade were not enabled, the firmware in use is/was 6.2.5, which is quite old.

 

Some possibly useful infos about the USB ports on the back.

After Christmas I came back home with two USB drives (a 4TB and a 2TB, both un-powered drives) - the idea was to re-align the directory structure on both the NAS and those two HDs. They appeared as two separate "volumes" and they were perfectly accessible. They were attached at the back of the RN104 for "some time". I say "some" because I cannot remember for how long, but at least a month, so the RN104 must have rebooted on the weekly schedule three or four times.

Then one Sunday, it didn't, the NAS "hang" at restart.

I thought that maybe the USBs drives were taking too much juice and I removed them, and the NAS booted fine.

I think that I did not plug them back in for few weeks until I needed to look into one of them (the 4TB), and I plugged the 4TB in.

I think (not sure) that the NAS failed to start again at the following reboot - if so I would have unplugged the drive again and the NAS would have started as normal: but it might have been plugged in for more than one week, meaning that I am not sure if this was the first time the NAS attempted to reboot since then, or it might have started up another time successfully. Also note that I might have unplugged the lead from the disk, not the lead from the NAS, which is under the desk.

 

Neither of those drives is self-bootable, they are only (and the have only been) purely data storage, plugged in a normal windows7 Dell.

 

After the  first episode I looked for precautions on USB use, but i could not see any on the manual.

 

On the link

https://community.netgear.com/t5/Using-your-ReadyNAS-in-Business/RN104-External-USB-disk-disconnect-...

I read

“There is a current investigation regarding USB detection on the new firmware. For now, a work around is to reboot the NAS after connecting the USB.”

which could suggest that a RN104 could reboot with two USB disks connected (were theyself powered or USB powered ?), if so, then the problem might have started few weeks ago, when it first failed to reboot with the USB disks connected.

 

In the last couple of weeks I also did a Balance and Defrag, but I don’t think I went into Scrub: Balance claimed back some space (from 0.9Gb free to 1.1GB free, on a  4x4TB population with 12GB capacity)

 

 

Any idea of what could have caused this message?

What can I do to get the NAS to reboot normally again?

I have prepared a USB recovery drive with FW6.2.5, but I am awaiting for comments before using it

 

Thanks in advance,  Berillio

p.s. I have the  logs from 2020-02-26

Model: RN104|ReadyNAS 100 Series 4- Bay
Message 1 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

Is one of the USB drives still connected?  It sounds like the NAS is somehow getting fooled into trying to boot from one of them, though I don't know how that could happen.

Message 2 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

No, there were NO drive connected when the NAS went off and rebooted as scheduled to do. Both drives were purely data drives, neither of them has ever been bootable. I tried to reboot with one and both connected, I also tried to use the fron USB port, but it had no effect (not that I was expecting anything).

Message 3 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

Thanks for coming back, (sorry I should have said earlier)

Message 4 of 56
Marc_V
NETGEAR Employee Retired

Re: RN 104 failed to start with "Boot from USB" - no FW update

@berillio

 

You can proceed with the USB Boot recovery and yes recovery using the 6.2.5 is the best FW to use though updating it is still suggested.

 

 

Regards

 

Message 5 of 56
StephenB
Guru

Re: RN 104 failed to start with "Boot from USB" - no FW update

You might want to make sure the backup button isn't stuck in first.  

Message 6 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

Marc V,  Thanks for your advice,

Stephen B, Thanks for the tip

 

Nope .

 

Inserted USB with recovery of FW 6.2.5

NAS was unplugged from power suppy – replugged it in;

Press Backup button and repower, keeping Backup pressed;

After ~10 secs (or less) LCD confirms going into Recovery mode and Reading from USB;

After ~30 seconds “Recovery done”, NAS powers off;

Remove USB stick;

Repower NAS;

“Boot from USB” message.

 

Tried same again, this time network Ethernet line unplugged (just in case).

Same USB message again.

 

Checked the USB – seemed fine but recreated it on a W7 machine: -

Formatted to FAT32 (as it was already) (Quick Format un-checked, Create a MS-DOS disk un-checked): run usbrecovery.exe with Administrator priviledges.

 

Checked that the Backup button wasn’t sticking – there is a positive “click” when pressed and when released.

Unfortunately no changes: after NAS powered itself off, when repowering, the message “Boot from USB” appeared immediately, just like before.

The times of the various operations (before going into recovery mode, and the recovery itself), seemed a little longer ~25 seconds, but I could be easily mistaken.

Message 7 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

After the USB recovery, did you boot with the recover USB drive still inserted?  If not, try it, and see if it does another recovery.  Just because you hear a click doesn't mean the button is having any effect.

Message 8 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

Sandshark.

Yes I did, it run another recovery, it switched itself off, and back to square one again when I repowered it.

And I just did another half a dozen of reboot/recovery/restart, and even filmed it. Stuck in a loop

 

Regarding the Backup button:  If I press it, it will go down making a click. When I let it go, it comes out making a click.

BUT.  

It make no difference whatsoever if I keep it pressed when I repower it or not. The Backup button will be flashing for those 10 seconds or more until it goes into recovery, then it will be lit solid. Then “Recovery done”, and it powers down.

That supports the “button stuck down” hypothesis.

Now, the button moves freely under my finger and clicks, but that “is not having any effect”, hence the switch itself is stuck in the ”pressed” position. How do I free it ?

 

Just for info, I have never pressed the backup button before, or if I did, it would have been years ago; so it may well feel rusty, although it feels very springy under my finger (but that is the button, not the switch itself).

Message 9 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

 

As I said, I filmed it. But the 20” clip of the button being pressed and bouncing back with an audible click (a number of times) is 43MB; (but then we know that the switch is stuck in the pressed position).

 The 12m clip of all the various reboots, with or without the backup button pressed is 1,5GB.

And that was with the lowest setting in the phone…

Message 10 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

It does sound like you have confirmed a bad backup button.  Based on when the 104 was still available new and the warranty duration, I suspect you are out of warranty.  If not, definately contact Netgear for an RMA.

 

I don't have a 104, but the backup button on other ReadyNAS I have looked at is normally closed with one side grounded and the other to a pull-up resistor and IC.  So, at least on ones where it's normally closed, a cracked solder joint is one thing that can make a backup button permanently in the "backup" state and that could result from a handling issue.  Of course, a broken button, damaged pull-up resistor or IC, and maybe some other stuff might also do that.  Netgear does not make schematics available, so troubleshooting requires a lot of guessing along with experiments to confirm or deny those guesses.

 

If you have the skills, or know someone who does, checking the solder joints of the button and doing a continuity check on it could help.  Without a good NAS to test, it's impossible to know what the proper default state of the switch is.  But if it's permanently open, you could short across it and see if that changes things.

 

If you have to pay somebody, you might want to consider a new NAS instead.  Even broken, the 104 has some value, as I see people looking for power bricks and drive caddies all the time.  SATA backplanes also sometimes get damaged, as can the chassis or display.  So subtract anything you might get for it in the current state (in one piece, or parted out) from the cost of a new NAS and compare that to the cost of a replacement.  Include in your assessment what value having a faster NAS with more RAM (thus not potentially having an OS update limitation somewhere in the future) has to you.

Message 11 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

Thank you, Sandshark.

Two issues here, my 104 and the Data on it. The data is obviously more important.

- data

My understanding is that another 104 chassis would access the data as normal (provided I respect the disk order and I do not confuse the slots, is that correct?).

I do not know which other chassis (if any) could be used to read the data, given that I thought I read somewhere that the 104 is getting close to the end of its lifetime, and it would make sense to get a newer, better NAS. A chassis with more slots (6 or 😎 would make sense, given that I was already starting to run short of space, I had~1Gb free out of the 12Gb from the 4x4TB I have, and I could add more (and bigger) disks. But I have not looked at any NAS for four or five years.

- My 104:

I am definitely NOT an electronic engineer. My soldering skill are poor. But there is the little guy in the electronic shop up the road, who fixed my power amp. But before getting to the switch and the possible dry joint, is there a maintenance/service manual I could download, so I could take it apart properly?

Thanks again for your advices.

Message 12 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

I found a page “Procedure to swap enclosures” with lots of useful pictures and I could dismantle the RN104. There is a small PCB with the front USB, the switches and the LEDs. The Backup (and likewise the Power) button operate a PAIR of switches instead of a single one.

It looks perfectly immaculate (a bit dusty maybe). I find it difficult to imagine a dry joint, it is not a component which is exposed to any heat, but one of the switches could have developed a fault. Both you and Stephen B pointed in that direction, there must be an history behind that.

Tomorrow I will try to contact the little electronic shop, but I do not know if, considering the Corona virus situation, I will find it open or not.

 

I wonder if NETGEAR could provide me with a spare PCB…..

Message 13 of 56
StephenB
Guru

Re: RN 104 failed to start with "Boot from USB" - no FW update


@berillio wrote:

 

My understanding is that another 104 chassis would access the data as normal (provided I respect the disk order and I do not confuse the slots, is that correct?).

I do not know which other chassis (if any) could be used to read the data,

You can migrate the disks to any OS-6 NAS.  The RN214 is the current entry model.

 

More slots of course do give you more capacity (and more flexibility).  Though you can also just go with larger disks (4x12 TB for example).

 

Inventory seems low for most on-line resellers though (and the RN214 is very overpriced on Amazon US at the moment - you can get an RN628x for the same price).  So check out the various models.

 

 

Message 14 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

I did some testing. I have a voltmeter reading of ~2000 ohms with the switches in resting position (I may be reading the circuitry), and near zero when each one is pressed. Pretty identical readings for either switch. Wouldn’t that suggest that the switches aren’t faulty?

Is there any other testing which I can do before reassembling the chassis?

I wonder if removing the battery may help. If that would simply wipe out my configurations, I suppose I could leave with that (if that was the case, could I find any configuration in my logs? I configured the NAS five or six years ago, never touched it since, apart from replacing a WD Red which died, so I forgot almost everything about it).

Stephen B, thanks for clarification and suggestions. As it happens, I am in the UK, so pricing here is sort of different, also because NASes, here, aren’t that common yet.

Sandshark, because of the little use of NASes here, the 2nd hand value of my chassis bits may be probably (and unfortunately) negligible.

Message 15 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

Those measurements would indicate that the switch is normally open and functional.  Then resting, you are reading around the open switch (through circuity) and when pushed, you are reading across the short it creates.  Unfortuantely, that's a sign that there is likely nothing you can do to fix the situation, assuming your diagnosis of the NAS believing the switch is always pushed is right (and it sounds to me like it is).

Message 16 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

“Unfortunately, that's a sign that there is likely nothing you can do to fix the situation”

We think that the switches are not faulty and therefore responsible for this behaviour.

So, what else could cause it?

Did the NAS initiate a FW update by mistake (from my part, maybe? I certainly never intended to set the Automatic update, but I cannot exclude ticking a box by mistake). If that was the case, the automatic update “should” have upgraded to 6.5.2, but, if it DID update, did it really update to that FW or maybe it updated to 6.10.3? Because, if that was the case, it might have failed to update properly, because it should have not be done directly but in stages: is that what it could have happened? Maybe I have a 6.10.3 “failed” update, and therefore my recovery to such an early FW is not “legit” (it should be possible to revert from 6.5.2 to 6.2.5). Hence the RN104 asks me to do it again? And it seems to be stuck in a loop because I am repeating the same “wrong” recovery, I should do a recovery to 6.10.3?

 

Can I invoke any procedure with the reset switch in the back? Could I do a “Factory default”? On the Hardware Manual there is a strong warning :

"The factory default reboot process resets the storage system to factory settings, erases all data, resets all defaults, and reformats the disk to X-RAID2."

But what if the disks are removed beforehand? It cannot reformat an empty space. So anyone could try a factory reset without losing their data – if that was the case, the manual would have suggested, after the warning, to “remove the disks” before doing the Factory default. But it didn’t. Do I miss something here?

If I could do a Factory reset, that could bring me to the same case of a “brand new” kit, which may boot up normally. I could then update the original FW installed in the factory to the 6.2.5 which I had before, if I need to have the SAME fw I had before.

Stephen B says “You can migrate the disks to any OS-6 NAS.”.

That would suggest that it does not really matter which FW is used, provided is an OS-6

 

I asked (in a previous reply) what would happen if I removed the battery: it seems that on a PC motherboard, removing the battery for at least 20 minutes has the same resetting effect on the BIOS as shorting the jumper – which is not present on the RN104 motherboard. Could that have any beneficial effect (ie reset a “Reboot from USB” flag) ^^?

 

TBH, I find difficult to believe at an hardware fault. This NAS was operative for ~6 years, and that is not really a long time.

But, if that was really the case... well, how could I justify invest in another one, maybe better quality, more expensive but more bays and more expansion capacity, which could last me a lot longer … which could also die on me so suddenly?

Message 17 of 56
StephenB
Guru

Re: RN 104 failed to start with "Boot from USB" - no FW update

The NAS boots from an OS partition on the disks, and a factory default just reformats the disks and reinstalls the software from the flash.

 

So you could attempt that procedure.  It requires the boot menu though, which could be a problem. You can overcome that problem by zeroing one of the disks, inserting it by itself into the NAS and powering up.  That normally does a fresh factory install.

 

There were a couple of issues when going from old firmware to current firmware:

 

 

 

 

 

 

 

Message 18 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

I suppose a bad SYSLINUX.CFG in flash could make it always boot from USB, but it's hard to fathom how unless that's what happens whene there is no valid SYSLINUX.CFG.  A USB recovery should have fixed that, though; as it replaces SYSLINUX.CFG.

 

Message 19 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

Stephen B & Sandshark, Thank you both.

In reverse order, Sandshark, I tried to look into the ISO and GZ files with Winrar, but it cannot look into them  (to check for SYSLINUX.CFG). Is there any other archiver/other which could do so – assuming there is any way of actually reading its contents and/or editing  it – not that I think you wanted me to do so?

StephenB, there were few points to address on your post, but one is very quick. In the last link,

the user talks about “few minutes” for recovery; then power up and "updating" which takes 15 minutes.

In my case, we are talking of

~30 seconds with the button pressed before “Booting”begins (or without pressing, same thing)

6/8 seconds of Booting before “Recovery” begins

30 seconds of Recovery, before the message “Recovery done” is displayed

2 seconds later the NAS is depowered.

Powering up presents the “NETGEAR Storage Welcome” and within 1 second “Booting from USB”.

Are those the times which you would expect for a (successful) recovery? I mean, in my case is recovery happening (but failing), or it should be a much longer procedure, and in my case it is not happening at all?

Message 20 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update


StephenB , maybe I did not say explicitally that the 11TB of data is the ORIGINAL copy, and I do not have a backup, so I do it now.
But I presume that you understood that anyway, and you procedure consist of :
a) sacrify one disk (of an array of four, which is a “redundant” array) by zeroing its contents (I can use WD utility to do that)
b) Power up NAS and hot insert the “zeroed” disk – which should cause a fresh install
c) hot insert the other three disks, which contain what now is a “non redundant” data set.
The NAS should then rebuild the full data set recopying the data on the first disk which is empty.
On the other hand I might have misunderstood your post, and you were simply reasoning on the best procedure to install 6.10.3, the latest firmware (which, although recommended, IS NOT AT ALL my priority, which is to give me access again to my data I simply said that I cannot exclude the possibility that automatic updates were enabled by mistake).
I am currently running diagnostic on a 4Tb WD Red which I could use as a test for the procedure. But that is now ~5% done, and should take in excess of another 31 hours.

Message 21 of 56
StephenB
Guru

Re: RN 104 failed to start with "Boot from USB" - no FW update


@berillio wrote:

 

StephenB, there were few points to address on your post, but one is very quick. In the last link,

the user talks about “few minutes” for recovery; then power up and "updating" which takes 15 minutes.

In my case, we are talking of

~30 seconds with the button pressed before “Booting”begins (or without pressing, same thing)

6/8 seconds of Booting before “Recovery” begins

30 seconds of Recovery, before the message “Recovery done” is displayed

2 seconds later the NAS is depowered.

Powering up presents the “NETGEAR Storage Welcome” and within 1 second “Booting from USB”.

Are those the times which you would expect for a (successful) recovery? I mean, in my case is recovery happening (but failing), or it should be a much longer procedure, and in my case it is not happening at all?


I haven't needed to do a USB recovery on my ReadyNAS, but your times are generally about what I would expect. Note in his case the "updating" step was installing the firmware on the hard drives.

 

It is possible this one could be short

  • 30 seconds of Recovery, before the message “Recovery done” is displayed

Perhaps someone who has done the procedure can comment on that.

 

 


@berillio wrote:


On the other hand I might have misunderstood your post, and you were simply reasoning on the best procedure to install 6.10.3, the latest firmware (which, although recommended, IS NOT AT ALL my priority, which is to give me access again to my data I simply said that I cannot exclude the possibility that automatic updates were enabled by mistake).


You had brought up the idea of doing a factory default, and I was just trying to respond to that.  The factory default is destructive, there is no avoiding that.

 


@berillio wrote:


But I presume that you understood that anyway, and you procedure consist of :
a) sacrify one disk (of an array of four, which is a “redundant” array) by zeroing its contents (I can use WD utility to do that)
b) Power up NAS and hot insert the “zeroed” disk – which should cause a fresh install
c) hot insert the other three disks, which contain what now is a “non redundant” data set.
The NAS should then rebuild the full data set recopying the data on the first disk which is empty.


No, that is not what would happen.  Step (c) would resync those disks to your new array, leaving you with a new but empty file system.

 

 


@berillio wrote:


I am currently running diagnostic on a 4Tb WD Red which I could use as a test for the procedure. But that is now ~5% done, and should take in excess of another 31 hours.


Is this disk part of the array?

 

Assuming no, It is worth zeroing the disk (perhaps using the quick write zeros test), powering down the NAS and inserting that disk by itself, and powering it up.  If that fails to install it will confirm that the problem is in the chassis.  If it succeeds, then it will point more towards corruption on the hard disks.

 

FWIW, 31 hours is far longer than I'd expect for the long non-destructive test.  That's closer to what I see with a 10 TB WD Red.  Are you using a USB 2.0 adapter?

Message 22 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

StephenB, TY for your patience in putting up with me.

The disk is not part of the array – but it once was: it is a 4Tb WD red which failed ~1 year ago and replaced with an identical WD red. It is accessed via a StarTech USB 2.0 SATA/IDE adapter, as you correctly guessed. At this moment it is being checked via Window Surface Scanner vers 2.0 (never used this utility before). I tried WD Data Lifeguard Diagnostic, it passed the Quick test but failed the Extended test after 5 minutes with “Error Code 08 - Too many bad sectors”;  I was expecting the same result from WSS, but maybe WD DLGDIAG excluded those bad sectors before giving up. WSS has reported no errors so far and it is currently @35% with ~21h to go. Interestingly, it reports a capacity of 3726.02 GB, while WD DLGDIAG reported 4000.79 GB (different units?)

I do not think I have any other unused SATA disks – I would have not used a bad one (although the WD failure was odd and unexpected). Even assuming that it clears the surface tests and it is zeroed, the NAS may read its id, and refuse to use as “failed disk”. I will search for another SATA drive, maybe I can find an unused small USB which I can used for this test.

Message 23 of 56
berillio
Aspirant

Re: RN 104 failed to start with "Boot from USB" - no FW update

I found a small WD USB drive, almost empty. BUT I also realised that I would need the adaptor for the 2.5” drive, which would be in the box, which I cannot find. It’s most likely in the loft, and it may be impossible to fetch it now. Sorry, we may have to wait. 40%, 19h to go.

Message 24 of 56
Sandshark
Sensei

Re: RN 104 failed to start with "Boot from USB" - no FW update

The SYSLINUX.CFG file it boots from is on the NAS flash, not the USB drive.  But unless you can boot to Tech Support mode and can FTP in, you have no way to examine it.  But, as I said, it should be replaced with the USB recovery, which sounds like it is working properly.

Message 25 of 56
Top Contributors
Discussion stats
  • 55 replies
  • 4154 views
  • 0 kudos
  • 4 in conversation
Announcements