RN104 Stuck at "Booting..."

[daKlone]

My old faithful RN104 running f/w 6.10.9 has encountered a problem which I need some help with:

Initially, it sat say "Booting..." indefinitely (no percentage bar appears).  All 4 drive lights are on, the "Act" light is on continuously and the "Power" light is flashing.

I could SSH into it, and discovered that I could make any changes to any files as it came up with "no free space" errors.  df -h showed plenty of disk space available however.

Tried to do a firmware reset via USB, but it asked me to Telnet into it for debug and wouldn't accept the root password I'd previously set and now I can't SSH to it any more.

So now I'm stuck...what should I try now?

[StephenB]

---

@daKlone wrote:

Tried to do a firmware reset via USB, but it asked me to Telnet into it for debug and wouldn't accept the root password I'd previously set and now I can't SSH to it any more.

---

Can you log into the web ui?

If you can, then download the full log zip file and look in there for clues.

---

@daKlone wrote:

df -h showed plenty of disk space available however.

---

Generally you need to use btrfs commands to determine the actual free disk space on the volume.

[daKlone]

No, I can't access it at all now...

[StephenB]

---

@daKlone wrote:

No, I can't access it at all now...

---

Have you ever used tech support mode?

[daKlone]

Hmm, not that I recall, no.

[StephenB]

---

@daKlone wrote:

Hmm, not that I recall, no.

---

Start by booting the NAS into tech support mode using the boot menu.

Instructions are on pages 28-29 here:

• https://www.downloads.netgear.com/files/GDC/READYNAS-100/ReadyNAS_%20OS6_Desktop_HM_EN.pdf

Next you will need to access the NAS using Telnet.  If you are using a windows PC, I suggest you use the Putty client for that.

• https://www.chiark.greenend.org.uk/~sgtatham/putty/

When you connect to the NAS, log in as root.  The password is infr8ntdebug.

After that see if you get any errors when you run this command

rnutil chroot

That starts RAID, and attempt to chroot to the OS stored on the disks.  That might fail in your case (depending on what went wrong with your USB recovery attempt).

Other commands to mount the data volume manually are possible, but they depend on your expansion history.  If rnutil chroot works, try entering

lsblk

 and then post the output here.

[daKlone]

"chroot:  can't change root directory to '/sysroot': No such file or directory"

Tried the 'OS Reinstall' option too, which seemed to do something but resulted in no change.

[StephenB]

---

@daKlone wrote:

"chroot:  can't change root directory to '/sysroot': No such file or directory"

---

Did you run

rnutil chroot

Or just enter chroot?

[daKlone]

Tried again...didn't realise that the 'rnutil chroot' had worked as there was no "root@" prompt initially...but it looks like it did:

[StephenB]

It looks like the volume has never been expanded

You can try to mount the data volume with

mdadm --assemble --really-force /dev/md127 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
btrfs device scan
mount /dev/md127 /data

---

[daKlone]

Interestingly, it reported that /dev/sda3 /dev/sdb3 /dev/sdc3 and /dev/sdd3 were busy and skipped, but was still able to mount the data volume.

So I now have access to the data volume 😁

What's next in terms of diagnosing the problem?

[daKlone]

I have news!

Doing a bit of digging, I managed to get the device to boot by unplugging the network while it was booting.  I can now get to the web admin page and log-in via SSH.

Logging-in via SSH I notice that systemd-journal and rsyslogd are using 55% and 42% CPU respectively, making everything very sluggish.

I also noticed that /var/log/daemon.log and /var/log/syslog are expanding at a phenomenal rate, both reaching 100M+ in the space of a couple of minutes!

'tail -f /var/log/syslog' gives:

Which seems to suggest that 'minidlnad' is throwing continuous errors due to not finding some files, which is filling the log files and bogging down the CPU.

Any suggestions as to what to do next?

[StephenB]

---

@daKlone wrote:

Interestingly, it reported that /dev/sda3 /dev/sdb3 /dev/sdc3 and /dev/sdd3 were busy and skipped, but was still able to mount the data volume.

---

Were you in tech support mode?

[StephenB]

---

@daKlone wrote:

Any suggestions as to what to do next?

---

Turn off plex and DLNA.  

[daKlone]

So I turned off DLNA but Plex had disappeared from the 'Apps' section so I deleted it manually (just in case)...and calm has been restored!

Just to complete the story, I forgot to mention that as soon as I was able to log-in to the web GUI I re-loaded the firmware again (v6.10.9).

It's a bit disconcerting that I still don't know what exactly went wrong, but I'm re-building it bit-by-bit to see if it goes wrong again at any point.

Thank you Stephen B, as always, ready and able to assist! 😎

[StephenB]

---

@daKlone wrote:

It's a bit disconcerting that I still don't know what exactly went wrong, but I'm re-building it bit-by-bit to see if it goes wrong again at any point.

Thank you Stephen B, as always, ready and able to assist! 😎

---

I'm glad I could help.

I'm not sure exactly what went wrong either, but the plex media library (in .apps) appeared to have been partially deleted.