RN316 does no finish booting and OS install gives error "could not extracting files) last FW

[helio]

Hi all, without any warnings the RN3165 stopped completing the boot at about 99%.

On the first day I could login and see one fan only working. It looks the power supply fan is dead. 

Tried the OS reinstall but it gives file extraction error. 

I cannot log in and the shares are not available anymore.

Does the RN user standard mini power supply or have the cables been made proprietary?

How to recover the data? Please, help.

Many Thanks.

[Sandshark]

This is likely a problem with the data volume, not the hardware.  It's highly unlikely to be the power supply, as it wouldn't get that far in booting.  The first best step is to remove the drives (labeling them so they go back in in the same order later) and use a scratch drive (containing no data you care about) and see if the NAS will boot and create a new volume.  If it does, and I suspect it will, then you probably have a volume problem, not a hardware one.  It is possible one of the drives is bad, so you can use the drive test in the boot menu or (better) a PC and the drive manufacturer's tools to check their health once you know the chassis is good.

If all the hardware checks out, you can ask for assistance here in recovering your volume, or at least the data from it, if you need to.  But hopefully you have a backup, in which case a factory default and restoration of data from backup is the easiest path forward.

[helio]

Hi Sandshark, right on, it booted with different 3 disks. It asked me to create a new Volume. I did not know if was ok to do it so I did not.
Is it ok to create a new volume when having the new 3 disks in or will that cause issues when I insert the  6 original disks configured in  Flex-RAID Volume? BTW, all disks appear in blue, no red, in the original Flex Raid. I thought I was protected ;(

I have spare disks. How do I recover the data without having a backup? Thank you.

[Sandshark]

It won't hurt to create a new volume on the new drive(s).  All configuration information is stored on the drives, not in the NAS itself.  But you are probably going to want to use the NAS to recover the data, so don't plan on that new volume being where you recover to.

Assuming you are at RAID5, you could try booting (in read-only mode via the reset button menu is best) with one of the drives missing, then trying each as the missing one, but I'm doubtful that's going to be the answer.

Recovering volumes is not my forte, and there are a lot of different things that can be wrong.  Just going in and checking takes some knowlege of Linux and the command line, and it is very east to make things worse instead of better.  It's happened to my twice, once a very long time ago on an NV+ when it was current and once on an EDA500 because of an eSATA cabling issue, and I just destroyed the volume and recovered from backup.

With no backup, the best method is, unfortunately, paid support from Netgear.  Since they apparently no longer offer "per incident" support, than means buying a support contract.  And if data recovery ends up being necessary (which, at this point, there is no reason to believe is the case), that's extra.

@StephenB or @rn_enthusiast may be able to better help you now that we know it's not a hardware issue.

[helio]

Apparently there is no disc failure. The issue seems to be the Volume data somehow getting corrupted, even if this was not supposed to happen.

I thought protection from this happening  was the avantage of buying this unit and using  Flex Raid,

The system configuration data was written to the disks, so even if a disk failed the configuration could still  be recovered. 

I feel deceived, Netgear promised protection when using this unit with Flex Raid but that is not true.

Could someone from Netgear assist to recover the data, before we start looking at other means of obtaining re-compensation for our loss? 

Thank you.

[Sandshark]

The primary purpose of RAID is availability -- loss of a drive does not generally require taking down the system to do a repair and data restoration.  It's not backup, and it's not foolproof.  In most cases, it also makes separate backup completely redundant, but not always.

It sounds to me like your OS partition is too full, something about which you should have received warnings in the log and email (if enabled).  That would leave no working space for the OS to be extracted in a re-install and can also provent the volume form mo9unting or, in some cases, even cause damage to it.

Can you boot into support mode?  Are you at all familiar with the Linux command line?  That is what would be needed to go in and clean up whatever has filled it.

[helio]

Yes, I can boot in support mode alright but am not so familiar with Linux CL  I have used it years ago with Putty.exe

I would need instructions on what to delete from where.

I could send a Teams invitation for a meeting allowing for screen sharing to do the deed.

Do you know from which directory  or folders we need to delete files and what is safe to delete?

Thanks a lot.

Helio

[Sandshark]

Actually, you telnet into support mode.  @StephenB is more of an expert on where and how to look for files that might be filling the OS partition.  I'm more of a hardware guy.

[helio]

I really appreciate all you have already done 🙂

@StephenB, apologies if I disturb.  Could I ask you to write an action plan that I could use to sort out this situation?

Thank you in advance for your input.

[StephenB]

The first step is to get into tech support mode

1. boot up into tech support mode
2. telnet into the NAS.  Username is root and the password is infr8ntdebug
3. Enter rnutil chroot (which will start RAID and chroot)
4. Check the OS partition fullness by entering btrfs fi df //

Let us know the OS partition fullness by posting the output of the btrfs command here.

[helio]

Hi Steve, thank you for sending this in.

Here are the results:

# rnutil chroot
root@3C62350D01347:/# btrfs fi df //
Data, single: total=3.01GiB, used=1.44GiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=497.50MiB, used=479.89MiB
GlobalReserve, single: total=240.00MiB, used=222.45MiB

Let me know if you had like to see this over a Teams meeting, I would need an email to send the invitation to.

Best regards,

[StephenB]

---

@helio wrote:

Data, single: total=3.01GiB, used=1.44GiB

---

This isn't full (used is 1.44 GiB out of 4), but I think it would be good to balance it (total allocated is 3 GiB)

This command would do that:

btrfs balance start //

It will give a warning (including a 10 second count down), but the OS partition is small, and it normally won't take very long (a minute or so).  

Then I suggest trying to mount the data volume, and see if that works.

If you are using XRAID, then the commands needed are:

# btrfs device scan
# btrfs fi show
# mount /dev/md127 /data

Note that if this works, you can then make a backup of your data (if you don't have one).

---

@helio wrote:

Let me know if you had like to see this over a Teams meeting, I would need an email to send the invitation to.

---

I'm just another ReadyNAS owner, and I really can't do remote support.

[helio]

Hey Stephen, 

It is not using X-Raid but Flex-Raid.

is the command the same if not, please, let me know, i stay put waiting for your reply.

Thank you.

[helio]

Meanwhile the balance gave an error:

Starting balance without any filters.
ERROR: error during balancing '//': No space left on device
There may be more info in syslog - try dmesg | tail

[helio]

bash: ERROR:: command not found
root@3C62350D01347:/# There may be more info in syslog - try dmesg | tail
bash: There: command not found
root@3C62350D01347:/# dmesg | tail
[ 264.060849] disk 0, o:1, dev:sdb5
[ 264.060853] disk 1, o:1, dev:sdc5
[ 264.060856] disk 2, o:1, dev:sda5
[ 264.060860] disk 3, o:1, dev:sdd5
[ 264.060864] disk 4, o:1, dev:sde5
[ 264.061090] md125: detected capacity change from 0 to 5000299151360
[ 264.081399] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[ 264.081605] BTRFS: device label 0ed5edda:root devid 1 transid 2563326 /dev/md0
[ 264.082254] BTRFS info (device md0): has skinny extents
[ 304.882558] BTRFS info (device md0): 15 enospc errors during balance

[StephenB]

---

@helio wrote:

Meanwhile the balance gave an error:

Starting balance without any filters.
ERROR: error during balancing '//': No space left on device
There may be more info in syslog - try dmesg | tail

---

Try 

btrfs balance start -dusage=0 //

and see if that completes.

If it does, try increasing dusage in steps (try 50 next)

---

@helio wrote:

It is not using X-Raid but Flex-Raid.

is the command the same if not, please, let me know,

---

If you have a single volume, then the only thing you need to change is the volume name (/data) on the last command.  That needs to be your volume name (which you specified when you created the volume).

[helio]

Note that is supposed to be 6 disks of 4TB , the  volume is called "data"

root@3C62350D01347:/# btrfs balance start -dusage=50 //
ERROR: error during balancing '//': No space left on device
There may be more info in syslog - try dmesg | tail
root@3C62350D01347:/# dmesg | tail
[ 264.061090] md125: detected capacity change from 0 to 5000299151360
[ 264.081399] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[ 264.081605] BTRFS: device label 0ed5edda:root devid 1 transid 2563326 /dev/md0
[ 264.082254] BTRFS info (device md0): has skinny extents
[ 304.882558] BTRFS info (device md0): 15 enospc errors during balance
[ 1510.489423] BTRFS info (device md0): 4 enospc errors during balance
[ 1753.653588] BTRFS: device label 0ed5edda:data devid 3 transid 4126965 /dev/md125
[ 1753.653684] BTRFS: device label 0ed5edda:data devid 2 transid 4126965 /dev/md126
[ 1753.653768] BTRFS: device label 0ed5edda:data devid 1 transid 4126965 /dev/md127
[ 1884.973521] BTRFS info (device md0): 4 enospc errors during balance

[helio]

Here the mounting results:

root@3C62350D01347:/# btrfs device scan # btrfs fi show # mount /dev/md127 /Data
Scanning for Btrfs filesystems
root@3C62350D01347:/#

dmesg | tail
[ 264.060853] disk 1, o:1, dev:sdc5
[ 264.060856] disk 2, o:1, dev:sda5
[ 264.060860] disk 3, o:1, dev:sdd5
[ 264.060864] disk 4, o:1, dev:sde5
[ 264.061090] md125: detected capacity change from 0 to 5000299151360
[ 264.081399] EXT4-fs (md0): VFS: Can't find ext4 filesystem
[ 264.081605] BTRFS: device label 0ed5edda:root devid 1 transid 2563326 /dev/md0
[ 264.082254] BTRFS info (device md0): has skinny extents
[ 304.882558] BTRFS info (device md0): 15 enospc errors during balance
[ 1510.489423] BTRFS info (device md0): 4 enospc errors during balance

[helio]

Hi, it did not work, still cannot see my data shares:

Scanning for Btrfs filesystems

root@3C62350D01347:/# btrfs fi show

Label: '0ed5edda:root'  uuid: ae33cd9d-ca6c-4cfa-9b1d-eec946243757

        Total devices 1 FS bytes used 2.13GiB

        devid    1 size 4.00GiB used 4.00GiB path /dev/md0

Label: '0ed5edda:data'  uuid: cf02b0e0-e9f7-4b61-b57e-26975417ebb4

        Total devices 3 FS bytes used 12.68TiB

        devid    1 size 9.07TiB used 7.68TiB path /dev/md127

        devid    2 size 4.55TiB used 3.15TiB path /dev/md126

        devid    3 size 4.55TiB used 3.15TiB path /dev/md125

root@3C62350D01347:/# mount /dev/md127 /data

[helio]

# btrfs fi df //
ERROR: not a btrfs filesystem: //

[StephenB]

---

@helio wrote:

# btrfs fi df //
ERROR: not a btrfs filesystem: //

---

Did you reboot?

This command worked before, correct?

[helio]

yes, it worked before.

I rebooted after running 

mount /dev/md127 /data

to see if it worked.

What to do next?

[helio]

How can I mount the file system back?

[Sandshark]

I noticed that the metadata looks quite full:

Metadata, DUP: total=497.50MiB, used=479.89MiB

I suspect that's why it's saying there is no remaining space, but I have no idea how it got to be so full to start with.

[StephenB]

---

@helio wrote:

I rebooted after running 

mount /dev/md127 /data

to see if it worked.

---

Did you reboot normally?

If you did, reboot in tech support mode again, and repeat the steps until you get the data volume mounted. (Don't try the balance).

At that point I think you should back up all the data, and do a factory default (setting up the NAS again).  We can start the SMB service if you need to back up over the network.