Forum Discussion

Tutor

Dec 20, 2024

Solved

RN314 don't boot anymore after trying to fix "no space left on device" error.

Hi,

Here's the full story that led me to a ReadyNAS RN314 that no longer starts with a "flush_space+5a" message on the LCD.

Actually, after seeing that I lost (I mean visible on the network) the SMB, NFS shares from my RN314 I tried to login via the Web UI.

But each time it was looping on the login screen until it asked me to follow th recovery password procedure that I did not follow. Instead I logged in via SSH and I discovered that the OS was complaining about "no space left on device" on the / (root) making /tmp unsuable for all the pplications!

I ran the command df to see where the space was really missing and there was still almost 1TB on the data file system (/dev/md127) and 1,3 GB on the / (/dev/md0) and ieven the I-nodes were not used at 100%. Weird as I was not able to still create a file in /tmp (no space left on device), it should be something related to btrfs.

I deleted some log files from /var/log to free some MB and I tried to check the /dev/md0 with the command btrfs filsystem check / (if I rember well the syntax).

It ran the night and when I wake up the ssh connection was closed. I reconnected and it says that the filesystem was in read only. Then I asked a reboot and unfortunatly the NAS stop booting with the LCD displaying "flush_space+5a" msg and since it does boot anymore.

After googled a bit, I found that maybe reinstalling the OS via the boot menu can soved the issue without loosing all my 8TB of data. I tried and unfortunatly no change I got still the "flush_space+5a" like in the normal boot mode.

I don't want to try the USB recovery method as I understood well the /dev/md0 will be reflashed and all my data on /dev/md127 will be reset loosing all my data.

I there a way to boot like in Tech support mode (via the boot menu) or an other way to gain access to my data that I could backup on an external disk before resting everything.

I hope I will not loose almost 10 years of data.

Br,

Cataldo.

PS: The firmware installed was the latest available.

Booting

ReadyNAS OS 6 on Legacy models

StephenB
Dec 25, 2024
curso wrote: Sorry it was my cut and paste that I messed up choosing a wrong format 8-(.

Yeah, the forum software doesn't handle dashes correctly.

Thx for the update. It looks like there was no file corrupted. So I think the system is ok now.

26 Replies

Replies have been turned off for this discussion

StephenB
Guru - Experienced User
Dec 20, 2024
curso wrote:

After googled a bit, I found that maybe reinstalling the OS via the boot menu can soved the issue without loosing all my 8TB of data. I tried and unfortunatly no change I got still the "flush_space+5a" like in the normal boot mode.

That doesn't work when the OS partition is full. The reinstall does a partial reinstall, and doesn't delete the files that are filling it up.

curso wrote:

even the I-nodes were not used at 100%.

There are no inodes in BTRFS, and the RN314 uses BTRFS for the OS partition as well as the data volume.

curso wrote:

I don't want to try the USB recovery method as I understood well the /dev/md0 will be reflashed and all my data on /dev/md127 will be reset loosing all my data.

I there a way to boot like in Tech support mode (via the boot menu)

Don't try a USB recovery at this point. It will not destroy your data, but I don't think it will fix your problem either.

If you boot up in tech support mode, you can access the NAS using Telnet. If you use Windows, then I recommend using putty as the client.

https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html

Make sure you select telnet as protocol (using the pulldown on the right).

When you connect to the NAS, log in as root. The password is infr8ntdebug.

Once logged in, type

rnutil chroot

That will start the raid, mount the OS partition and do a chroot. It will not assemble or mount the data volume. If the issue is the OS partition, then we shouldn't need to mount the data volume manually.

If that works, then try entering

btrfs fi usage //

Copy/paste the output into a reply here.
- curso
  Tutor
  Dec 21, 2024
  Hi Stephen,
  
  Thanks for your quick reaction.
  
  I just logged in as root in Tech support mode.
  Launched the rnutil chroot command but I got a "killed" message like you can see
  I guess it failed with the message "flush_space+5a" on the LCD. I still tried the brtf command in case but it failed.
  
  # btrfs fi usage // ERROR: not a btrfs filesystem: //
  - StephenB
    Guru - Experienced User
    Dec 21, 2024
    Does
    
    rnutil start_raids
    
    fail the same way?