ReadyNas 214 - BTRFS error during scrub - Stuck in readonly

[Artanis]

Hello everyone.

I hope you can help me out... on Febraury 15 my ReadyNas 214 (4x8TB) started its scheduled scrub. After a couple of days I noticed I couldn't add new files in the NAS and realized the scrub wasn't yet completed. I thought it might've been normal (maybe the scrub needs to be completed first), but then since it was already 3 days and the log didn't show "scrub completed" I started to get suspicious. I've downloaded the full logs, and in the KERNEL.LOG is see:

Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in btrfs_commit_transaction:2233: errno=-5 IO failure (Error while writing out transaction)
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): forced readonly
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS warning (device md126): Skipping commit of aborted transaction.
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in cleanup_transaction:1856: errno=-5 IO failure
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): delayed_refs has NO entry
Feb 15 08:16:45 ArtanisNAS kernel: BTRFS warning (device md126): failed setting block group ro, ret=-30
Feb 15 10:09:28 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

And then

Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152
Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

Repeated over and over again, for over 4000 lines since it's started the scrub, basically. I'm quite worried right now... I don't see a scrub percentage so I don't know if it's still going, but the fact that this error keeps showing is really worrying (and, well, the fact the unit appears to be in readonly state).

I can access it, and the admin console, but I can't edit any file. Can you please help? I have lots of data in this unit ... 

By the way, the unit did not improperly shutdown and it is connected to UPS.

Thank you for any help you can give.

I forgot to mention, the firmware is 6.9.1 

I didn't update despite being new version as I was afraid some app might get broken. 

[StephenB · Accepted solution]

---

@Artanis wrote:

Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in btrfs_commit_transaction:2233: errno=-5 IO failure (Error while writing out transaction)
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): forced readonly
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS warning (device md126): Skipping commit of aborted transaction.
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in cleanup_transaction:1856: errno=-5 IO failure
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): delayed_refs has NO entry
Feb 15 08:16:45 ArtanisNAS kernel: BTRFS warning (device md126): failed setting block group ro, ret=-30
Feb 15 10:09:28 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152
Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

You ended up with a corrupt file system.

The best thing to do is back up all your files, then do a factory default from the system->settings page.  

That will re-format everything, so you will need to reconfigure the NAS from scratch (and reinstall your apps).  Then restore the files from the backup.

[StephenB · Accepted solution]

---

@Artanis wrote:

Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in btrfs_commit_transaction:2233: errno=-5 IO failure (Error while writing out transaction)
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): forced readonly
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS warning (device md126): Skipping commit of aborted transaction.
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS: error (device md126) in cleanup_transaction:1856: errno=-5 IO failure
Feb 15 08:16:44 ArtanisNAS kernel: BTRFS info (device md126): delayed_refs has NO entry
Feb 15 08:16:45 ArtanisNAS kernel: BTRFS warning (device md126): failed setting block group ro, ret=-30
Feb 15 10:09:28 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152
Feb 18 09:44:56 ArtanisNAS kernel: BTRFS error (device md126): bad tree block start 11025967846460254743 18195154993152

You ended up with a corrupt file system.

The best thing to do is back up all your files, then do a factory default from the system->settings page.  

That will re-format everything, so you will need to reconfigure the NAS from scratch (and reinstall your apps).  Then restore the files from the backup.

[Artanis]

Thanks for the reply.

Is there no other way to fix this? If not, how much time do I have to do the backup? It's a lot of data...

Is there something I must NOT do (i.e. reboot, shut the unit off...) to avoid losing the data?

Thank you again.

[StephenB]

---

@Artanis wrote:

Is there no other way to fix this? 

---

You could engage paid Netgear support (who can access the NAS remotely).  If they decide data recovery is needed it would be expensive.  https://kb.netgear.com/69/ReadyNAS-Data-Recovery-Diagnostics-Scope-of-Service

You could also attempt to repair the file system yourself using linux commands.  This is risky, and if you don't already know linux I wouldn't attempt it.

---

@Artanis wrote:

Is there something I must NOT do (i.e. reboot, shut the unit off...) to avoid losing the data?

---

I wouldn't reboot it until I'd made a backup.

---

@Artanis wrote:

If not, how much time do I have to do the backup? It's a lot of data...

---

Hard to say - it would depend on exactly what is going on.  If you have failing disk(s), for example, the volume could completely fail at any time.

In general, any device can fail w/o warning, so in the future you should consider putting a backup plan in place for your data.

[Artanis]

Thank you for your assistance, it's much appreciated.

As far as I know, all disks are fine (they are rather new, S.M.A.R.T. values check out fine). 

Regarding a backup it's really bad timing. I was about to prepare a backup solution with another PC using Unraid, but I was waiting some hardware to come so I don't have it in place yet.

The support is really expensive though, even just to diagnose it. 😞 I'll try to backup as fast as I can. Thank you again.