NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
1002030001 - Commit failed when making or checking permissions after firmware 6.7.5 update
My ReadyNAS 214 device has been running very smoothly for months. I use it for backing up and storing a lot of static content on my home network. This morning I noted that I could not copy files from...
Thanks for the reply. Inside dmesg.log, there are a large number of the following lines, about 12 per second:
- [Wed Jul 12 07:43:37 2017] BTRFS critical (device md127): unable to find logical 2595263971328 len 4096
Inside systemd-journal.log, there were 2246 instances of:
Jul 11 07:37:08 [redacted] kernel: BTRFS critical (device md127): unable to find logical 2595263873024 len 4096
Inside mounts.log:
/dev/md127 on /data type btrfs (ro,noatime,nodiratime,nodatasum,nospace_cache,clear_cache,subvolid=5,subvol=/)
/dev/md127 on /apps type btrfs (ro,noatime,nodiratime,nodatasum,nospace_cache,clear_cache,subvolid=257,subvol=/.apps)
/dev/md127 on /home type btrfs (ro,noatime,nodiratime,nodatasum,nospace_cache,clear_cache,subvolid=256,subvol=/home)
/dev/md127 on /run/nfs4/home type btrfs (ro,noatime,nodiratime,nodatasum,nospace_cache,clear_cache,subvolid=256,subvol=/home)
Looks like read only if "ro" is an indication
The first segment of the btrfs.log:
Label: '11b7bc56:data' uuid: bfc1ea85-d4ed-4b14-a0dd-357b1b85a202
Total devices 1 FS bytes used 11.72TiB
devid 1 size 16.36TiB used 11.75TiB path /dev/md127
=== filesystem /data ===
Data, single: total=11.74TiB, used=11.72TiB
System, DUP: total=8.00MiB, used=1.28MiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.50GiB, used=717.62MiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=252.00MiB, used=0.00B
=== subvolume /data ===
=== btrfs dump-super /dev/md127
superblock: bytenr=65536, device=/dev/md127
*[redacted] is my modification for server name
Thanks,
Michael
Your volume is mounted read-only, which explains the symptoms. In the logs, you should find mention of something like "falling back to read-only". This is because of some BTRFS corruption.
As far as my experience with BTRFS goes, you're better off going for the backup + wipe + restore option, imho.
It seems that this what was done on this occurrence: https://community.netgear.com/t5/Using-your-ReadyNAS/btrfs-search-forward-2d4-error-on-RN204/m-p/1199638#M122268
This will save you huge amount of time. Even if you contact NETGEAR Support, there is a good chance they'll dump the filesystem anyway (I doubt they will fix the filesystem in place).
Well, I won't shoot the messenger. But, argh.
I did a search through all the logs using Notepad++ for the terms you suggested. Lots of results for Read-Only file system, but nothing suggesting the cause of the issue, overtly. I don't know enough about each individual log to look for a root cause. I only assume the firmware update instigated it. The only regular activity on this NAS is the nightly backups and they've been going fine.
Fortunately, I kept my old NAS around and can use it for backup/wipe/restore. I'll probably start after work, so if anyone else wants to chime in (hello, Netgear Mod), feel free.
Thanks for the support.
Michael
Sorry for bringing the bad news...
What's your configuration on the shares in regards to snapshots, bit rot protection, compression and on your volume in regards to quota accounting and scheduled maintenance?
I don't have any snapshots (I actually do a backup of highly critical stuff to the external drives). There are only two users and one group (me, my wife, and our default group), so there is no quota. Bit rot is off, no compression. I've got alerts set for everything and email config. No emails came my way. Periodically I set a defrag in motion, but it's been a while. Also, I reboot every now and then, but not in the last few weeks except for last week with the firmware update.
Sounds like a good configuration.
BTRFS quotas can be enabled on the volume even if there is no quota set on the shares. To avoid confusion, I tend to call the first one quota accounting. If you don't use quotas on any share, you should disable quota accounting.
You should still check the health of the drives in disk_info.log. If you look at the alert thresholds for the drives, they're insane imo: https://kb.netgear.com/30046/ReadyNAS-OS-6-Disk-Failure-Alerting
Wow, that is a pretty forgiving threshhold. I broke the piggy bank on the drives and got the WD RED NAS drives. They seem to be doing great. All four look a lot like this:
Device: sda
Controller: 0
Channel: 0
Model: WDC WD60EFRX-68L0BN1
Serial: [redacted]
Firmware: 82.00A82
Class: SATA
RPM: 5700
Sectors: 11721045168
Pool: data
PoolType: RAID 5
PoolState: 1
PoolHostId: 11b7bc56
Health data
ATA Error Count: 0
Reallocated Sectors: 0
Reallocation Events: 0
Spin Retry Count: 0
Current Pending Sector Count: 0
Uncorrectable Sector Count: 0
Temperature: 43
Start/Stop Count: 16
Power-On Hours: 7905
Power Cycle Count: 16
Load Cycle Count: 32I think I found the original error. And, as I try to copy data out, it seems most of my shares are inaccessible. Thank goodness I am paranoid and have a backup of all this data. Not a happy camper.
Jul 11 07:07:44 [SERVER] kernel: WARNING: CPU: 3 PID: 2117 at fs/btrfs/extent-tree.c:3654 btrfs_start_dirty_block_groups+0x280/0x444()
Jul 11 07:07:44 [SERVER] kernel: BTRFS: Transaction aborted (error -22)
Jul 11 07:07:44 [SERVER] kernel: Modules linked in: vpd(PO)
Jul 11 07:07:44 [SERVER] kernel: CPU: 3 PID: 2117 Comm: btrfs-transacti Tainted: P W O 4.4.68.alpine.1 #1
Jul 11 07:07:44 [SERVER] kernel: Hardware name: Annapurna Labs Alpine
Jul 11 07:07:44 [SERVER] kernel: [<c0015980>] (unwind_backtrace) from [<c0012388>] (show_stack+0x10/0x14)
Jul 11 07:07:44 [SERVER] kernel: [<c0012388>] (show_stack) from [<c039a9f0>] (dump_stack+0x94/0xa8)
Jul 11 07:07:44 [SERVER] kernel: [<c039a9f0>] (dump_stack) from [<c001ffd4>] (warn_slowpath_common+0x84/0xb4)
Jul 11 07:07:44 [SERVER] kernel: [<c001ffd4>] (warn_slowpath_common) from [<c0020034>] (warn_slowpath_fmt+0x30/0x40)
Jul 11 07:07:44 [SERVER] kernel: [<c0020034>] (warn_slowpath_fmt) from [<c029e21c>] (btrfs_start_dirty_block_groups+0x280/0x444)
Jul 11 07:07:44 [SERVER] kernel: [<c029e21c>] (btrfs_start_dirty_block_groups) from [<c02b0128>] (btrfs_commit_transaction+0x154/0xc0c)
Jul 11 07:07:44 [SERVER] kernel: [<c02b0128>] (btrfs_commit_transaction) from [<c02ab2f4>] (transaction_kthread+0x1f4/0x204)
Jul 11 07:07:44 [SERVER] kernel: [<c02ab2f4>] (transaction_kthread) from [<c003a6c0>] (kthread+0xfc/0x114)
Jul 11 07:07:44 [SERVER] kernel: [<c003a6c0>] (kthread) from [<c000f368>] (ret_from_fork+0x14/0x2c)
Jul 11 07:07:44 [SERVER] kernel: ---[ end trace 3fa7939600f37903 ]---
Jul 11 07:07:44 [SERVER] kernel: BTRFS: error (device md127) in btrfs_start_dirty_block_groups:3654: errno=-22 unknown
Jul 11 07:07:44 [SERVER] kernel: BTRFS info (device md127): forced readonly
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
