RN314 - Every morning: volume error read only.

Hi Stephen,

firmware is 6.10.2

the data volume is about 9 TB full and 6.8 TB free.

(4 disks - 3 times 6 TB one time 10 TB)

System.log doesn't show any disk or brtfs errors.

Kernel.log does!

Here an example (but the error are going on and on after that:)

Nov 19 07:00:47 NAS kernel: md127: detected capacity change from 0 to 17988620648448
Nov 19 07:00:47 NAS kernel: Adding 1046524k swap on /dev/md1. Priority:-1 extents:1 across:1046524k
Nov 19 07:00:47 NAS kernel: BTRFS: device label 7c6e3e74:data devid 1 transid 221653 /dev/md127
Nov 19 07:01:59 NAS kernel: e1000e: eth0 NIC Link is Down
Nov 19 07:01:59 NAS kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Nov 19 07:01:59 NAS kernel: 8021q: adding VLAN 0 to HW filter on device eth0
Nov 19 07:01:59 NAS kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
Nov 19 07:01:59 NAS kernel: NFSD: starting 90-second grace period (net ffffffff88d74240)
Nov 19 07:02:01 NAS kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx
Nov 19 07:02:01 NAS kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Nov 19 07:03:03 NAS kernel: nfsd: last server has exited, flushing export cache
Nov 19 07:03:04 NAS kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
Nov 19 07:03:04 NAS kernel: NFSD: starting 90-second grace period (net ffffffff88d74240)
Nov 19 07:29:49 NAS kernel: ------------[ cut here ]------------
Nov 19 07:29:49 NAS kernel: WARNING: CPU: 3 PID: 2743 at fs/btrfs/extent-tree.c:7004 __btrfs_free_extent+0xaf4/0xb31()
Nov 19 07:29:49 NAS kernel: BTRFS: Transaction aborted (error -28)
Nov 19 07:29:49 NAS kernel: Modules linked in: vpd(PO)
Nov 19 07:29:49 NAS kernel: CPU: 3 PID: 2743 Comm: btrfs-transacti Tainted: P O 4.4.184.x86_64.1 #1
Nov 19 07:29:49 NAS kernel: Hardware name: NETGEAR ReadyNAS 314 /ReadyNAS 314 , BIOS 4.6.5 04/03/2013
Nov 19 07:29:49 NAS kernel: 0000000000000000 ffff88005b56fa90 ffffffff88363e61 ffff88005b56fad8
Nov 19 07:29:49 NAS kernel: 0000000000000009 ffff88005b56fac8 ffffffff880dd14a ffffffff88282c3d
Nov 19 07:29:49 NAS kernel: 00000000ffffffe4 ffff880023c66930 ffff88007c0e41e0 0000000000000000
Nov 19 07:29:49 NAS kernel: Call Trace:
Nov 19 07:29:49 NAS kernel: [<ffffffff88363e61>] dump_stack+0x4d/0x63
Nov 19 07:29:49 NAS kernel: [<ffffffff880dd14a>] warn_slowpath_common+0x8f/0xa8
Nov 19 07:29:49 NAS kernel: [<ffffffff88282c3d>] ? __btrfs_free_extent+0xaf4/0xb31
Nov 19 07:29:49 NAS kernel: [<ffffffff88063f94>] warn_slowpath_fmt+0x47/0x49
Nov 19 07:29:49 NAS kernel: [<ffffffff8828222f>] ? __btrfs_free_extent+0xe6/0xb31
Nov 19 07:29:49 NAS kernel: [<ffffffff88282c3d>] __btrfs_free_extent+0xaf4/0xb31
Nov 19 07:29:49 NAS kernel: [<ffffffff882dff48>] ? btrfs_merge_delayed_refs+0x60/0x43c
Nov 19 07:29:49 NAS kernel: [<ffffffff88286769>] __btrfs_run_delayed_refs+0xa78/0xcbf
Nov 19 07:29:49 NAS kernel: [<ffffffff88288537>] btrfs_run_delayed_refs+0x66/0x24f
Nov 19 07:29:49 NAS kernel: [<ffffffff88289144>] btrfs_write_dirty_block_groups+0x124/0x316
Nov 19 07:29:49 NAS kernel: [<ffffffff88288705>] ? btrfs_run_delayed_refs+0x234/0x24f
Nov 19 07:29:49 NAS kernel: [<ffffffff88298eb8>] commit_cowonly_roots+0x208/0x2ac
Nov 19 07:29:49 NAS kernel: [<ffffffff8829b36f>] btrfs_commit_transaction+0x4b5/0x9a2
Nov 19 07:29:49 NAS kernel: [<ffffffff8829bb1b>] ? start_transaction+0x2bf/0x3bc
Nov 19 07:29:49 NAS kernel: [<ffffffff882966b3>] ? btrfs_cleanup_transaction+0x458/0x458
Nov 19 07:29:49 NAS kernel: [<ffffffff88296784>] transaction_kthread+0xd1/0x1b3
Nov 19 07:29:49 NAS kernel: [<ffffffff88079cb2>] kthread+0xdc/0xe4
Nov 19 07:29:49 NAS kernel: [<ffffffff88079bd6>] ? kthread_worker_fn+0x129/0x129
Nov 19 07:29:49 NAS kernel: [<ffffffff888e6c6f>] ret_from_fork+0x3f/0x80
Nov 19 07:29:49 NAS kernel: [<ffffffff88079bd6>] ? kthread_worker_fn+0x129/0x129
Nov 19 07:29:49 NAS kernel: ---[ end trace 79a8d4758041e9eb ]---
Nov 19 07:29:49 NAS kernel: BTRFS: error (device md127) in __btrfs_free_extent:7004: errno=-28 No space left
Nov 19 07:29:49 NAS kernel: BTRFS info (device md127): forced readonly
Nov 19 07:29:49 NAS kernel: BTRFS: error (device md127) in btrfs_run_delayed_refs:2995: errno=-28 No space left
Nov 19 07:29:49 NAS kernel: BTRFS warning (device md127): Skipping commit of aborted transaction.
Nov 19 07:29:50 NAS kernel: BTRFS: error (device md127) in cleanup_transaction:1864: errno=-28 No space left
Nov 19 07:30:35 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458747142144 wanted 221668 found 221661
Nov 19 07:30:35 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458747142144 wanted 221668 found 221661
Nov 19 07:30:35 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458747142144 wanted 221668 found 221661
Nov 19 07:30:35 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458747142144 wanted 221668 found 221661
Nov 19 07:30:36 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:36 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:36 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:36 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:38 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:38 NAS kernel: BTRFS error (device md127): parent transid verify failed on 10458579861504 wanted 221668 found 221661
Nov 19 07:30:41 NAS kernel: verify_parent_transid: 10 callbacks suppressed