RN314 with 2 EDA500, new volume sync causes system lockup
When creating a new volume on my second EDA500, the sync process will eventually cause a system lockup. My indication that this has occurred is the progress information on the LED screen of my RN314 will go dark (it goes dark between refreshes, but this is different as it no longer provides updated progress information). The LED screen itself is still functional, but no longer provides progress information. At this point, graceful power down, admin page, and log and diagnostic retrieval via RAIDar are no longer available. A hard power restart is required to regain full access to the system. Destroying the new volume at this point is the only means I have to restore stability to my NAS. If left in the locked state, I will eventually lose SSH and SMB access to the device. After a hard reset and stopping the sync, I can pull logs and diagnostic information. There are no errors reporting that I can tell. The sync process simply locks up the system and eventually results in widespread failure.
Setup:
RN314 running 1 volume over 4 x 4TB drives in RAID10; RNOS 6.10.2
EDA500 running 1 volume over 4 x 3TB drives in RAID10
Second EDA500 with no volume, but 4 x 6TB drives I'd like to run in RAID10 if I can ever get the volume created
Troubleshooting:
Restarted the volume sync several times
Replaced my eSata cable
Logs available upon request
Model: RN31400|ReadyNAS 300 Series 4- Bay (Diskless)
Checking your disks (diskinfo, smart_history) all seems to be clean and healthy. the only status log I am wary of would be both of your volume are playing on 80-90% of capacity consumed, this greatly degrades performance of the NAS. Also, quota reaches over 90%.
You may want to try disabling quota and freeing up space on both of your volumes and run volume maintenance before recreating your 3rd volume.
Re: RN314 with 2 EDA500, new volume sync causes system lockup
The EDA500 can be problematic that way because it uses eSATA port expansion. Basically, it sends the data to/from the dives through an interface designed for just one. With all the read/writes required for a sync, it can get really bogged down. That, in turn, bogs down the BTRFS processes that create the volume and can also bog down the readynasd (GUI) process. I did that to me on a 516, so I can imagine its worse on the lower-powered 314.
I always used to open an SSH session during syncs, scrubs, and balances so I could run TOP and see what was gong on if I lost GUI and/or SMB connectivity.
The good news for you is that even though readynasd sometimes reached 100% CPU usage (which I feel is a bug in this situation), so I lost all connection except via ssh, the process actually continued on and ultimately completed. Hopefully, that will happen for you as well. If the drive lights are still active, it probably is. But since a sync on an EDA500 can take an etermity, and all those bogged down processes are slowing it more, it is hard to deal with not having at least some idea of when it will complete.
Re: RN314 with 2 EDA500, new volume sync causes system lockup
So I purged my snapshots to increase available capacity and ran volume maintenance on my two existing volumes. The second volume running on my EDA500 caused the same lock-up during my first attempt at Scrubbing the volume. The second attempt to Scrub succeeded.
Once maintenance was complete, I recreated the third volume on my second EDA500. It made it 19.3% through the sync before it locked up again.
I am monitoring over ssh now (top) and I can see that there is an increased number of processes stuck in uninterruptible sleep. Most of them appear to be systemd-journal and are waiting on btrfs_sync_file. The sync process itself (md125_resync) is stuck in uninterruptible sleep as well and waiting on raise_barrier.
I'm not sure what else I can do. Would disconnecting one of the EDA500 possibly improve my chances of creating a new volume? I might try that next.
Model: RN31400|ReadyNAS 300 Series 4- Bay (Diskless)
Re: RN314 with 2 EDA500, new volume sync causes system lockup
I don't think the second EDA500 has any impact unless there are operations trying to be done concurrently on it.
It's not a good idea to disconnect and re-connect an EDA volume without doing an EXPORT and IMPORT (EXPORT is on the Volume menu, IMPORT is automatic at power-on once the volume is re-connected).
