NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
EDA500 on RN516 - Scrub very slow
This problem has been previously reported by another user on an earlier version of the OS: ReadyNAS-516-2x-EDA500-Scrub-on-EDA500-very-slow. It persists in OS6.7.5.
As scheduled, my main data volume (6x6TB XRAID=21.88TB) started a scrub and competed it in about 30 hours. My eda-1 volume (5x4TB XRAID=12.5TB) then began it's scrub. That initially locked up the NAS. The log shows eda scrub starting before data volume scrub was finished, which may be the reason for the lockup and likely extended the data volume scrub time. I need to space the out more in the schedule). I had to reboot and then manually re-started the eda scrub. It's going to take a lifetime (0.01% complete in 5 hours). Top shows CPU usage of around 4% for md125_raid5 with a priority of 20 and 2% for md125_resync.with a priority of 39.
The really off thing is that I swear the eda volume scrub was in the 20% range at lock-up even though it had only been running a bit over a day. Could it have been running faster when the data volume scrub was ongoing?
Is there anything I can do to safely speed this up?
When will Netgear address this issue?
14 Replies
Replies have been turned off for this discussion
A scrub kicks off both a btrfs scrub and an mdadm resync. 5 disks relying on 1 eSATA connection to perform extreme recalculations on 2 fronts (RAID and FS) is very intensive and is very slow. You could move your EDA500 disks to your head unit and let the operations continue, then move them back.
Does it do them concurrently? Maybe that's the issue, but 54 days still seems like a very long time. Why would runing two processes take less CPU time than me running just one via SSH? It didn't take anywhere near that long for the orignal sync, so why should a resync take that long? I'll have to kick off another and see what /proc/mdstat says about resync progress while this is going on. Maybe the two processes are fighting over access to the same area of the array and that slows them both down, but would that not also occur on the main array?
I have to admit I did not let it complete, but I did let it go more than two days to see if it was just the reported progress that was wrong.. Maybe it would have sped up at some point. After two days, I Googled how to find the progress in SSH and that's when I found the progress shown in SSH was identical to that shown in the GUI, so I then trusted the progress report and my resulting time to go estimate were accurate and aborted it.
As far as moving the array to the main chassis for this, that's just not a real solution. I keep everything I need daily access to on the main array and computer backups and such on the EDA500 (actually, now two of them).
I believe it is concurrent.
In the initial sync we can do things faster as there's not existing stuff to sync across. If you replace a disk in your EDA500 you'll find the sync to rebuild is longer than the initial sync when creating the volume.
The larger the disk capacity the longer things will take as there's more to check.54 days for a scrub does still seem like a very long time even in an EDA500.
If moving the disks to the main chassis is not practical you could find that additional main units is better than using EDA500 units for you.
I would think a volume in any of our current main units would significantly outperform one in the EDA500
Thank you so much for this post, as I was thinking I was the only one with this issue. Aparently you an I were the only ones who purchased the EDA500 :) I too am having this issue... I was likely the one you referenced as another user, but I have had this issue across multiple versions and across multiple factory defaults. In fact, after hearing that a factory default may help I purchased a RN316 + 6x8TB drives (yep quite a bit of money). This purchase was soley to backup the RN516 volumes and factory defualt it. I did this and built the volume on the EDA500 with 5x WD60EFRX drives, effectively 16TB in size. The volume created on the RN516 in less than 2 days time. Since then I have copied all my data back and been running successful, all the time thinking I have fixed the issue. It has been a few months, so I figured I should run the defrag, balance and yes, the scrub on the volumes. I started with the data volume which is 6x wd80EFRX drives (nearly 30TB) in RAID6 and the scrub completed in less than 36 hours (very fast). So I proceeded to start the scrub on the EDA500 volume (eda1). I started the scrub over 48 hours ago an it is under 4% complete and FrontView is basically useless during this process. Keep in mind this is a RAID6 volume with 6TB drives, with a total volume size of about 16TB and it is running about 100 times slower than the data volume. This is a problem with the EDA500 and I am not sure why I was unable to get a response from mdgm-ntgr, StephenB or someone else within Netgear. I still have enough space on the 316 to rsync the data, AGAIN, and remove the eda1 volume to rebuild it, but it is a terrible solution. Netgear, please comment on this issue with a workaround or a solution to make the eda500 useful. It is unfortunate that I keep mostly inconsequential data on the eda's (yep I have 2 and regret both of them), because I do not trust either of them. The other thread I opened and got no response from is the following:
And still no response from anyone at Netgear. If I run a scub on the volume on my EDA500 from SSH (btrfs scrub start /eda1), it runs 50X faster than if I run it from the GUI. That's comparable with the speed on the main array. Note that I am not using the -B option to not background the task. So what in the world is the GUI doing wrong when it starts a scrub, either scheduled or manually?
top - 00:28:48 up 180 days, 19:53, 1 user, load average: 7.23, 4.97, 4.90 Tasks: 278 total, 1 running, 277 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.2 us, 3.5 sy, 0.0 ni, 2.3 id, 93.8 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem: 16298148 total, 15713032 used, 585116 free, 235312 buffers KiB Swap: 2094844 total, 4 used, 2094840 free. 13143588 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 9332 root 20 0 0 0 0 S 8.3 0.0 637:58.07 md126_raid6 22272 root 39 19 0 0 0 D 4.0 0.0 91:25.11 md126_resync 20002 root 20 0 0 0 0 S 0.7 0.0 0:04.52 kworker/u8:10 3659 root 20 0 992516 8868 4920 S 0.3 0.1 831:17.00 leafp2p 3807 root 19 -1 2385380 105452 9828 S 0.3 0.6 272:46.17 readynasd 16698 snmp 20 0 73784 21028 3168 S 0.3 0.1 106:19.94 snmpd 17715 root 20 0 0 0 0 S 0.3 0.0 0:09.30 kworker/u8:0 20001 root 20 0 0 0 0 S 0.3 0.0 0:03.74 kworker/u8:9 22275 root 19 -1 32144 196 12 S 0.3 0.0 3:05.40 btrfs 22318 root 20 0 0 0 0 S 0.3 0.0 0:02.22 kworker/u8:1 31738 root 20 0 0 0 0 D 0.3 0.0 14:33.99 nfsd 1 root 20 0 202328 5312 3384 S 0.0 0.0 0:53.92 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:04.63 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 5:45.48 ksoftirqd/0 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H 7 root 20 0 0 0 0 S 0.0 0.0 56:41.87 rcu_sched 8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh 9 root rt 0 0 0 0 S 0.0 0.0 0:07.15 migration/0 10 root rt 0 0 0 0 S 0.0 0.0 0:41.41 watchdog/0 11 root rt 0 0 0 0 S 0.0 0.0 0:40.48 watchdog/1 12 root rt 0 0 0 0 S 0.0 0.0 0:07.94 migration/1 13 root 20 0 0 0 0 S 0.0 0.0 3:45.60 ksoftirqd/1 15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/1:0H 16 root rt 0 0 0 0 S 0.0 0.0 0:42.05 watchdog/2Notice over 93% wait and almost 0% CPU consumption. The resync process is still running and has been for over 62 hours and is currently at 3.5%.
I have no (0) installed applications and AntiVirus is disabled. In fact, after the factory reset a few months ago I have really done nothing with it.
I am willing to leave it running and provide logs and/or remote access for Netgear to troubleshoot and determine the cause. It seems this issue is now affecting more than just me.
Interresting that there is no response form Netgear on this one.
An update: I stopped the scrub and re-scheduled the start. It did re-start (from scratch, not where left off) and was going a little faster -- it was going to take days instead of weeks. Once again, I lost access to shares and admin interface, but SSH worked this time. cat showed fwbroker processes taking up a huge chunk of the CPU. I stopped the sync via SSH (btrfs scrub cancel) to see I could then access my data. Sure enough, the fwbroker tasks stopped or settled to near zero CPU and share and admin access were restored. I then resumed the scrub (btrfs scrub resume) and it is going MUCH faster, though share and admin access are still badly affected. It had taken 56 hours to scrub a little over 2Tib and in the last 12 hours has done another 4 (now only visible via SSH (btrfs scrub status) since the GUI is not aware of the scrub restart.
- Why does scrub on the EDA500 take so much longer than the main array?
- Why does scrub on the EDA500 take up so much CPU time that the NAS becomes inaccessible, yet it's not helping the scrub duration? That never happened on the array of my older Pro's running 6.x or the main array of my RN516.
- Why can't we resume a scrub via the GUI?
- Why does the resumed (GUI not in the loop) scrub work so much faster (times more like the main RN516 array)?
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
