NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
btaroli
Jan 15, 2017Prodigy
6.6.1 Scrub Hammers CPU
And when I say hammers, I mean it runs with -1 prio and causes 6-8 kernel worker threads each vying to consume 100% of CPU. So rabid is this consumption that all other background processes, including...
Laserbait
Jul 09, 2017Luminary
Yes, scrub is running, currently 4.85% completed.
btaroli
Jul 10, 2017Prodigy
https://manpages.debian.org/stretch/procps/ps.1.en.html
-A is a synonym for -a, selecting all processes
What you seem to be missing is an option for output format. I generally use -f, but there are others.
If you only see the name of the program being run, then ALL Btrfs commands show up as "btrfs". You need to see the command arguments in order to know which function is being called, such as scrub.
So try "ps -Af"
- btaroliJul 12, 2017Prodigy
Got onto this subject in a thread on 6.7.5 but was recently reminded this one was still out there and "unanswered". Wanted to refresh this thread and get back on a single topic. Other thread was https://forum1.netgear.com/t5/ReadyNAS-Beta-Release/ReadyNAS-OS-6-Software-Version-6-7-5-Question
I was encouraged by some changes Skywalker mentioned had happened in 6.7.5, wondering if the scrub behavior may have changed. Initial attempts suggested similar impact to several services on the NAS, including DVBLink, PLEX, AV, and Time Machine.
Skywalker wrote:
@btaroli ... The scrub process already sets its ionice level to the lowest (idle) priority, although it's not obvious from normal top output since the I/O priority only gets changed on the child thread that actually does the I/O. From our testing, the CPU isn't taxed much at all by scrub, so the I/O priority is the only thing that should really matter.In testing, I found still that these processes were starved (esp for video transcode by DVBLink and PLEX), presumably for CPU. It was suggested by Michael_Oz that I install htop to get a better look, and this did help. As Skywalker indicated, btrfs scrub has a couple of threads. But htop wasn't showing these consistenly coming up as in idle ioprio class. Indeed, btrfs scrub has the -c and -n options specifically to set ioprio on the scrub. It's not clear how it's being done now, but it doesn't seem to be having the intended effect.
# btrfs scrub usage: btrfs scrub <command> [options] <path>|<device> btrfs scrub start [-BdqrRf] [-c ioprio_class -n ioprio_classdata] <path>|<device> Start a new scrub. If a scrub is already running, the new one fails. btrfs scrub cancel <path>|<device> Cancel a running scrub btrfs scrub resume [-BdqrR] [-c ioprio_class -n ioprio_classdata] <path>|<device> Resume previously canceled or interrupted scrub btrfs scrub status [-dR] <path>|<device> Show status of running or finished scrub verify checksums of data and metadata
What I found was that once I had set "idle" (ionice -c 3 -p ####) on all three btrfs scrub processes/threads THEN I began to see that PLEX, DVBLink, Time Machine were all able to complete their work normally without impact.
I was playing a bit more with htop and sorting on various columns. When I sorted reverse on IO_PRIORITY, though, I noticed something really curious and may actually expose why it's important that btrfs scrub must run fully in idle ioprio in order to ensure no impact to services. In particular, I noticed the following processes running in low priority classes:
idle btrfs scrub B7 clamd PLEX (background process) watchdog B6 DVBLink (ffmpeg) B4 kworker/u8:* DVBLink (server)
PLEX (server)
PLEX (transcoder) Dropbox Transmission md*_raid6 dbus_daemon ksoftirqd/*It looks very much like services (including clamd) are using ioprio in B6 and B7, and might get starved if something like btrfs scrub (or any other background maintenance process) wasn't running in idle class. Once btrfs scrub was confirmed (and reported by htop) has being in idle class it stopped interfering with other processes.
Indeed when I look at the values for nice and ioprio that processes are using, I wonder if some of them don't need to be ratcheted down in priority quite as far as they are. Too many things set down in B6 and B7 might created problems, no? Have to leave room for things that really do belong down there. :D Think I'll whisper over in PLEX and DVBLink forums on that subject.
- SkywalkerJul 12, 2017NETGEAR Expert
btrfs scrub has 3 threads: (1) the master thread, that just waits for completion, (2) the thread that does the actual scrub I/O work, and (3) the status update thread that wakes up every 5 seconds to grab the latest status info from the kernel. The brfs utility only sets the I/O priority (ionice) level on the I/O thread. You can verify that by looking at iotop output. Setting the other two threads to idle priority has essentially no effect. It's likely that the variations in the quality of the other services you saw after changing the I/O priority on threads 1 and 3 was coincidental. If the services getting affected are starved for CPU, then changing I/O priority isn't going to matter much.
The scrub operation of course has to do a lot of I/O. It also involves calculating checksums for every data and metadata block and compares that to the checksum stored on disk. So that can certainly chew up some CPU. That work would be done by a kworker thread, so setting nice levels or I/O priority on the btrfs scrub process isn't going to affect those processes.
It's still unclear to me what functions of DVBLink, PLEX, AV, and Time Machine are getting starved in your case. Is it transcoding? That's a CPU-intensive operation that doesn't do too much I/O, which would suggest there's contention for CPU resources. Or is it writing large files? Writing small files? Reading large files? Small files?
Related Content
NETGEAR Academy

Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!