× NETGEAR will be terminating ReadyCLOUD service by April 1st, 2023. For more details click here.
Reply

Spin-down problem and how I resolved it

vbif
Aspirant

Spin-down problem and how I resolved it

Since the Spin-down option was built in it has been never working properly. After each and every Update (wich I appreciate a lot by the way) I've been trying to activate this function always with the same result: the noflushd starts to spinning down the disks and then cancelled it or spins the disks up after a very short time:

Jun  9 13:51:25 NAS noflushd[1219]: Spinning down disk 1 (/dev/sda).
Jun  9 13:51:28 NAS noflushd[1219]: Spinning down disk 2 (/dev/sdb).
Jun  9 13:52:01 NAS noflushd[1219]: Spinning up disk 1 (/dev/sda) after 0:00:33.
Jun  9 13:52:01 NAS noflushd[1219]: Spinning up disk 2 (/dev/sdb) after 0:00:31.

So, as lazy as I was all this time, I think I have a solution now. The big problem for the spin-down function on the rn102 system is that the root file system is located on the disks (/dev/md0) itself, the /var/log is located there, the /tmp !!! is located there. So we have a problem here, there are so many processes that continuously writes and logs to the disks so that noflushd has to spin up the disks over and over again.

I do some changes to my system, and now I see that noflushd keep disks down for hours!

 

Jun 10 07:27:43 NAS noflushd[12648]: Spinning down disk 1 (/dev/sda).
Jun 10 07:27:45 NAS noflushd[12648]: Spinning down disk 2 (/dev/sdb).
Jun 10 11:02:59 NAS noflushd[12648]: Spinning up disk 1 (/dev/sda) after 3:35:14.
Jun 10 11:02:59 NAS noflushd[12648]: Spinning up disk 2 (/dev/sdb) after 3:35:11.

Hier are the changes I've made:

/etc/pam.d/frontview

#%PAM-1.0
#auth       requisite    pam_tally2.so  onerr=fail deny=5 unlock_time=300
auth       required     pam_unix.so
#account    required     pam_tally2.so
account    required     pam_unix.so

the first and the third lines were commented out, so that the /var/log/tallylog is no longer updated. I think, on my system with only access from home network I could give up a small security feature for a better power consumption.

then the /etc/rsyslog.conf was modified so that only .err messages are been writen to /var/log/...

auth,authpriv.err			/var/log/auth.log
*.err;auth,authpriv.none		-/var/log/syslog
daemon.err			-/var/log/daemon.log
daemon.*			/run/lock/daemon.log
kern.*				/run/lock/kern.log
kern.err				-/var/log/kern.log
lpr.err				-/var/log/lpr.log
mail.err				-/var/log/mail.log
user.err				-/var/log/user.log

 the daemon.log and the kern.log are now located at /run/lock wich lives in ram. I know, /run/lock is not a suitable place for logs, but it is only tmfs partitions big enough to hold the logs.  The daemon.log is usefull to track down the noflushd work. 

 

My plan is to use USB flash drive for all the logs, and to change the /tmp to the tmpfs file system so that noflushd have acctually some chance to spin down the disks.

 

I run the latest firmware 6.10.1

Model: RN102|ReadyNAS 100 Series 2- Bay
Message 1 of 6
StephenB
Guru

Re: Spin-down problem and how I resolved it

I don't have many of the log files you reference (and don't have /etc/rsyslog.conf either).  Normally the NAS logging is done with journalctl.

 

I am thinking you might have installed rsyslog manually - is that the case?

Message 2 of 6
vbif
Aspirant

Re: Spin-down problem and how I resolved it

that is an interesting point. But I haven't installed rsyslog manually, may be it came with some Netgear apps as a dependence, I did test some of them.

by the way journalctl also writes to the root aka md0

Message 3 of 6
StephenB
Guru

Re: Spin-down problem and how I resolved it


@vbif wrote:

that is an interesting point. But I haven't installed rsyslog manually, may be it came with some Netgear apps as a dependence, I did test some of them.


There is (or was) a log monitoring app that installs rsyslog.  I suggest uninstalling any apps you no longer use (and if you've done that already, manually uninstall rsyslog).  Some of them can destabilize the NAS (particularly when upgrading firmware).

 


@vbif wrote:

by the way journalctl also writes to the root aka md0


Yes of course.  But you got spindown to work without changing journalctl.  My recollection is that Netgear set up logging to defer writes (at least to some degree) in order to avoid spinup.  There are some posts where they say they said that they made some adjustments to flushd.

 

FWIW, I wouldn't have made your mods - at least not all of them.  /tmp is rarely used, so I'd leave that alone.  Writing logs to RAM sounds like a bad idea to me, as you'd likely lose them when you need them the most.  I think tallylog is normally updated when users attempt to log in - which likely will spin up the disks anyway.  So I wouldn't be inclined to fiddle with pam or tallylog either.

 

But it'd be interesting to see the results of making each mod individually (after removing rsyslog, and seeing what difference that makes).

 

 

Message 4 of 6
vbif
Aspirant

Re: Spin-down problem and how I resolved it

->There is (or was) a log monitoring app that installs rsyslog.  I suggest uninstalling any apps you no longer use (and if you've done that already, manually uninstall rsyslog).  Some of them can destabilize the NAS (particularly when upgrading firmware).

 

I don't have any apps installed, but the rsyslog was still active. For now I just deactivated it with systemctl disable rsyslog.

 

->Yes of course.  But you got spindown to work without changing journalctl.  My recollection is that Netgear set up logging to defer writes (at least to some degree) in order to avoid spinup.  There are some posts where they say they said that they made some adjustments to flushd.

 

So, without rsyslog running:

Jun 14 14:26:49 NAS noflushd[30958]: Spinning up disk 1 (/dev/sda) after 7:45:55.
Jun 14 14:26:49 NAS noflushd[30958]: Spinning up disk 2 (/dev/sdb) after 7:45:52.

that is OK

 

But hier I still have some questions

Jun 14 14:56:50 NAS noflushd[30958]: Spinning down disk 1 (/dev/sda).
Jun 14 14:56:53 NAS noflushd[30958]: Spindown of disk 1 (/dev/sda) cancelled.
Jun 14 14:56:53 NAS noflushd[30958]: Spinning down disk 2 (/dev/sdb).
Jun 14 15:09:11 NAS noflushd[30958]: Spinning up disk 2 (/dev/sdb) after 0:12:16.
Jun 14 15:32:12 NAS noflushd[30958]: Spinning down disk 1 (/dev/sda).
Jun 14 15:32:15 NAS noflushd[30958]: Spinning down disk 2 (/dev/sdb).
Jun 14 16:17:13 NAS noflushd[30958]: Spinning up disk 1 (/dev/sda) after 0:44:58.
Jun 14 16:32:15 NAS noflushd[30958]: Spinning down disk 1 (/dev/sda).

I think, it may be caused by the journalctl or some other logging daemon, at this time of the day the was nobody at home using the nas

 

->Writing logs to RAM sounds like a bad idea to me, as you'd likely lose them when you need them the most.

 

the RAM thing was actually temporary and used only for a daemon.log, I needed a log that wouldn't be written on the root system for the noflushd. At this time I have no knowledge of the journalctl.

For the tallylog - I'm sure it was updated due to some cron jobs or some other activity from readynasd, The configuration for tallylog is in the pam.d file for 'frontview' service. I don't know what is it, but the tallylog was written without any user been logging in.

 

So my suggestion is sitll to transfer all kind of system stuff aka logging to the USB and the /tmp to the tmpfs drive. 

Message 5 of 6
StephenB
Guru

Re: Spin-down problem and how I resolved it


@vbif wrote:

I think, it may be caused by the journalctl or some other logging daemon, at this time of the day the was nobody at home using the nas

At this time I have no knowledge of the journalctl.


It's easy to confirm that, just run journalctl from ssh and look for log entries made at about those times.  Journalctl is easy to use, just google it and you'll find several guides.  There are other possibilities (snapshots being taken being one of several).

 

I get that spindown/spinup often does give you fairly short spindown times - and FWIW I think Netgear could do a better job of testing for regressions in this area when they update firmware.

 

But is there a reason that you need to be so aggressive on eliminating all spinups?  The changes you are making might cause you problems when you update firmware (or maybe even when hotfixes are applied).  If you're lucky they will just be overwritten by the new firmware from time to time.  And you'll still end up with the USB drive spinning up.

 

Message 6 of 6
Top Contributors
Discussion stats
  • 5 replies
  • 1188 views
  • 0 kudos
  • 2 in conversation
Announcements