× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

RN104 randomly freezing since update to 6.2.0 (curr. 6.2.2)

chri6020
Aspirant

RN104 randomly freezing since update to 6.2.0 (curr. 6.2.2)

Hello,

since i upgraded my RN104 to OS 6.2.X the NAS randomly freezes every couple of days (no front-end, no SSH and no reply to pings). The only way to get access again to the RN104 is disconnecting / connect the powercable.
First I suspected that the freezes are related to my backup jobs as I had an issue with NTFS (see http://www.readynas.com/forum/viewtopic.php?f=7&t=78879) but I did change the backup HDDs to EXT4, ran the backup jobs manually without any problems and disabled them afterwards to exclude this cause. Still the RN104 freezes randomly.

One big issue is that I cannot find any clue in the log files. I checked them several times after the crashes and the last entries correspond to some cron jobs being executed. The following job caught my attention because usually the system crashes happen sometimes after this one has been executed:


Dec 31 08:09:01 nas /USR/SBIN/CRON[9005]: (root) CMD ( [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -depth -mindepth 1 -maxdepth 1 -type f -ignore_readdir_race -cmin +$(/usr/lib/php5/maxlifetime) ! -e
xecdir fuser -s {} 2>/dev/null \; -delete)

According to my research there have been some issues regarding this job and ubuntu / debian systems:



Due to that and after checking out the forum, where I came across this post viewtopic.php?f=65&t=79141 which mentions a possible memory leak, I decided to setup a cron job to monitor the memory usage.
The log indicates that something strange happens as the swap usage is slowly increasing and but I cannot find any processes which are acutally using swap space (bash: for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | grep kB). Though the swap usage is not very high when the system crash happens (last crash after TIME:08:10:01 mark):

TIME:08:08:01
total used free shared buffers cached
Mem: 507188 482900 24288 0 40232 165996
-/+ buffers/cache: 276672 230516
Swap: 523964 144 523820

TIME:08:09:01
total used free shared buffers cached
Mem: 507188 483980 23208 0 40304 166104
-/+ buffers/cache: 277572 229616
Swap: 523964 144 523820

TIME:08:10:01
total used free shared buffers cached
Mem: 507188 483396 23792 0 40376 166212
-/+ buffers/cache: 276808 230380
Swap: 523964 144 523820

TIME:08:27:01
total used free shared buffers cached
Mem: 507188 262836 244352 0 6656 143568
-/+ buffers/cache: 112612 394576
Swap: 523964 0 523964

TIME:08:28:03
total used free shared buffers cached
Mem: 507188 485304 21884 0 4692 205772
-/+ buffers/cache: 274840 232348
Swap: 523964 0 523964

TIME:08:29:01
total used free shared buffers cached
Mem: 507188 482048 25140 0 9584 195600
-/+ buffers/cache: 276864 230324
Swap: 523964 0 523964

TIME:08:30:01
total used free shared buffers cached
Mem: 507188 459456 47732 0 3784 184444
-/+ buffers/cache: 271228 235960
Swap: 523964 4 523960


In addition I monitor the running processes of the system but I could not find anything yet. If required I can attach log files.

Today I came across this post viewtopic.php?f=65&t=79284, which states that others have similar issues. I did not reinstall the OS nor did I reset the system to factory defaults as I do not want to ''loose'' my data, but according to the post it did not have the desired effect. I never experienced those issues while using readynas OS 6.1.X.

System details:
Services: SMB, HTTP, HTTPS, SSH, ReadyDLNA, Anti-Virus
Disks: 2x4 TB Segate disks (ST400VN000-1H4168) with RAID 1 and checksums are enabled
Shares: 7 shares, 4 with bit-rot protection and 3 without. No snapshots
Third party Apps: Anti-Virus Plus, SMB Plus, Syslog Server, linux-dash, LogAnalyzer (currently disabled)
Network: Bound Adaptive Load Balancing, WOL enabled
Additional: 4 users and UPS protected, all backup jobs disabled, no external drives, no disk spindown, no power scripts, no clean/defrag/sync jobs

My plan for now is that I will monitor the system behaviour for some more weeks. First I'll run a mem-test from the Boot-Menu (maybe also a Disk-Test) to check for bad RAM. I'll also disable all services except SMB, HTTPS and SSH and see if this has any effect. I could also disable bit-rot protection for all drives and checksums to check if it helps. If any of the mentioned measure fix the issue and I have a stable system for at least 2 weeks I will step by step enable the deactivated functionality and to see what could cause the issue.
Last but not least I could uninstall all third party apps to clean up the system or do a factory reset and reinstall the OS from Boot-Menu to see if things get better, but as mentioned before I do not really want to do this and it seems as it has no effect. I also could grant remote access to my RN104 if that helps to figure out the problem.

I am willing to help to resolve this issue within certain limits, but I also think that netgear should put some effort into reproducing and resolving the issue asap.

Otherwise I would like to go back to OS 6.1.9 until the issues are resolved as me and my family actually are relying on the data on the NAS. How can I downgrade the system?
Message 1 of 14
L1zard
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Hi,
I have RN102 with 6.2.2 and I have got similar problem. NAS freeezes suddenly after 2-3 days uptime. Tommorow I can put log records here, but they are very uninformative. With 6.1.9 (or 6.2.0?) It worked properly.
I have transmission intalled (active) and vpnserver, plex, istat (unactive). Antivirus is off.
Message 2 of 14
mdgm-ntgr
NETGEAR Employee Retired

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

L1zard, can you send me your logs (see the Sending Logs link in my sig)?
Message 3 of 14
L1zard
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Logs were sent via private message.

This problem looks very similar to problems described by different forum members in topic:

http://www.readynas.com/forum/viewtopic.php?f=20&t=78697
Message 4 of 14
tommylommytommy
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Hi.

I'm having the same problems as alot of threads now describe. I had 6.1.9 on my RN104 since it came out and everything has worked flawlessly, untill i folishly updated to 6.2.2.

Then it took like 30 minutes and it froze, power-button not responding, only way to reboot was to pull power-cord. So it came back up and everything seemed fine for a few minutes til next freeze came. So when i pull the power, it works for a short period of time... as long as i copy data or 'do something' with the nas, it works. But at soon as i let it idle it freezes up. I've tried downgrading to 6.2.1 and 6.2.0, since it told me i couldn't downgrade to 6.1.9 when i tried that, but nothing helps.

Volume is about 80% full, no apps installed, no snapshots, using buildt inn ftpserver and sambashares.

Edit: I'll try turning of bitrot & cow which i've been reading about...

😞

Tommy.
Message 5 of 14
mdgm-ntgr
NETGEAR Employee Retired

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Can you send your logs too?
Message 6 of 14
tommylommytommy
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Turning off bitrot protection didn't help....

I'll send the logs when i get home from work, mdgm.
Message 7 of 14
tommylommytommy
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

mdgm: Logs sent.

Came home from work, it had frozen.
Pulled power to reboot.
Cleared logs.
Waited untill it froze up again.
Pulled power to reboot again.
Exported logs.

btw i know i have over 80% full, but i have 700 GB free and this was not causing a problem on 6.1.9.

🙂

Edit: forgot the logs in the mail hehe, resent with logs.
Message 8 of 14
mdgm-ntgr
NETGEAR Employee Retired

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

tommylommytommy I notice you have two WD Green disks installed which are not on the compatibility list.

Two of your disks, one of the WD RED disks and also your WD20EARS have huge Load Cycle Counts.

Perhaps the issue you are facing is disk related?

Are you able to power down, remove your disks (label order), connect them to your PC and test them using WD Data LifeGuard Diagnostics?
Message 9 of 14
chri6020
Aspirant

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Regarding my posts this thread: http://www.readynas.com/forum/viewtopic.php?f=20&t=78697&start=75

chri6020 wrote:
chri6020 wrote:
Yesterday I set /proc/sys/kernel/panic and /proc/sys/kernel/panic_on_oops to restart my RN104 automatically if a kernel issue occures. Today I checked the logs and the system did reboot, which points out that this is most likely a kernel issue.
Today I'll just set the /proc/sys/kernel/panic to see if it's a kernel panic or a oops. How can I access the serial console (/dev/ttyS0) to attache a remote system to monitor the kernel issue (connector + pin description)?


Just check because my system was frozen once more. It rebootet on the 13.01.14 with the only /proc/sys/kernel/panic set and froze once more today. Thus for me it seems that the kernel has an issue.


I still got no reply on how to monitor the seriel console to see why the kernel is panicking. Any help?

Also I sent my most recent logs (today after the reboot) as described by mdgm .
Message 10 of 14
mdgm-ntgr
NETGEAR Employee Retired

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

Message 11 of 14
sparhawk765
Tutor

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

I'm having the same issue. Ever since I upgraded to 6.2.0 my 104 with 4 disks in a RAID0 had begun to freeze randomly forcing me to pull the power to restart it.

No response from webUI, SSH or ping. It "seems" to happen more often when under I/O-load like reading or writing files to it.

I found this thread and installed the 6.2.3 Beta today and proceeded to filling the 104 with some files, this is from clean upgrade + factory reset. 20-30 min in the box froze so back to the drawingboard.
Message 12 of 14
mdgm-ntgr
NETGEAR Employee Retired

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

RAID-0 is not recommended at all. If a disk fails all the data on the RAID-0 volume using that disk is lost. Have you tried using e.g. X-RAID?
Message 13 of 14
sparhawk765
Tutor

Re: RN104 randomly freezing since update to 6.2.0 (curr. 6.2

I'm aware that raid-0 offers no redundancy and neither does JBOD which I used before that.
The data I'm storing on it is not that critical.
Message 14 of 14
Top Contributors
Discussion stats
  • 13 replies
  • 2628 views
  • 0 kudos
  • 5 in conversation
Announcements