NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
CarlEdman
Jan 19, 2014Luminary
"No logs exist" and things get worse
I have been running a ReadyNAS Ultra 6 Plus (6x2 TByte in X-Raid 2 mode) for some years with few problems and much success. The device was always upgraded to the latest RAIDiator and has been running 4.2.25 since whenever that was released. A few non-standard services have been running for at least a few months with problems: SSH, Plex Media Server, and the CrashPlan backup program. There have been no configuration changes for at least a few months.
Today, for the first time, I have encountered a serious problem. First, when logging myself into ReadyNAS FrontView, I got a notice of new log items, but when I went to the logs, the page indicated that "No logs exist." According to the advice on this forum, I attempted to reboot the server from the admin page, but that did not have any effect. So I tried to remember where the server was physically located, found it, and did a reboot by pressing the power button twice (i.e., a soft reboot).
When the server was rebooted things were worse, rather than better. Now I could no longer open any of the SMB shares from any of my home workstations. Logging myself into FrontView showed still no logs, but neither was there any error. All the shares were still listed under Shares/Share Listing. Status/Health showed all green lights. The only item which was even close to being out of the safe range was Temp Sys at 57 degrees Celsius, but that might always have been around that level.
When I ssh-ed into the server, things appeared fine. The underlying data files in the filesystem appear still to be there. I did a 'ps -edalf' (I'll be damned if I can remember what all those switches mean, but they just came from muscle memory from my days as UN*X sysadmin). Nothing suspicious there at first glance, but I've appended the results below.
Does anybody have any help or advice? I'd hate to nuke the settings, reconfigure everything, and reinstall and reconfigure my addons (if I even remember how exactly to do so).
Today, for the first time, I have encountered a serious problem. First, when logging myself into ReadyNAS FrontView, I got a notice of new log items, but when I went to the logs, the page indicated that "No logs exist." According to the advice on this forum, I attempted to reboot the server from the admin page, but that did not have any effect. So I tried to remember where the server was physically located, found it, and did a reboot by pressing the power button twice (i.e., a soft reboot).
When the server was rebooted things were worse, rather than better. Now I could no longer open any of the SMB shares from any of my home workstations. Logging myself into FrontView showed still no logs, but neither was there any error. All the shares were still listed under Shares/Share Listing. Status/Health showed all green lights. The only item which was even close to being out of the safe range was Temp Sys at 57 degrees Celsius, but that might always have been around that level.
When I ssh-ed into the server, things appeared fine. The underlying data files in the filesystem appear still to be there. I did a 'ps -edalf' (I'll be damned if I can remember what all those switches mean, but they just came from muscle memory from my days as UN*X sysadmin). Nothing suspicious there at first glance, but I've appended the results below.
Does anybody have any help or advice? I'd hate to nuke the settings, reconfigure everything, and reinstall and reconfigure my addons (if I even remember how exactly to do so).
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
4 S root 1 0 0 80 0 - 517 poll_s 08:45 ? 00:00:00 init [2]
1 S root 2 0 0 80 0 - 0 kthrea 08:45 ? 00:00:00 [kthreadd]
1 S root 3 2 0 80 0 - 0 run_ks 08:45 ? 00:00:00 [ksoftirqd/0]
1 S root 6 2 0 -40 - - 0 cpu_st 08:45 ? 00:00:00 [migration/0]
1 S root 7 2 0 -40 - - 0 cpu_st 08:45 ? 00:00:00 [migration/1]
1 S root 8 2 0 80 0 - 0 worker 08:45 ? 00:00:00 [kworker/1:0]
1 S root 9 2 0 80 0 - 0 run_ks 08:45 ? 00:00:00 [ksoftirqd/1]
1 S root 10 2 0 80 0 - 0 worker 08:45 ? 00:00:02 [kworker/0:1]
1 S root 11 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [khelper]
1 S root 158 2 0 80 0 - 0 bdi_sy 08:45 ? 00:00:00 [sync_supers]
1 S root 160 2 0 80 0 - 0 bdi_fo 08:45 ? 00:00:00 [bdi-default]
1 S root 162 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kblockd]
1 S root 163 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kacpid]
1 S root 164 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kacpi_notify]
1 S root 165 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kacpi_hotplug]
1 S root 272 2 0 80 0 - 0 hub_th 08:45 ? 00:00:00 [khubd]
1 S root 279 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [md]
1 S root 380 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [rpciod]
1 S root 406 2 0 80 0 - 0 kswapd 08:45 ? 00:00:04 [kswapd0]
1 S root 471 2 0 80 0 - 0 fsnoti 08:45 ? 00:00:00 [fsnotify_mark]
1 S root 473 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [aio]
1 S root 489 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [nfsiod]
1 S root 505 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [crypto]
1 S root 522 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kthrotld]
1 S root 567 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [scsi_tgtd]
1 S root 599 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_0]
1 S root 602 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_1]
1 S root 605 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_2]
1 S root 608 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_3]
1 S root 611 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_4]
1 S root 614 2 0 80 0 - 0 scsi_e 08:45 ? 00:00:00 [scsi_eh_5]
1 S root 620 2 0 80 0 - 0 worker 08:45 ? 00:00:00 [kworker/u:5]
1 S root 627 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [bond0]
1 S root 648 2 0 80 0 - 0 worker 08:45 ? 00:00:00 [kworker/1:1]
1 S root 692 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kstriped]
1 S root 695 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [ksnapd]
1 S root 696 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kondemand]
1 S root 697 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [kconservative]
1 S root 700 2 0 60 -20 - 0 rescue 08:45 ? 00:00:00 [usbhid_resumer]
5 S root 830 1 0 80 0 - 450 poll_s 08:45 ? 00:00:00 ifplugd -i eth1
1 S root 839 2 0 80 0 - 0 md_thr 08:45 ? 00:00:02 [md0_raid1]
1 S root 846 2 0 80 0 - 0 md_thr 08:45 ? 00:00:00 [md1_raid6]
1 S root 852 2 0 80 0 - 0 md_thr 08:45 ? 00:00:01 [md2_raid5]
1 S root 882 2 0 80 0 - 0 kjourn 08:45 ? 00:00:00 [kjournald]
5 S root 921 2 0 80 0 - 0 bdi_wr 08:46 ? 00:00:00 [flush-9:0]
5 S root 941 1 0 76 -4 - 586 poll_s 08:46 ? 00:00:00 udevd --daemon
1 S root 1994 1 0 80 0 - 424 poll_s 08:46 ? 00:00:00 /usr/sbin/acpid -c /etc/acpi/events -s /var/run/acpid.soc
5 S root 2000 1 0 80 0 - 422 poll_s 08:46 ? 00:00:00 /usr/sbin/raidard
5 S root 2048 1 0 80 0 - 1365 poll_s 08:46 ? 00:00:00 /usr/sbin/sshd
1 S root 2055 2 0 60 -20 - 0 rescue 08:46 ? 00:00:00 [kdmflush]
1 S root 2089 2 0 80 0 - 0 kjourn 08:46 ? 00:00:00 [jbd2/dm-0-8]
1 S root 2090 2 0 60 -20 - 0 rescue 08:46 ? 00:00:00 [ext4-dio-unwrit]
1 S root 2170 1 0 80 0 - 667 skb_re 08:46 ? 00:00:00 /usr/sbin/quota_nld
5 S daemon 2175 1 0 80 0 - 453 poll_s 08:46 ? 00:00:00 /sbin/portmap
5 S root 2222 1 0 80 0 - 437 poll_s 08:46 ? 00:00:00 /sbin/syslogd -m 0
5 S daemon 2231 1 0 80 0 - 489 hrtime 08:46 ? 00:00:00 /usr/sbin/atd
5 S root 2236 1 0 80 0 - 425 syslog 08:46 ? 00:00:00 /sbin/klogd -x -c 3
5 S root 2248 1 0 80 0 - 940 poll_s 08:46 ? 00:00:00 /usr/sbin/cupsd
5 S 21 2253 1 0 80 0 - 688 poll_s 08:46 ? 00:00:00 /usr/bin/dbus-daemon --system
1 S root 2255 1 0 80 0 - 494 hrtime 08:46 ? 00:00:00 /usr/sbin/cron
5 S root 2433 1 0 80 0 - 3725 poll_s 08:46 ? 00:00:00 /usr/sbin/smbd -D
5 S root 2517 2433 0 80 0 - 3725 poll_s 08:46 ? 00:00:00 /usr/sbin/smbd -D
5 S nobody 2658 1 0 80 0 - 2864 poll_s 08:46 ? 00:00:00 proftpd: (accepting connections)
5 S root 2665 1 0 80 0 - 431 poll_s 08:46 ? 00:00:00 ifplugd -i bond0 -q -f -d0 -w -I
5 S root 2695 1 0 80 0 - 431 poll_s 08:46 ? 00:00:00 ifplugd -i eth0 -q -f -u0 -d0 -w -I
5 S root 2712 1 0 80 0 - 431 poll_s 08:46 ? 00:00:00 ifplugd -i eth1 -q -f -u0 -d0 -w -I
5 S root 2806 1 0 80 0 - 453 poll_s 08:46 ? 00:00:00 /sbin/udhcpc -i eth0 -H Storion -t 3 -p /var/run/udhcpc.e
5 S root 2894 1 0 80 0 - 784 hrtime 08:46 ? 00:00:00 /frontview/bin/monitor_enclosure
5 S root 2943 1 0 85 5 - 9136 poll_s 08:46 ? 00:00:00 /usr/sbin/apache-ssl -f /etc/frontview/apache/httpd.conf
5 S root 2951 1 0 80 0 - 2704 poll_s 08:46 ? 00:00:00 nmbd -D
1 S root 2953 2951 0 80 0 - 2615 pipe_w 08:46 ? 00:00:00 nmbd -D
5 S root 2972 1 0 80 0 - 4835 poll_s 08:46 ? 00:00:00 /usr/sbin/minidlna
5 S root 2984 1 0 80 0 - 531 poll_s 08:46 ? 00:00:00 /usr/sbin/ntpd
5 S root 2999 1 0 80 0 - 424 poll_s 08:46 ? 00:00:00 /usr/sbin/upnpd eth0
5 S root 3017 1 0 80 0 - 444 poll_s 08:46 ? 00:00:00 /usr/sbin/lld2d eth0
4 S root 3031 1 0 80 0 - 50556 futex_ 08:46 ? 00:00:18 /c/.plex/plexserver
5 S root 3299 1 0 80 0 - 3906 hrtime 08:46 ? 00:00:00 /usr/sbin/mt-daapd
4 S root 3462 3031 0 95 15 - 26828 pipe_w 08:47 ? 00:00:13 Plex Plug-in [com.plexapp.system] /c/.plex/Library/Applic
1 S admin 3509 1 0 78 -2 - 481 poll_s 08:47 ? 00:00:00 /lib/nut/hidups -x lowbatt_pct=0 /dev/usb/hiddev0
1 S admin 3553 1 0 78 -2 - 483 poll_s 08:47 ? 00:00:00 /sbin/upsd
5 S root 3581 1 0 78 -2 - 440 hrtime 08:47 ? 00:00:00 /sbin/upsmon -p
0 S root 3665 1 0 80 0 - 425 n_tty_ 08:47 ttyS0 00:00:00 /sbin/getty -L ttyS0 9600 tty1
4 S root 3797 3031 0 80 0 - 16207 pipe_w 08:48 ? 00:00:03 Plex Plug-in [com.plexapp.agents.wikipedia] /c/.plex/Libr
4 S root 3799 3031 0 80 0 - 16429 pipe_w 08:48 ? 00:00:04 Plex Plug-in [com.plexapp.agents.themoviedb] /c/.plex/Lib
1 S root 3919 2 0 80 0 - 0 worker 08:54 ? 00:00:00 [kworker/u:0]
0 S root 5071 1 0 99 19 - 173227 futex_ 09:24 ? 00:00:05 /usr/bin/java -Dfile.encoding=UTF-8 -Dapp=CrashPlanServic
1 S root 5135 2 0 80 0 - 0 worker 09:27 ? 00:00:00 [kworker/0:2]
1 S root 5148 2 0 80 0 - 0 bdi_wr 09:27 ? 00:00:00 [flush-253:0]
5 S admin 5477 2943 0 85 5 - 9381 poll_s 09:30 ? 00:00:00 /usr/sbin/apache-ssl -f /etc/frontview/apache/httpd.conf
5 S admin 6035 2943 0 85 5 - 9513 poll_s 09:34 ? 00:00:02 /usr/sbin/apache-ssl -f /etc/frontview/apache/httpd.conf
4 S root 7262 3031 0 80 0 - 16609 pipe_w 09:47 ? 00:00:03 Plex Plug-in [com.plexapp.agents.localmedia] /c/.plex/Lib
4 S root 7307 3031 0 80 0 - 16760 pipe_w 09:47 ? 00:00:01 Plex Plug-in [com.plexapp.agents.movieposterdb] /c/.plex/
4 S root 7347 3031 0 80 0 - 15762 pipe_w 09:47 ? 00:00:01 Plex Plug-in [com.plexapp.agents.opensubtitles] /c/.plex/
4 S root 7504 3031 0 80 0 - 15836 pipe_w 09:48 ? 00:00:02 Plex Plug-in [com.plexapp.agents.lastfm] /c/.plex/Library
5 S root 7967 2433 0 80 0 - 3780 poll_s 09:52 ? 00:00:00 /usr/sbin/smbd -D
4 R root 8174 2048 0 80 0 - 2063 - 09:54 ? 00:00:00 sshd: root@pts/0
4 S root 8180 8174 0 80 0 - 1245 wait 09:54 pts/0 00:00:00 -bash
4 R root 8282 8180 0 80 0 - 1105 - 09:55 pts/0 00:00:00 ps -edalf
24 Replies
Replies have been turned off for this discussion
- StephenBGuru - Experienced UserHow full is the OS partition?
- CarlEdmanLuminaryJust a single partition (I think--I certainly never created any additional partitions) with reported 9 TByte of usable space 49% full.
- CarlEdmanLuminarySince the last report, my ReadyNAS has continued to deteriorate without my intervention.
Now, on the health status page, there is listed as malfunctioning:Fan CAS 0 RPM Out of Spec
Temp AUX -17.5 C / 0 F [Normal 0-0 C / 32-32 F] Out of Spec
I do not recall seeing either of those items listed at all previously.
Moreover, Frontview is failing left and right.
For example, my timezone has changed to being listed as "GMT -11:00 Samoa" and the NTP synchronization has been turned off. Correcting these setting and hitting apply just results in them being reset to the incorrect values.
Now, under Services/Standard File Protocols all the protocols are suddenly listed as disabled. Click the checkbox to enable followed by apply, just resets the buttons to off.
Finally, under Volumes/Volume Settings/RAID Settings, the listing has changed to:Configuration: RAID Level X-RAID2, 6 disks
Status: Dead. This volume is no longer available.
Is my ReadyNAS dead? - CarlEdmanLuminaryFor what it is worth, after a reboot, the system lists the RAID as live again (for now), but all the other symptoms remain.
- mdgm-ntgrNETGEAR Employee Retired
# df -i
# df -h - CarlEdmanLuminaryI'd try, except the ReadyNAS won't accept ssh logons any more.
- mdgm-ntgrNETGEAR Employee RetiredThe 4 GB OS partition is separate to the data volume. This is likely full.
I think you should contact tech support. - CarlEdmanLuminaryThanks, mdgm. Will do.
- CarlEdmanLuminaryUpdate on resolution: After some trials and tribulations, tech support were able to log themselves in remotely and fix the problem.
Tech support did not say what the problem was, but said they moved a bunch of /tmp files to a separate partition. Reviewing these tmp files turns up a huge number of jna[0-9]*.tmp files inside tmp. Each was 136 kBytes, but they totaled 2.2 gBytes. My guess is that some process went rogues and just spammed these tmp files until the system partition was filled up and everything started going to hell. I could have fixed that myself had I known that this was the issue before things go so bad that I could no longer ssh myself into the ReadyNAS. - fastfwdVirtuoso
CarlEdman wrote: a huge number of jna[0-9]*.tmp files .... My guess is that some process went rogue
Crashplan is written in Java, right? I'd look there.
Related Content
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!