× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: RN312 “stops working”

WildfireTech
Guide

RN312 “stops working”

So, this has happened: this week the ReadyNAS just “stops working”. I can log into the admin webpage, and everything looks perfectly fine and there are no errors or any other loud warnings.

However, at the same time, neither my Mac Mini, my MacBook, my wife’s Windows laptop, my iPad, my android tablet, nor my iPhone can see the device. It’s “working just fine“ as long as you don’t want to access any data (so what’s the point, really?).

Anyone else seen this type of behavior? This seems to have escalated about two weeks ago.

Model: RN31200|ReadyNAS 300 Series 2- Bay (Diskless)
Message 1 of 16
StephenB
Guru

Re: RN312 “stops working”

What firmware are you running?

 


@WildfireTech wrote:

However, at the same time, neither my Mac Mini, my MacBook, my wife’s Windows laptop, my iPad, my android tablet, nor my iPhone can see the device. It’s “working just fine“ as long as you don’t want to access any data (so what’s the point, really?).

Perhaps download the log zip file, and see if there are any errors related to Samba.  You can send the mods ( @JohnCM_S or @Marc_V ) a private message (PM) with a downloadable link the log zip, and ask them to analyze it.  Do that using the envelope icon in the upper right of the forum page.  Don't post the zip publicly.

 

Are all these devices accessing with SMB?  If not, what protocols are you using?  When the problem occurs, does it always affect all devices? Or does it just affect some?

 

Also, on the Windows laptop - have you tried entering \\NAS-IP-ADDRESS in the file address bar when you can't "see" the NAS?  Use the real IP address for the NAS of course (and be careful to use the correct slash direction).  It'd be helpful to understand if this is a discovery issue, or an access issue.

 

 

Message 2 of 16
WildfireTech
Guide

Re: RN312 “stops working”

Thanks for the reply.  We are using SMB (for windows) and AFS (for the Macs).  I have looked at the log files and they don't show any errors that I can see, but I will upload them to the devs as you recommended.

 

As for the access, we are always using the IP address for connections as I haven't set up a DNS for the house that includes the NAS so that you have to know it is there to access it (security by obscurity).

Message 3 of 16
StephenB
Guru

Re: RN312 “stops working”


@WildfireTech wrote:

 

As for the access, we are always using the IP address for connections 


Ok.  On the Windows system, are you entering the IP address in the file explorer address bar?  Or are you looking for an icon in the left pane under "network"?

Message 4 of 16
WildfireTech
Guide

Re: RN312 “stops working”

Using the IP address on all systems, at all times.  No Finder lookups, no Explorer lookups.

Message 5 of 16
JohnCM_S
NETGEAR Employee Retired

Re: RN312 “stops working”

Hi WildfireTech,

 

Can you send to us the NAS system logs via PM (private message)? We will check what is causing the issue.

 

Regards,

Message 6 of 16
StephenB
Guru

Re: RN312 “stops working”


@WildfireTech wrote:

Using the IP address on all systems, at all times.  No Finder lookups, no Explorer lookups.


So how are you accessing the shares from the Windows PC?

Message 7 of 16
WildfireTech
Guide

Re: RN312 “stops working”

Access via Windows over the SMB shares with IP address direction (i.e. M: drive is 10.0.0.xxx mapped).

 

So, the "same thing but different" just happened; RN321 just totally went offline, but stayed up. Couldn't connect to web admin.  Shares that were mounted dropped.  Couldn't connect new shares.  iPhone app couldn't communicate with it.  Didn't respond to power button depress.  Had to pull the freaking power cord to get the unit to shut down.

 

Rebooted, NO Errors, NO Warnings, NO messages of any import to review.  The unit just "went on holiday" with no record, save the work I lost in trying to save data to this blasted thing.

 

I will send the logs as directed above, but I have little doubt that there is anything that anyone will ever find since the "admin console" flags no errors.

Message 8 of 16
StephenB
Guru

Re: RN312 “stops working”


@WildfireTech wrote:

Access via Windows over the SMB shares with IP address direction (i.e. M: drive is 10.0.0.xxx mapped).

 

So, the "same thing but different" just happened; RN312 just totally went offline, but stayed up. Couldn't connect to web admin.  Shares that were mounted dropped.  Couldn't connect new shares.  

Obviously more than the web admin interface went down.  I'd look for a disk issue first, so maybe look at overall disk health in disk_info.log and look for disk or btrfs errors in system.log and kernel.log.

 

What apps are you running?

 

Did you look for a full OS partition in the logs?  Do this by looking in volume.log and scrolling down to the === df -h === section.

=== df -h ===
Filesystem      Size  Used Avail Use% Mounted on
udev             10M  4.0K   10M   1% /dev
/dev/md0        4.0G 1002M  2.8G  27% /

/dev/md0 is the OS partition, and normally it is about 25-30% full.

 

Perhaps enable ssh, and see if you can connect that way when the problem happens again.

Message 9 of 16
WildfireTech
Guide

Re: RN312 “stops working”

Apps running :

  • MySql
  • "Contacts and Calendars Server"
  • "linux-dash"
  • "phpMyAdmin"

Services "on" are:

  • SMB
  • AFP
  • NFS
  • FTP
  • iTunes
  • HTTP
  • HTTPS
  • SSH
  • Antivirus
  • File Search

No issues, errors, flags, or warnings in the disk_info.log

No "disk" or "brtfs" errors in the system.log

No "disk" or "brtfs" errors in the kernel.log

 

In volumne.log (md0 not full but is a lot fuller than you stated):

       /dev/md0 4.0G 3.0G 662M 83% /

 

Should I look to do some disk cleanup?  What would be using the root filesystem that I could purge?

Message 10 of 16
StephenB
Guru

Re: RN312 “stops working”


@WildfireTech wrote:

Apps running :

  • MySql
  • "Contacts and Calendars Server"
  • "linux-dash"
  • "phpMyAdmin"

 

In volume.log (md0 not full but is a lot fuller than you stated):

       /dev/md0 4.0G 3.0G 662M 83% /

 

Should I look to do some disk cleanup?  What would be using the root filesystem that I could purge?


83% is quite full, and could interfere with firmware updates.  Though it shouldn't cause a hang in normal operation.  Still, it should be looked at.

 

Likely some of these apps are installed on the OS partition.  If that's the case, the usual approach is to copy the folders to the data volume (generally into the existing /data/.apps folder) and replace the original folder with a soft link to the new one.

 

There are some other possibilities on space, so report back on the usage after you've moved the apps to the data volume.

 


@WildfireTech wrote:

Apps running :

  • MySql
  • "Contacts and Calendars Server"
  • "linux-dash"
  • "phpMyAdmin"

Services "on" are:

  • SMB
  • AFP
  • NFS
  • FTP
  • iTunes
  • HTTP
  • HTTPS
  • SSH
  • Antivirus
  • File Search

 


Do you need all these services on the NAS?  You probably should disable ones you aren't using (as they will take some memory resources).  From a troubleshooting perspective, it is possible that the apps or maybe the services are causing or triggering the problem.  

 

What firmware are you running?

Message 11 of 16
WildfireTech
Guide

Re: RN312 “stops working”

I did some du-ing this morning, and this is what I'm finding (everythingn except /data and /var which are both mounted on the RAID)

 

727M    /apps

6.4M    /bin

4.0K    /dev

11M     /etc

30M     /frontview

4.0K    /homes

33M     /lib

4.0K    /lib64

4.6M    /opt

32K     /root

16M     /run

11M     /sbin

5.4M    /tmp

350M    /usr

 

With some back-of-the-napkin calculations, this is around 1.2GB.  However, / has 3.1GB used.

 

Any ideas on why the OS is finding a 300% usage?

 

Message 12 of 16
StephenB
Guru

Re: RN312 “stops working”

/apps and /home are mount points to the data volume, so the information there might be misleading.

 

It's best to do this by remounting to OS partition as /mnt, and then analyze space there.  Among other things, that lets you see data hidden under the mount points.

# mount --bind / /mnt

 

If you connect USB drives to the NAS, then look in /mnt/media, which can sometimes have a lot of files that shouldn't be there.

 

Also, the OS file system is btrfs, so there are aspects of free space that du and df won't tell you.  You should also use the btrfs commands here.  For instance, btrfs fi du, and btrfs fi df.

 

Then when done, you unmount with

# umount /mnt

 

 

Message 13 of 16
WildfireTech
Guide

Re: RN312 “stops working”

*sigh*. happened again.  NAS responds to ping, but cannot get the admin web system up, cannot SSH, cannot SFTP to it, cannot mount any drives.

 

Guess it's time to restart it again and see what I can figure out.

Message 14 of 16
WildfireTech
Guide

Re: RN312 “stops working”

Okay, here's the update:

 

     Total   Exclusive  Set shared  Filename

     0.00B       0.00B       0.00B  1

     0.00B       0.00B       0.00B  apps

   6.28MiB       0.00B     6.28MiB  bin

     0.00B       0.00B       0.00B  boot

     0.00B       0.00B       0.00B  data

     0.00B       0.00B       0.00B  dev

   6.04MiB       0.00B     6.04MiB  etc

  27.49MiB       0.00B    27.49MiB  frontview

     0.00B       0.00B       0.00B  ftp_ban.tbl

     0.00B       0.00B       0.00B  home

  30.39MiB       0.00B    30.39MiB  lib

     0.00B       0.00B       0.00B  lib64

     0.00B       0.00B       0.00B  media

     0.00B       0.00B       0.00B  mnt

   4.30MiB       0.00B     4.30MiB  opt

     0.00B       0.00B       0.00B  proc

     0.00B       0.00B       0.00B  root

     0.00B       0.00B       0.00B  run

  10.01MiB       0.00B    10.01MiB  sbin

     0.00B       0.00B       0.00B  srv

     0.00B       0.00B       0.00B  sys

     0.00B       0.00B       0.00B  tmp

 337.86MiB       0.00B   337.86MiB  usr

   2.02GiB    44.00KiB     2.02GiB  var

 

Looks like var is super full.  So I dug into it:

     Total   Exclusive  Set shared  Filename

  89.73MiB    21.85MiB    67.89MiB  backups

 191.49MiB   166.68MiB    24.81MiB  cache

  16.53MiB       0.00B    16.53MiB  cores

     0.00B       0.00B       0.00B  ftp

   1.60GiB     1.18GiB   438.04MiB  lib

     0.00B       0.00B       0.00B  local

  65.75MiB    57.74MiB     8.01MiB  log

     0.00B       0.00B       0.00B  mail

     0.00B       0.00B       0.00B  netatalk

     0.00B       0.00B       0.00B  opt

  17.68MiB    15.62MiB     2.07MiB  readynasd

     0.00B       0.00B       0.00B  run

     0.00B       0.00B       0.00B  spool

     0.00B       0.00B       0.00B  tmp

     0.00B       0.00B       0.00B  www

 

and in lib I found this (looks like MySql and clamav are using almost 1.6G)

     Total   Exclusive  Set shared  Filename

     0.00B       0.00B       0.00B  apache2

  33.49MiB       0.00B    33.49MiB  apt

 472.78MiB       0.00B   472.78MiB  clamav

  20.00KiB       0.00B    20.00KiB  connman

     0.00B       0.00B       0.00B  dbus

   6.03MiB       0.00B     6.03MiB  dpkg

     0.00B       0.00B       0.00B  ftp

     0.00B       0.00B       0.00B  initscripts

     0.00B       0.00B       0.00B  insserv

     0.00B       0.00B       0.00B  mdadm

     0.00B       0.00B       0.00B  misc

   1.10GiB       0.00B     1.10GiB  mysql

     0.00B       0.00B       0.00B  nfs

     0.00B       0.00B       0.00B  nut

     0.00B       0.00B       0.00B  pam

     0.00B       0.00B       0.00B  php5

     0.00B       0.00B       0.00B  readyNASVault.version

     0.00B       0.00B       0.00B  replisync

   2.53MiB       0.00B     2.53MiB  samba

     0.00B       0.00B       0.00B  snmp

 120.00KiB       0.00B   120.00KiB  systemd

     0.00B       0.00B       0.00B  ucf

     0.00B       0.00B       0.00B  update-rc.d

     0.00B       0.00B       0.00B  urandom

     0.00B       0.00B       0.00B  vim

 

I found the PID in PS:

mysql     5887  2984  1 15:38 ?        00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-error=/var/log/mysql/error.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306

 

I moved /var/lib/mysql to /apps/mysql_data and killed all the PIDs and restarted MySql.

 

MySql seems to be working, and disk usage is down to 40% (from 83%).

Message 15 of 16
StephenB
Guru

Re: RN312 “stops working”


@WildfireTech wrote:

   1.10GiB       0.00B     1.10GiB  mysql

I moved /var/lib/mysql to /apps/mysql_data and killed all the PIDs and restarted MySql.

 

MySql seems to be working, and disk usage is down to 40% (from 83%).


I'm glad you found it.  Unfortunately many apps do put stuff directly into the OS partition (even though they aren't supposed to).

 

Message 16 of 16
Top Contributors
Discussion stats
  • 15 replies
  • 1553 views
  • 0 kudos
  • 3 in conversation
Announcements