RN516. Volume: System volume root's usage is 81%

I'm running an RN516 on OS 6.5.1 with 6x 6TB WD RED's.

Out of the blue I get a "Volume: System volume root's usage is 81%" error.

I'm not running *any* 3rd party apps and only use the unit as a file server at home.

The percentage went up to 88% once and went back down to 81% and it's hovering there so the unit keeps functioning, but something might give at some point and I need to nip this in the bud as painlessly as possible.

I got this unit from a former colleague and as such it is not registered in my name and I cannot apply for support since I am no longer in contact with that person (I had to RMA the unit under warranty once and back then I got some Netgear techie to snoop inside my NAS remotely and he saw it was fubar, so I know what one could do for me). So i have to ticket number. All I get now is to look into the knowledge base (came up zip) and ask here. I hope some of you dear experts (winking at mr.mdgm) can help me out.

When I got this error I knew nothing about SSH, hell I didn't even know what SSH stood for. Now, I still don't (I'm guessing one of the S's  stands for Shell), but I did read up in here on SSH enabling and running commands to find the sore spot.

I'm still a real n00b with SSH and I am in need for a little push in the right direction, or even need to be taken by the hand... this once.

So here's what I did so far:

admin@nas516:/$ df -h Filesystem Size Used Avail Use% Mounted on udev 10M 4.0K 10M 1% /dev /dev/md0 4.0G 3.3G 452M 88% / tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 3.9G 16M 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup tmpfs 3.9G 0 3.9G 0% /media /dev/md/data-0 28T 26T 1.7T 95% /data /dev/md/data-0 28T 26T 1.7T 95% /home /dev/md/data-0 28T 26T 1.7T 95% /apps tmpfs 50M 0 50M 0% /var/replicate/shm /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/home /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Backup /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Games /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Media /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Music /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Pictures /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Programs /dev/md/data-0 28T 26T 1.7T 95% /var/ftp/Videos /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Backup /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Games /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Media /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Music /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Pictures /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Programs /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/data/Videos /dev/md/data-0 28T 26T 1.7T 95% /run/nfs4/home

So I see that /dev/md0 is almost full (don't see RootFS, why is that?).

I then checked the inodes, because it seems that people do that (SSH N00b, don't know what it's for)

admin@nas516:/$ df -i Filesystem Inodes IUsed IFree IUse% Mounted on udev 1006093 510 1005583 1% /dev /dev/md0 0 0 0 - / tmpfs 1006709 1 1006708 1% /dev/shm tmpfs 1006709 708 1006001 1% /run tmpfs 1006709 4 1006705 1% /sys/fs/cgroup tmpfs 1006709 1 1006708 1% /media /dev/md/data-0 0 0 0 - /data /dev/md/data-0 0 0 0 - /home /dev/md/data-0 0 0 0 - /apps tmpfs 1006709 7 1006702 1% /var/replicate/shm /dev/md/data-0 0 0 0 - /var/ftp/home /dev/md/data-0 0 0 0 - /var/ftp/Backup /dev/md/data-0 0 0 0 - /var/ftp/Games /dev/md/data-0 0 0 0 - /var/ftp/Media /dev/md/data-0 0 0 0 - /var/ftp/Music /dev/md/data-0 0 0 0 - /var/ftp/Pictures /dev/md/data-0 0 0 0 - /var/ftp/Programs /dev/md/data-0 0 0 0 - /var/ftp/Videos /dev/md/data-0 0 0 0 - /run/nfs4/data/Backup /dev/md/data-0 0 0 0 - /run/nfs4/data/Games /dev/md/data-0 0 0 0 - /run/nfs4/data/Media /dev/md/data-0 0 0 0 - /run/nfs4/data/Music /dev/md/data-0 0 0 0 - /run/nfs4/data/Pictures /dev/md/data-0 0 0 0 - /run/nfs4/data/Programs /dev/md/data-0 0 0 0 - /run/nfs4/data/Videos /dev/md/data-0 0 0 0 - /run/nfs4/home

I really don't know what I'm seeing here.

I then listed the folders like so:

admin@nas516:/dev$ du -hxm --max-depth=1 / |sort -h du: cannot read directory `/etc/ssl/private': Permission denied du: cannot read directory `/etc/frontview/.msmtp.queue': Permission denied du: cannot read directory `/root': Permission denied du: cannot read directory `/var/backups/md': Permission denied du: cannot read directory `/var/log/apache2': Permission denied du: cannot read directory `/var/log/mysql': Permission denied du: cannot read directory `/var/spool/cron/crontabs': Permission denied du: cannot read directory `/var/cache/ldconfig': Permission denied du: cannot read directory `/var/cache/samba/msg': Permission denied du: cannot read directory `/var/lib/nut': Permission denied du: cannot read directory `/var/lib/samba/private/msg.sock': Permission denied du: cannot read directory `/var/lib/connman/ethernet_28c68ed5d6f6_cable': Permission denied du: cannot read directory `/var/lib/connman/ethernet_4494fc6e3766_cable': Permission denied du: cannot read directory `/var/lib/connman/ethernet_4494fc6e3767_cable': Permission denied du: cannot read directory `/var/lib/mysql': Permission denied du: cannot read directory `/var/lib/php5': Permission denied du: cannot read directory `/var/lib/sudo': Permission denied 0 /apps 0 /boot 0 /dev 0 /home 0 /media 0 /mnt 0 /proc 0 /root 0 /run 0 /selinux 0 /srv 0 /sys 1 /data 1 /lib64 1 /tmp 4 /lib32 7 /bin 8 /etc 8 /opt 10 /sbin 30 /frontview 43 /lib 289 /var 473 /usr 869 /

So var and usr seem the biggest and I started looking into them, and ofcourse the log folder.

admin@nas516:/var$ du -hxm --max-depth=1 . |sort -h du: cannot read directory `./backups/md': Permission denied du: cannot read directory `./log/apache2': Permission denied du: cannot read directory `./log/mysql': Permission denied du: cannot read directory `./spool/cron/crontabs': Permission denied du: cannot read directory `./cache/ldconfig': Permission denied du: cannot read directory `./cache/samba/msg': Permission denied du: cannot read directory `./lib/nut': Permission denied du: cannot read directory `./lib/samba/private/msg.sock': Permission denied du: cannot read directory `./lib/connman/ethernet_28c68ed5d6f6_cable': Permission denied du: cannot read directory `./lib/connman/ethernet_4494fc6e3766_cable': Permission denied du: cannot read directory `./lib/connman/ethernet_4494fc6e3767_cable': Permission denied du: cannot read directory `./lib/mysql': Permission denied du: cannot read directory `./lib/php5': Permission denied du: cannot read directory `./lib/sudo': Permission denied 0 ./agentx 0 ./cores 0 ./ftp 0 ./local 0 ./mail 0 ./opt 0 ./replicate 0 ./tmp 1 ./netatalk 1 ./nuuo 1 ./readydrop 1 ./spool 1 ./www 2 ./readynasd 36 ./log 46 ./lib 67 ./backups 141 ./cache 289 . admin@nas516:/var$ admin@nas516:/var$ cd ./cache admin@nas516:/var/cache$ du -hxm --max-depth=1 . |sort -h du: cannot read directory `./ldconfig': Permission denied du: cannot read directory `./samba/msg': Permission denied 0 ./apache2 0 ./dbconfig-common 0 ./forked-daapd 0 ./istat 0 ./ldconfig 0 ./man 1 ./samba 2 ./debconf 138 ./apt 141 . admin@nas516:/var/cache$ cd ./apt admin@nas516:/var/cache/apt$ du -hxm --max-depth=1 . |sort -h 102 ./archives 138 . admin@nas516:/var/cache/apt$ cd ./archives admin@nas516:/var/cache/apt/archives$ du -hxm --max-depth=1 . |sort -h 0 ./partial 102 . admin@nas516:/var/log$ du -ah 0 ./news/news.crit 0 ./news/news.err 0 ./news/news.notice 0 ./news 4.0K ./apt/term.log.12.gz 4.0K ./apt/term.log.11.gz 4.0K ./apt/term.log.10.gz 4.0K ./apt/term.log.9.gz 4.0K ./apt/term.log.8.gz 4.0K ./apt/term.log.7.gz 4.0K ./apt/term.log.6.gz 4.0K ./apt/term.log.5.gz 4.0K ./apt/term.log.4.gz 4.0K ./apt/term.log.3.gz 4.0K ./apt/term.log.2.gz 0 ./apt/term.log 4.0K ./apt/term.log.1.gz 4.0K ./apt/history.log.12.gz 4.0K ./apt/history.log.11.gz 4.0K ./apt/history.log.10.gz 4.0K ./apt/history.log.9.gz 4.0K ./apt/history.log.8.gz 4.0K ./apt/history.log.7.gz 4.0K ./apt/history.log.6.gz 4.0K ./apt/history.log.5.gz 4.0K ./apt/history.log.4.gz 4.0K ./apt/history.log.3.gz 4.0K ./apt/history.log.2.gz 0 ./apt/history.log 4.0K ./apt/history.log.1.gz 96K ./apt 0 ./fsck 4.0K ./frontview/initrd.log 4.0K ./frontview/backup/status_backup_000 4.0K ./frontview/backup 164K ./frontview/error.log 124K ./frontview/http-error.log 0 ./frontview/ssl_access.log 120K ./frontview/http-access.log 784K ./frontview/status.log 328K ./frontview/msmtp.log 0 ./frontview/.managed 100K ./frontview/cgi.log 0 ./frontview/.registered 1.9M ./frontview/ssl_access.log.old 20K ./frontview/msmtp.queue.log 3.5M ./frontview 8.0M ./journal/0e422faf8bd745e3a9c25b12ace37072/system@bfd79e7095da41d79a5603be6e48c6b4-0000000000557878-00053a66f0a25aca.journal 7.9M ./journal/0e422faf8bd745e3a9c25b12ace37072/system@bfd79e7095da41d79a5603be6e48c6b4-0000000000559f1d-00053a8fff4acb2c.journal 8.0M ./journal/0e422faf8bd745e3a9c25b12ace37072/system@bfd79e7095da41d79a5603be6e48c6b4-000000000055c647-00053ab90b994927.journal 4.6M ./journal/0e422faf8bd745e3a9c25b12ace37072/system.journal 29M ./journal/0e422faf8bd745e3a9c25b12ace37072 29M ./journal 160K ./samba/log.nmbd 516K ./samba/log.smbd.old 132K ./samba/log.smbd 808K ./samba du: cannot read directory `./apache2': Permission denied 0 ./apache2 248K ./readynasd/event_push.log 228K ./readynasd/fwbroker.log 632K ./readynasd/event_push.log.old 4.0K ./readynasd/ads.log 4.0K ./readynasd/install_debpkg.log 1.1M ./readynasd 132K ./netatalk.log 4.0K ./LeafP2P.log 40K ./proftpd.log du: cannot read directory `./mysql': Permission denied 0 ./mysql 0 ./mysql.err 4.0K ./readydropd.log 4.0K ./dbbroker.log 4.0K ./dbconfig-common/dbc.log 4.0K ./dbconfig-common 16K ./lastlog 0 ./faillog 4.0K ./dpkg.log.12.gz 4.0K ./dpkg.log.11.gz 4.0K ./dpkg.log.10.gz 4.0K ./dpkg.log.9.gz 4.0K ./dpkg.log.8.gz 4.0K ./dpkg.log.7.gz 4.0K ./dpkg.log.6.gz 4.0K ./dpkg.log.5.gz 4.0K ./dpkg.log.4.gz 4.0K ./dpkg.log.3.gz 4.0K ./dpkg.log.2.gz 4.0K ./dpkg.log.1 32K ./dpkg.log 4.0K ./wtmp.1 12K ./wtmp 0 ./btmp.1 0 ./btmp 16K ./dmesg.4.gz 16K ./dmesg.3.gz 16K ./dmesg.2.gz 16K ./dmesg.1.gz 48K ./dmesg.0 64K ./dmesg 824K ./rsyslog.log 4.0K ./alternatives.log.3.gz 4.0K ./alternatives.log.2.gz 4.0K ./alternatives.log.1 0 ./alternatives.log 1.4M ./rsyslog.log.old 4.0K ./mysql.log.7.gz 4.0K ./mysql.log.6.gz 4.0K ./mysql.log.5.gz 4.0K ./mysql.log.4.gz 4.0K ./mysql.log.3.gz 4.0K ./mysql.log.2.gz 0 ./mysql.log 4.0K ./mysql.log.1.gz 37M . admin@nas516:/var/log$

So log has nothing big in there it seems and the other two looked the same. I think.

admin@nas516:/var$ ls -al total 24 drwxr-xr-x 1 root root 202 Aug 16 00:34 . drwxr-xr-x 1 root root 302 Aug 24 17:35 .. drwxr-xr-x 1 root root 0 Mar 17 2014 agentx drwxr-xr-x 1 root root 734 Aug 24 04:20 backups drwxr-xr-x 1 root root 130 Jun 17 00:17 cache drwxr-xr-x 1 root root 0 Aug 1 2013 cores drwxr-xr-x 1 guest guest 94 Aug 24 19:13 ftp drwxr-xr-x 1 root root 326 Aug 16 00:34 lib drwxrwsr-x 1 root staff 0 May 21 2012 local lrwxrwxrwx 1 root root 9 Aug 16 00:34 lock -> /run/lock drwxr-xr-x 1 root root 1192 Aug 24 19:13 log drwxrwsr-x 1 root mail 0 Jun 4 2012 mail drwxr-xr-x 1 root root 76 Jun 1 01:40 netatalk drwxr-xr-x 1 root root 186 Aug 16 00:07 nuuo drwxr-xr-x 1 root root 0 Jun 4 2012 opt drwxr-xr-x 1 admin admin 38 Dec 9 2014 readydrop drwxr-xr-x 1 root root 298 Aug 24 21:46 readynasd drwxr-xr-x 1 root root 6 Aug 31 2014 replicate lrwxrwxrwx 1 root root 6 Aug 16 00:34 run -> ../run drwxr-xr-x 1 root root 46 Aug 16 00:33 spool drwxrwxrwt 1 root root 0 May 21 2012 tmp drwxr-xr-x 1 root root 20 Aug 16 00:34 www admin@nas516:/var$ cd /log -bash: cd: /log: No such file or directory admin@nas516:/var$ cd log admin@nas516:/var/log$ ls -al total 2408 drwxr-xr-x 1 root root 1192 Aug 24 19:13 . drwxr-xr-x 1 root root 202 Aug 16 00:34 .. -rw-r--r-- 1 root root 0 Aug 16 06:25 alternatives.log -rw-r--r-- 1 root root 358 Aug 16 00:38 alternatives.log.1 -rw-r--r-- 1 root root 133 Dec 9 2014 alternatives.log.2.gz -rw-r--r-- 1 root root 600 Sep 3 2014 alternatives.log.3.gz drwxr-x--- 1 root adm 3482 Aug 21 06:25 apache2 drwxr-xr-x 1 root root 746 Aug 16 06:25 apt -rw------- 1 root utmp 0 Aug 1 06:25 btmp -rw------- 1 root utmp 0 Jul 1 06:25 btmp.1 -rw-r--r-- 1 root root 781 Aug 24 19:13 dbbroker.log drwxr-xr-x 1 root root 14 Sep 3 2014 dbconfig-common -rw-r--r-- 1 root adm 61591 Aug 24 19:13 dmesg -rw-r--r-- 1 root adm 61591 Aug 24 17:35 dmesg.0 -rw-r--r-- 1 root adm 15260 Aug 16 00:34 dmesg.1.gz -rw-r--r-- 1 root adm 12320 Jul 20 17:50 dmesg.2.gz -rw-r--r-- 1 root adm 12317 Jan 16 2016 dmesg.3.gz -rw-r--r-- 1 root adm 12330 Jan 7 2016 dmesg.4.gz -rw-r--r-- 1 root root 28829 Aug 24 19:14 dpkg.log -rw-r--r-- 1 root root 2371 Jul 31 17:52 dpkg.log.1 -rw-r--r-- 1 root root 993 Oct 31 2015 dpkg.log.10.gz -rw-r--r-- 1 root root 309 Sep 30 2015 dpkg.log.11.gz -rw-r--r-- 1 root root 320 Aug 31 2015 dpkg.log.12.gz -rw-r--r-- 1 root root 153 Jun 30 18:03 dpkg.log.2.gz -rw-r--r-- 1 root root 155 May 31 18:03 dpkg.log.3.gz -rw-r--r-- 1 root root 289 Apr 30 18:05 dpkg.log.4.gz -rw-r--r-- 1 root root 453 Mar 31 18:02 dpkg.log.5.gz -rw-r--r-- 1 root root 151 Feb 29 17:02 dpkg.log.6.gz -rw-r--r-- 1 root root 295 Jan 31 2016 dpkg.log.7.gz -rw-r--r-- 1 root root 489 Dec 31 2015 dpkg.log.8.gz -rw-r--r-- 1 root root 362 Nov 30 2015 dpkg.log.9.gz -rw-r--r-- 1 root root 0 Dec 9 2014 faillog drwxr-xr-x 1 admin admin 292 Jul 12 2012 frontview drwxr-xr-x 1 root root 0 Oct 14 2013 fsck drwxr-xr-x 1 root root 64 Jun 30 2012 journal -rw-r--r-- 1 root root 28908 Aug 25 00:55 lastlog -rw-r--r-- 1 root root 3167 Aug 31 2014 LeafP2P.log drwxr-s--- 1 mysql adm 0 Aug 30 2014 mysql -rw-r----- 1 mysql adm 0 Aug 30 2014 mysql.err -rw-r----- 1 mysql adm 0 Aug 24 06:25 mysql.log -rw-r----- 1 mysql adm 20 Aug 23 06:25 mysql.log.1.gz -rw-r----- 1 mysql adm 20 Aug 22 06:25 mysql.log.2.gz -rw-r----- 1 mysql adm 20 Aug 21 06:25 mysql.log.3.gz -rw-r----- 1 mysql adm 20 Aug 20 06:25 mysql.log.4.gz -rw-r----- 1 mysql adm 20 Aug 19 06:25 mysql.log.5.gz -rw-r----- 1 mysql adm 20 Aug 18 06:25 mysql.log.6.gz -rw-r----- 1 mysql adm 20 Aug 17 06:25 mysql.log.7.gz -rw-r--r-- 1 root root 135058 Dec 7 2014 netatalk.log drwxr-sr-x 1 news news 56 Jun 8 2012 news -rw-r----- 1 root root 41577 Aug 24 19:13 proftpd.log -rw-r--r-- 1 root root 76 Dec 9 2014 readydropd.log drwxr-xr-x 1 admin admin 138 Aug 15 23:59 readynasd -rw-r----- 1 root adm 298662 Aug 25 00:59 rsyslog.log -rw-r--r-- 1 root root 1610553 Aug 24 19:32 rsyslog.log.old drwxr-xr-x 1 root root 56 Aug 23 19:24 samba -rw-rw-r-- 1 root utmp 12288 Aug 25 00:55 wtmp -rw-rw-r-- 1 root utmp 3456 Jul 20 17:51 wtmp.1

admin@nas516:/usr$ ls -al total 16 drwxr-xr-x 1 root root 80 Jun 16 14:48 . drwxr-xr-x 1 root root 302 Aug 24 17:35 .. drwxr-xr-x 1 root root 9240 Aug 16 00:37 bin drwxr-xr-x 1 root root 0 May 21 2012 games drwxr-xr-x 1 root root 44 Apr 21 2014 include drwxr-xr-x 1 root root 3990 Aug 16 00:37 lib drwxr-xr-x 1 root root 102 Dec 9 2014 lib32 drwxrwsr-x 1 root staff 72 Aug 16 00:33 local drwxr-xr-x 1 root root 2584 Aug 16 00:33 sbin drwxr-xr-x 1 root root 1020 Feb 18 2016 share drwxr-xr-x 1 root root 0 May 21 2012 src