Filesystem: /dev/md127
Size: 1.9T
Used: 1.7T
Available: 186GB
Use %: 91%
Mounted on: /data (and also repeated for /home, /apps, /run/nfs4/data/Recordings, /run/nfs4/data/home /var/ftpBackup)
So, showing that I am using 1.7TB of my 1.9 = nearly full.
However this isn't reflective of actual usage.
du -sh *
returns:
87M apps
5.7M bin
4.0K boot
873G data
4.0K dev
4.3M etc
24M frontview
0 ftp_ban.tbl
0 homes
36M lib
16K lost+found
0 media
4.0K mnt
5.1M opt
du: cannot access `proc/21012/task/21012/fd/4': No such file or directory
du: cannot access `proc/21012/task/21012/fdinfo/4': No such file or directory
du: cannot access `proc/21012/fd/4': No such file or directory
du: cannot access `proc/21012/fdinfo/4': No such file or directory
0 proc
32K root
1.2M run
7.9M sbin
4.0K selinux
4.0K srv
0 sys
28K tmp
211M usr
211M var
The only thing of any size is /data which is 873GB
So the overall disk usage should be maybe about 1TB, not 1.7TB
The fact the filesystem /dev/dm127 is mounted repeatedly on /home, /apps, /run/nfs4/data/Recordings, /run/nfs4/data/home /var/ftp/Backup -- I don't think this should be an issue as it's just different mount points for the same data, right? (the contents of /data/home is the same as /home for example).
So then I'm trying to figure out what on earth is taking up all the additional space.
Wondering if it's part of the auto backups (snapshots) but surely these would auto purge (over time and also hopefully when it gets close to full), and surely these would be incremental therefore not taking up an additional 700+GB
You should be using the btrfs commands for this type of analysis.
Generally the discrepancy comes from two basic sources.
One is that du doesn't account for btrfs metadata. With ext, metadata is saved in inodes, which are preallocated when the file system is created. BTRFS doesn't do that, and the amount of metadata space used is dynamic.
The other is that du ignores snapshots. Given the size of your discrepancy, I am guessing you have snapshots enabled. You should delete some of them from the web UI to free up space. One approach is to delete all the snapshots from shares that have a lot of churn (changed or deleted files - not adds or renames). Another is to delete the oldest snapshots from each share.
The way to clean up metadata is to schedule regular balance jobs (system->settings->maintenance). I run that quarterly, but if there a lot stuff changing on your NAS you might want to schedule it monthly.
btrfs fi show /data
Label: '0e36d6d7:data' uuid: 4325b83e-28ce-4f63-9e46-a5b09379d96b
Total devices 1 FS bytes used 1.63TiB
devid 1 size 1.81TiB used 1.64TiB path /dev/md127
btrfs fi df /data
Data, single: total=1.64TiB, used=1.63TiB
System, DUP: total=8.00MiB, used=224.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=2.00GiB, used=1.29GiB
Metadata, single: total=8.00MiB, used=0.00B
So yes here we're seeing the full 1.7T usage.
The meta data is small enough so are we good to presume this is caused by snapshots?
Seems crazy that the snapshot system is choking the drive, essentially only allowing me to use half of the disk space as it then uses the same again for snapshots.
I currently don't have access to the GUI (remote ssh only) so is there a way I can clean things up through SSH?
Also when I've tried to clean up old snapshots through the GUi before it's an absolute pain as you have to go into each one and delete, wait, repeat, which can be very laborour if you're looking at days/weeks/months. And ideally I don't want to be having to clear through them periodically. Is there a way to a) reduce the frequency of snapshots and b) make it auto purge the oldest if disk space gets beyond, say 80%?
And, is there a way to do any of this via SSH, as I won't have access to the GUI for a few weeks, and by then it'll probably hit 100% at this rate.
You need to use the GUI, since snapshot status is also maintained in the database. However, you can delete more than one at a time (in chrome, press the control key as you click).
Some shares simply shouldn't have snapshots enabled (for instance the folders Transmission uses for torrents).
If deleting snapshots doesn't solve your space problem, you should contact support. There have been some cases where old snapshots get "lost" by the GUI, and support needs to manually clean them. I believe in most cases they were created by an earlier firmware version, and not migrated properly.
...Also when I've tried to clean up old snapshots through the GUi before it's an absolute pain as you have to go into each one and delete, wait, repeat, which can be very laborour if you're looking at days/weeks/months. And ideally I don't want to be having to clear through them periodically. Is there a way to a) reduce the frequency of snapshots and b) make it auto purge the oldest if disk space gets beyond, say 80%?
You can set the frequency of snapshots for each share in the GUI already. Automatically taken snapshots are also "thinned" so that as they get older only monthly snapshots are kept. I'd like to see an age threshold also (purge everything more then x months old), but so far Netgear hasn't provided that tool. So every month I have to go in manually and delete the oldest snapshot for each share.
The system is supposed auto-delete the oldest snapshots when the space usage exceeds 95%
Sorry realised I can still do LAN through tunnelling - was having a bit of a brainfart.
Issue I now receive however is that the ReadyNAS GUI is getting served over HTTPS (localhost:8080 forwards me to https://localhost:8080/admin)
As a result, Chrome returns SSL Connection Error:
Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have.
Presuming this is a built in security feature of the tunnel because normally (locally) you'd just accept the invalid certificate and carry on. Trying to see if there's a way to work around this...
NB the tunnel is set up on source port 8080, to forward to ip.of.nas:80 (99% sure the ReadyNas GUI is on port 80).
It's not the snapshot upgrade issue. So creating that snapshot prune flag file should be enough to get the oldest automatic snapshots deleted to bring usage down when volume usage exceeds 80%.
