Reply

[RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

[RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

Me noticed my NAS is going out of the free space, so I get into the admin panel. Removing all the snapshots of both folders and home folders didn't change anything, actually.

 

How do I free the space, consumed by the non-existing snapshots?

 

My NAS overview page:

snap_overview.png

 

Volume summary info:

snap_stats.png

 

List of shared resources (please notice there are no snapshots):

snap_shared_folders.png

 

Configured maintenance:

snap_sched.png

DATA PARTITION SIZE:

root@databox:~# du -sh /data

4.1T    /data


FREE SPACE:

root@databox:~# df -H

Filesystem      Size  Used Avail Use% Mounted on
...
/dev/md127       10T  7.6T  2.5T  76% /data
...

 

 

 

 

 

 

 

 

 

 

 

 

 

Model: RNDU6000|ReadyNAS Ultra 6 Chassis only
Message 1 of 16

Accepted Solutions
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

I'd try deleting some of them manually (one at a time), and confirm that there are no errors.

 

There is a -c option, which waits for the deletion to be committed before the btrfs subvolume delete returns.  Might be worth trying that on one or two.

 

With some work, you could create a script to extract each snapshot path from the list, and then build the btrfs command to delete it.  But as I mentioned earlier, you'd need to be careful not to accidentally end up deleting a share by mistake.

View solution in original post

Message 13 of 16

All Replies
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@SauvageSausage wrote:

 

How do I free the space, consumed by the non-existing snapshots?

 


Start by running a balance from the volume settings wheel.  That should reclaim the missing space.

 


@SauvageSausage wrote:

 

root@databox:~# du -sh /data

4.1T    /data

 

root@databox:~# df -H

 


btrfs fi du -s /data will give more accurate results than du (though it will be very slow).

 

btrfs fi df /data completes quickly and gives more info than df

Message 2 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@StephenB wrote:

@SauvageSausage wrote:

 

How do I free the space, consumed by the non-existing snapshots?

 


Start by running a balance from the volume settings wheel.  That should reclaim the missing space.

 

Already done! 

 

 


btrfs fi du -s /data will give more accurate results than du (though it will be very slow).

 

btrfs fi df /data completes quickly and gives more info than df


Thanks for your tip! "btrfs fi du" is in progress; 'df' is:

 

root@databox:/# btrfs fi df /data
Data, single: total=7.20TiB, used=6.95TiB
System, DUP: total=32.00MiB, used=992.00KiB
Metadata, DUP: total=6.50GiB, used=3.06GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

 

 

 

Model: RNDU6000|ReadyNAS Ultra 6 Chassis only
Message 3 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

Can you also run this:

btrfs subvolume list -s /data

If there are any snapshots left for some reason, this command should list them.

 

Message 4 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

ARE ALL THOSE SNAPSHOTS? 😮

 

https://pastebin.com/WUrGhPQG

 

Model: RNDU6000|ReadyNAS Ultra 6 Chassis only
Message 5 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@SauvageSausage wrote:

ARE ALL THOSE SNAPSHOTS? 😮

 

https://pastebin.com/WUrGhPQG

 


Yes.  It's not clear to me why they aren't showing up in the web ui though.

 

 

Message 6 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

is it safe to bulk delete them using something like

 

btrfs subvolume delete $SNAPPATH

 

where $SNAPPATH is a path, ending with '/snapshot' ?

Message 7 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@SauvageSausage wrote:

is it safe to bulk delete them using something like

 

btrfs subvolume delete $SNAPPATH

 

where $SNAPPATH is a path, ending with '/snapshot' ?


I don't think that will work.  You could create a script to delete one at a time, but you'd to be careful, as a mistake could delete your data.

 

Try this first:

snapper delete 1-100000

and see if that gets rid of them.

 

 

 

Message 8 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

 

root@databox:~# pwd
/root

root@databox:~# snapper delete 1-100000
The config 'root' does not exist. Likely snapper is not configured.
See 'man snapper' for further instructions.

root@databox:~# sudo snapper delete 1-100000
The config 'root' does not exist. Likely snapper is not configured.
See 'man snapper' for further instructions.

 

UPDATE:

 

 

root@databox:~# ls /etc/snapper/configs
0  1  10  11  12  13  14  16  17  2  20  21  3  4  5  6  7  8  9

 

 

root@databox:~# cat /etc/snapper/configs/0

# subvolume to snapshot
SUBVOLUME="/data/torrents"

# filesystem type
FSTYPE="btrfs"


# btrfs qgroup for space aware cleanup algorithms
QGROUP=""


# fraction of the filesystems space the snapshots may use
SPACE_LIMIT="0.5"


# users and groups allowed to work with config
ALLOW_USERS="guest"
ALLOW_GROUPS="admin"

# sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
# directory
SYNC_ACL="no"


# start comparing pre- and post-snapshot in background after creating
# post-snapshot
BACKGROUND_COMPARISON="yes"


# run daily number cleanup
NUMBER_CLEANUP="yes"

# limit for number cleanup
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="50"
NUMBER_LIMIT_IMPORTANT="10"


# create hourly snapshots
TIMELINE_CREATE="yes"

# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"

# limits for timeline cleanup
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="10"
TIMELINE_LIMIT_DAILY="10"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="10"
TIMELINE_LIMIT_YEARLY="10"


# cleanup empty pre-post-pairs
EMPTY_PRE_POST_CLEANUP="yes"

# limits for empty pre-post-pair cleanup
EMPTY_PRE_POST_MIN_AGE="1800"

 

Message 9 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

root@databox:~# snapper delete 1-100000
The config 'root' does not exist. Likely snapper is not configured.
See 'man snapper' for further instructions.


root@databox:~# ls /etc/snapper/configs
0  1  10  11  12  13  14  16  17  2  20  21  3  4  5  6  7  8  9


root@databox:~# cat /etc/snapper/configs/0

# subvolume to snapshot
SUBVOLUME="/data/torrents"

# filesystem type
FSTYPE="btrfs"


# btrfs qgroup for space aware cleanup algorithms
QGROUP=""


# fraction of the filesystems space the snapshots may use
SPACE_LIMIT="0.5"


# users and groups allowed to work with config
ALLOW_USERS="guest"
ALLOW_GROUPS="admin"

# sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
# directory
SYNC_ACL="no"


# start comparing pre- and post-snapshot in background after creating
# post-snapshot
BACKGROUND_COMPARISON="yes"


# run daily number cleanup
NUMBER_CLEANUP="yes"

# limit for number cleanup
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="50"
NUMBER_LIMIT_IMPORTANT="10"


# create hourly snapshots
TIMELINE_CREATE="yes"

# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"

# limits for timeline cleanup
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="10"
TIMELINE_LIMIT_DAILY="10"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="10"
TIMELINE_LIMIT_YEARLY="10"


# cleanup empty pre-post-pairs
EMPTY_PRE_POST_CLEANUP="yes"

# limits for empty pre-post-pair cleanup
EMPTY_PRE_POST_MIN_AGE="1800"
Message 10 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@StephenB wrote:

@SauvageSausage wrote:

is it safe to bulk delete them using something like

 

btrfs subvolume delete $SNAPPATH

 

where $SNAPPATH is a path, ending with '/snapshot' ?


I don't think that will work.  You could create a script to delete one at a time, but you'd to be careful, as a mistake could delete your data.

 

Try this first:

snapper delete 1-100000

and see if that gets rid of them.

 


The snapper command will fail because Netgear has set up snapper with a config for each share (and no root config).

 

It should work if you first list the configs (which are set up as numeric):

snapper list-configs

then delete snapshots for each config

snapper -c 0 delete 1-100000
snapper -c 1 delete 1-100000
...

Of course this will only delete your snapshots if they are in a config - which might not be the case.  But it is easy to try, and there should be a lot fewer configs than there are snapshots in your case.

 

Message 11 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

thanks! but it seems you're right and snapshots aren't in the configs:

 

 

root@databox:~# snapper -c 0 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 1 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 2 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 3 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 4 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 5 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 6 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 7 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 8 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 9 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 10 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 11 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 12 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 13 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 14 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 15 delete 1-100000
Unknown config.

root@databox:~# snapper -c 16 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 17 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 18 delete 1-100000
Unknown config.

root@databox:~# snapper -c 19 delete 1-100000
Unknown config.

root@databox:~# snapper -c 20 delete 1-100000
Snapshot '1' not found.

root@databox:~# snapper -c 21 delete 1-100000
Snapshot '1' not found.

 

 

Message 12 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

I'd try deleting some of them manually (one at a time), and confirm that there are no errors.

 

There is a -c option, which waits for the deletion to be committed before the btrfs subvolume delete returns.  Might be worth trying that on one or two.

 

With some work, you could create a script to extract each snapshot path from the list, and then build the btrfs command to delete it.  But as I mentioned earlier, you'd need to be careful not to accidentally end up deleting a share by mistake.

Message 13 of 16

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

Thank you, it did help! There is a strange thing with balancing progress tho but I believe nothing fatal:

snap_stats_135_perc.png

Message 14 of 16
StephenB
Guru

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?


@SauvageSausage wrote:

Thank you, it did help! There is a strange thing with balancing progress tho but I believe nothing fatal:

snap_stats_135_perc.png


I've seen this myself - I think it happens when the volume has been vertically expanded in the past.  But it isn't fatal.

 

 

Message 15 of 16
Sandshark
Sensei

Re: [RNDU6000] How do I recover the space taken up by the 'ghost' snapshots?

I hasve found that the ReadyNAS does not automatically do a balance after expansion (which is recommended when expanding a BTRFS volume), and that results in no allocation of metadata space on a vertically expanded volume.  So, manually doing a balance afterward is highly recommened.

 

I've also noticed some inconsistancy as to whether the metedata is in RAID or DUP.  You can find out with btrfs filesystem df /data.

 

From everything I've read, RAID is the best configuration.  The following will change that:  btrfs balance start -mconvert=raid1 /data

 

Of course, it's a balance, so it'll take a while.

Message 16 of 16
Top Contributors
Discussion stats
  • 15 replies
  • 1344 views
  • 7 kudos
  • 3 in conversation
Announcements