NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Platypus69
Feb 19, 2021Luminary
Cannot copy files to RN316 although I have 22TB free...
Hi all. I have the following RN316: Firmware 6.10.4 Running 6 x 10TB IronWolf HDDs X-RAID 21.9TB free / 23.4TB used History: Last year I replaced all the 8TB Ironwold HDDs (from memory) one by...
Platypus69
Mar 06, 2021Luminary
Thanks for that.
Muchly appreciated.
Perhaps Deftrag was not a big deal as I have only ever copied files and never/hardly ever deleted or modified them. (But I do have Copy-on-Write enabled for all shares. Not sure what that has to do woith Bit-Rot Protection, but I digress.) So that's why I got away with it???
In any case I will report what happens tomorrow as it is still at 64% as of midnight tonight. Another aside: I cannot a Balance that fails with an error shows up as a green completed line in the UI logs!!!
But is sound like you're saying that I should still perform the following after this completes:
btrfs balance start -dusage=10 /data btrfs balance start -dusage=30 /data btrfs balance start -dusage=50 /data btrfs balance start /data
Happy to do that.
Otherwise, is my current scenario due to the fact that it is predominantly storing photos? So my Photos share stores over 380,000 files that are JPG and HEVC files which are on average around 4MB in size? So that accounts for your "unusable space"?
Thanks again for comprehensive reply, muchly appreciated.
StephenB
Mar 06, 2021Guru - Experienced User
Platypus69 wrote:
Another aside: I cannot a Balance that fails with an error shows up as a green completed line in the UI logs!!!
I'm not understanding this comment.
Platypus69 wrote:
But it sound like you're saying that I should still perform the following after this completes:
Give us the btrfs info after the balance completes - that will make it easier to give you any next steps.
Platypus69 wrote:
Otherwise, is my current scenario due to the fact that it is predominantly storing photos? So my Photos share stores over 380,000 files that are JPG and HEVC files which are on average around 4MB in size? So that accounts for your "unusable space"?
I don't think it's about the number of files or their size.
BTRFS allocates fairly large blocks (1 GiB). The data from more than one file can be stored in the same block.
When you fill the share initially, the file system should be pretty efficient. But later on, as files are modified or deleted there will be "holes" - unused space in the blocks. There is background processing that can consolidate them - but I'm not sure what triggers that background processing (and there's no easy way to see it).
But doing a balance will run all the blocks that meet the -dusage criteria through the btrfs allocator again, and that will consolidate the data (removing the wasted space).
- Platypus69Mar 07, 2021Luminary
Wow! The remaining 80% of the Balance took most of today.
data balance 2021-03-05 09:34:32 2021-03-05 10:29:27 completed ERROR: error during balancing '/data': No space left on device T data balance 2021-03-05 19:39:44 2021-03-05 19:49:07 completed ERROR: error during balancing '/data': No space left on device T data balance 2021-03-05 21:09:45 2021-03-05 21:27:23 completed ERROR: error during balancing '/data': No space left on device T data balance 2021-03-05 21:28:15 2021-03-05 21:28:19 completed Done, had to relocate 1 out of 23557 chunks data balance 2021-03-05 21:45:20 2021-03-05 21:46:05 completed Done, had to relocate 29 out of 23557 chunks data balance 2021-03-05 21:57:26 2021-03-05 21:57:31 completed Done, had to relocate 1 out of 23529 chunks data balance 2021-03-05 21:59:22 2021-03-05 21:59:27 completed Done, had to relocate 1 out of 23529 chunks data balance 2021-03-05 21:59:48 2021-03-05 21:59:53 completed Done, had to relocate 1 out of 23529 chunks data balance 2021-03-05 22:25:13 2021-03-05 22:25:18 completed Done, had to relocate 1 out of 23529 chunks data balance 2021-03-05 23:19:38 2021-03-05 23:19:44 completed Done, had to relocate 1 out of 23529 chunks data balance 2021-03-06 00:54:22 2021-03-06 00:54:28 completed Done, had to relocate 1 out of 23529 chunks data defrag 2021-03-06 00:54:49 2021-03-06 03:02:04 completed data balance 2021-03-06 11:23:49 2021-03-07 18:58:54 completed Done, had to relocate 8286 out of 23482 chunks
Out of curiosity is there a correlation between the 8286 chunks relocated and the 10 odd TB that Data was reduced by? So something like 8286 x 1GB = 10TB approximately?
BTRFS:
Label: '*:root' uuid: * Total devices 1 FS bytes used 1.48GiB devid 1 size 4.00GiB used 3.61GiB path /dev/md0 Label: '*:data' uuid:* Total devices 2 FS bytes used 13.24TiB devid 1 size 18.17TiB used 10.02TiB path /dev/md127 devid 2 size 27.28TiB used 4.83TiB path /dev/md126 === filesystem /data === Data, single: total=14.81TiB, used=13.23TiB System, RAID1: total=32.00MiB, used=2.04MiB Metadata, RAID1: total=6.85GiB, used=6.80GiB Metadata, DUP: total=10.50GiB, used=8.44GiB GlobalReserve, single: total=512.00MiB, used=24.00KiB === subvolume /data ===
Does that look all good now? I have not lost data?
So what should I do now?
Happy to do following as was suggested before.
btrfs balance start -dusage=10 /data btrfs balance start -dusage=30 /data btrfs balance start -dusage=50 /data btrfs balance start /data
Or anything else such as Scrub or a Defrag.
I don't want to again have the same issue of 10TB being free but not being able to copy any files to the RN316.
So if these operations take another week, that is perfectly fine. I'd rather have everythign optimised now.
Thanks again!!!
- StephenBMar 07, 2021Guru - Experienced User
Platypus69 wrote:
Does that look all good now? I have not lost data?
So what should I do now?
All is good now, and there is no need to run anything immediately. You can of course restore the data you deleted.
But I do suggest setting up a maintenance schedule using the volume schedule control on the volume settings wheel (again, mine uses a four-month cycle, running one maintenance task each month). You could alternatively enable autodefrag on the share settings, and run the remaining tasks on a 3 month cycle.
You could also re-enable snapshots on the remaining shares if you like - ideally setting up custom snapshots with fixed retention. I use 3 months on my own systems for most shares.
If you don't have a backup plan in place for your NAS you should definitely set up one - for instance, purchasing a large USB backup disk.
Platypus69 wrote:
Out of curiosity is there a correlation between the 8286 chunks relocated and the 10 odd TB that Data was reduced by? So something like 8286 x 1GB = 10TB approximately?
Each chunk is 1 GiB. So while there is a correlation, you reclaimed somewhat more space than that. There could have been some chunks that were allocated but completely unused - not sure if the log would show them as relocated or not.
- Platypus69Mar 07, 2021Luminary
Thanks for that. Will do.
Thanks A BILLION for everyone's help and all practical advice!!!!!!
And patience :)
I do have another NAS/HDD for the photos but am wondering if I just RAR up each month and then store them in AWS Deep Glacier for $0.00099 per GB-month.
Notice that my New DS1819+ can sync with BackBlaze, so maybe I'll take advantge of that???
- rn_enthusiastMar 07, 2021Virtuoso
Out of interest, how did your full balance distribute the data between md127 and md126, in the end?
:)
- StephenBMar 07, 2021Guru - Experienced User
rn_enthusiast wrote:
Out of interest, how did your full balance distribute the data between md127 and md126, in the end?
It's still running - at the moment the stats are
Label: '2fe72582:data' uuid: a665beff-2a06-4b88-b538-f9fa4fb2dfef Total devices 2 FS bytes used 13.64TiB devid 1 size 16.36TiB used 9.55TiB path /dev/md127 devid 2 size 10.91TiB used 4.11TiB path /dev/md126It started at
Label: '2fe72582:data' uuid: a665beff-2a06-4b88-b538-f9fa4fb2dfef Total devices 2 FS bytes used 13.54TiB devid 1 size 16.36TiB used 12.72TiB path /dev/md127 devid 2 size 10.91TiB used 1.27TiB path /dev/md126
So it has shifted quite a few blocks to md126.
Related Content
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!