× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Re: Best practice for rearranging large directories

Blues11
Luminary

Best practice for rearranging large directories

Situation: I have two large Shares (one is about 4TB, the other about 2TB) on this ReadyNAS with four 8TB drives in a RAID5 X-RAID configuration. Of the available 23TB, about 15TB are free. The larger of the two has 4 top level directories under which there are between 500 and 800 subdirectories, each of which has 1GB to 5GB of data. (These directories infrequently change or grow in size.)

 

To make access easier, I'd like to amalgamate those 4 top level directories into one large directory with about 2200 directories.

Question: Using the interface of ReadyNAS OS (6.10.4 Hotfix 1) is there a way to drag and drop all of these directories into a new share? I am hoping that ReadyNAS OS will simple change the indexes. And I am hoping to avoid having to copy them and then delete them in macOS. (I'm in an all Mac environment.)

 

Thank you in advance for any advice.

Model: RN51600|ReadyNAS 516 6-Bay Diskless
Message 1 of 13
Sandshark
Sensei

Re: Best practice for rearranging large directories

Are you comfortable with a command line interface?  Using SSH to access the Linux command line of the  NAS and then issuing commands from that is the fastest way to accomplish what you want.  As long as you are not moving files between shares, it will just move the pointers to the files, not the file content.

Message 2 of 13
StephenB
Guru

Re: Best practice for rearranging large directories

@Blues11  wrote:

To make access easier, I'd like to amalgamate those 4 top level directories into one large directory with about 2200 directories.


You might find that easier, but browsing will likely be quite a bit slower.

Message 3 of 13
Blues11
Luminary

Re: Best practice for rearranging large directories

StephenB and Sandshark, thank you for your responses.

 

First, I'd forgotten about the Move command in Unix and that it simply updates the index, not the files themselves.

 

Upon reflection, I looked at the amount of space available on the ReadyNAS and realized that I could just do a copy because the amount of data is not that large. Then I thought about the access speed and whether four directores with a few thousand subdirectories will really be noticeably speedier than having all of them just below one high level directory.

 

So, I decided that I'd test my supposition in a new share. So far, I've copied 900+ directories onto the new share and I have not noticed any slower access speeds. I think I'll go ahead and continue to copy all the files onto the new share and see if the speeds become slower. If everything works OK, then I'll delete the old share with the four top level directories and their data and free up the terabytes of space.

 

If you're interested, when I'm done I'll post again and let you know.

 

Thank you both so much for your knowledge and desire to share it.

Message 4 of 13
Sandshark
Sensei

Re: Best practice for rearranging large directories


@Blues11 wrote:

StephenB and Sandshark, thank you for your responses.

 

First, I'd forgotten about the Move command in Unix and that it simply updates the index, not the files themselves.

 

It doesn't really apply to you any longer, but for completion's sake for anyone else looking for solutions, a mv between shares does do a copy and delete on a ReadyNAS because shares are BTRFS sub-volumes.  mv between directories in the same share does not.  Those can be important when using snapshots.  A mv between directories won't totally re-snapshot the content, but one between shares will.  So the mv between shares will still have the file in the snapshot of the first share as well as being on the new one, taking addiitonal space until purged. 

Message 5 of 13
StephenB
Guru

Re: Best practice for rearranging large directories


@Sandshark wrote:
a mv between shares does do a copy and delete on a ReadyNAS because shares are BTRFS sub-volumes.  

Yes.  But you can do a cp --reflink between shares - which is the same speed as a mv, and doesn't add any space.  Then do a delete of the original.

Message 6 of 13
Blues11
Luminary

Re: Best practice for rearranging large directories

This continuing tale: But sort of a new topic:

So I deleted the Share with the four first level directories (with a combing total about 2200+ subdirectories). The ReadyNAS OS told me that it would take some time. That was 22 hours ago. The share is gone, but virtually none of its space (about 3.5TB) has been freed.

 

I just ran a Defrag but that didn't seem to change the available space. Is there something else I need to do?

 

Thank you again for all of your help.

Message 7 of 13
Sandshark
Sensei

Re: Best practice for rearranging large directories

Defrag was likely the wrong thing to do if you use snapshots.  It does take a while for the available space to update as BTRFS releases it.  The GUI doesn't always stay up to date, either, so make sure you refresh it.  A balance may help if you don't see anything changing.

Message 8 of 13
Blues11
Luminary

Re: Best practice for rearranging large directories

It's been nearly 3 days (close to 70 hours) since I deleted the Share with about 4TB of data and the available space on the ReadyNAS is still unchanged. This seems like an excessive amount of time for the OS not to show the space as free. I use a second ReadyNAS (a very old one, but running the same OS) as a nightly backup of this NAS. Although it is configured a bit differently, when I deleted the same data on that one, it showed the space available within minutes.

 

So, could someone please advise what needs to be done to force the ReadyNAS to make that space available? Would restarting it make it perform properly? Is somethng more required, like, reinstalling the OS (but not deleting any data)?

 

Thank you.

Message 9 of 13
Sandshark
Sensei

Re: Best practice for rearranging large directories

Did you try the balance that I suggested?

Message 10 of 13
Blues11
Luminary

Re: Best practice for rearranging large directories

Hi. After getting the feedback that running defrag might not have been wise, I was hesitant to do anything else until someone said "Do Balance".

 

Anyway, I just ran Balance and it took barely a second or two. But there's no change to the amount of free space.

 

Thanks again.

Message 11 of 13
rn_enthusiast
Virtuoso

Re: Best practice for rearranging large directories

Balance didn't complete then. I can take a look in your logs to spot where that missing space is, if you like. 

Go to System > Logs > Download logs. This will download a zip file containing the logs. Upload this file to Google Drive, Dropbox or similar. PM me the link to download it along with the name of the share that you deleted. 

Cheers 

Message 12 of 13
Blues11
Luminary

Re: Best practice for rearranging large directories

Thank you for your gracious offer -- and to all here who have provided assistance -- but there's news:

 

Rebooting solved the problem!

 

The ReadyNAS runs 24/7 and the last time it was restarted was several months ago, long before this issue began. After the rebalance didn't seem to really do anything (and if it actually ran at all, it did so in milliseconds), I decided to try a simple restart.

 

And now the chart says:
data 7.2 TB

Snapshots 94 GB
Free Space 14.6 TB

 

The problem is fixed. Thank you again to you all for your suggestions and support.

Message 13 of 13
Top Contributors
Discussion stats
  • 12 replies
  • 2137 views
  • 4 kudos
  • 4 in conversation
Announcements