I have SSH'd into my 316 and am doing cp and mv's... trying to move a bunch of data from one share to another (/data/share1 to /data/share2).It's slow. Like, really slow (maybe 90 seconds per GB). Is this a limitation with SMB, BTFRS, Linux, or ReadyNAS? Is this to b expected or are there ways to speed it up?Thanks,John

https://www.readynas.com/forum/viewtopi ... 35#p422574

This is really awesome - you saved me so much time it's not even funny!Thanks!

For posterity, the post from that thread reads:Skywalker wrote:StephenB wrote:Xophile wrote:Isn't it possible to move data between shares directly with BTRFS?Each share is set up as a btrfs subvolume. That allows you to control snapshots on a share-by-share basis.A direct consequence is that moving files from one share to another is done as a copy/delete - the same as moving files between two disks on a PC.Well, not exactly. You can "instantly" copy files around between subvolumes using cp --reflink, and then remove the original. So, to move a file from the Backup share to the Documents share, you could do:# cp -a --reflink /data/Backup/file /data/Documents/# rm /data/Backup/fileSeriously? Really?!So which folder should I put my downloads into? Transmission downloads to the /data/Downloads/ share on my system.Now I have to move all files using an SSH terminal? All because I am using the /data/Videos /data/Music /data/Pictures shares that were there out of the box...?On the NV+, I could of course map to /c/ and use this to move between shares. I had assumed this was the case with the 104 - simply map to /data/How can it be acceptable that ordinary move operations are not possible on this device? What is being done to address this? Aside from forcing users to use a single share? The fact that BTRFS has this limitation is irrelevant - I need to be able to move large files without coping them...!

rotordave wrote:How can it be acceptable that ordinary move operations are not possible on this device? What is being done to address this? Aside from forcing users to use a single share? The fact that BTRFS has this limitation is irrelevant - I need to be able to move large files without coping them...!You apparently missed a post further down: Skywalker wrote:...future firmware releases will support this sort of operation via SMB2's FSCTL_SRV_COPYCHUNK command, which is for server-side copies. But I don't know which SMB clients use that command, other than Windows Server 2012.It seems to me that the web ui could also use the --reflink option.The bottom line is that it works this way for now. Netgear might be able to put in limited support for --reflink in various spots. But this is a tradeoff - the snapshots are also a feature that many users want. The ability to turn snapshots on/off on a share-by-share basis is also needed, and supporting that means that moves between shares (subvolumes) are treated as copy/delete by the file system. Some users will prefer the snapshot control, others don't care about snapshots, and would rather have the direct move. I imagine there could also be a setting when you create the share that doesn't set it up as a subvolume - permanently disabling snapshots. There'd be no way to undo that without deleting the share. You could attempt to do this manually in ssh. Rename "Downloads" temporarily in SSH, and create a Downloads folder manually with the normal permissions. There might be some side effects, but if it works you'd end up with a downloads folder that is not a subvolume.

StephenB wrote:You apparently missed a post further down: Skywalker wrote:...future firmware releases will support this sort of operation via SMB2's FSCTL_SRV_COPYCHUNK command, which is for server-side copies. But I don't know which SMB clients use that command, other than Windows Server 2012.It seems to me that the web ui could also use the --reflink option.The bottom line is that it works this way for now. Netgear might be able to put in limited support for --reflink in various spots. But this is a tradeoff - the snapshots are also a feature that many users want. The ability to turn snapshots on/off on a share-by-share basis is also needed, and supporting that means that moves between shares (subvolumes) are treated as copy/delete by the file system. Some users will prefer the snapshot control, others don't care about snapshots, and would rather have the direct move. I agree that it would be a lot simpler for Netgear to include support for the --reflink option in the GUI, particularly as they can refuse support if SSH access is enabled. As the prime purpose of a NAS is file storage and management I think this is a fairly basic request. I move files between different folders and shares on a pretty regular basis on my main machine and also a WHS server, I'd like to be able to do the same efficiently and immediately on the ReadyNAS. I don't yet fully understand all the implications of the snapshot capability but I believe I read another thread in which a user found that his disk usage seemed too high after moving some files. It turned out that the reason was snapshot storage left over in the folder from which the files had been moved. Deleting those snapshots solved the excess usage issue.If Netgear were to implement the "--reflink" method in the GUI for a move command and add an option to choose to either move or delete the related snapshots that would seem to solve both the above requirements you mention? As I understand it after a "--reflink" move the files would still be stored in the same physical location/blocks so the snapshots would still be relevant, just referenced from a different share?Or am I missing a complication which makes that not possible?

Copy/move between shares is sloooooooooooow...

42 Replies

Replies have been turned off for this discussion

StephenB
Guru - Experienced User
Jul 04, 2014
john_es wrote:
I assume that this won't work across volumes such as main unit to chassis, right?
If you are moving across volumes, then you have to copy/delete.

john_es wrote:
I assume that this won't work across volumes such as main unit to chassis, right?

ddiss

Aspirant

Sep 02, 2014

StephenB wrote:
rotordave wrote:
How can it be acceptable that ordinary move operations are not possible on this device? What is being done to address this? Aside from forcing users to use a single share? The fact that BTRFS has this limitation is irrelevant - I need to be able to move large files without coping them...!

You apparently missed a post further down:
Skywalker wrote:
...future firmware releases will support this sort of operation via SMB2's FSCTL_SRV_COPYCHUNK command, which is for server-side copies. But I don't know which SMB clients use that command, other than Windows Server 2012.

The list of clients capable of issuing SMB2 FSCTL_SRV_COPYCHUNK server-side copy requests is documented on the upstream Samba wiki:
https://wiki.samba.org/index.php/Server-Side_Copy#Client_Support

The request cannot currently be used to offload copies between shares, but nevertheless offers significant performance improvements for intra-share copies.
I recommend that Netgear consider enabling Samba's Btrfs VFS module to expose the Btrfs specific server-side copy enhancements. The module will also offer transparent file compression support in Samba 4.2.

TeknoJnky
Hero
Sep 02, 2014
good info, somehow I missed all this before and ran into the out of space issue when attempting to 'move' media files from a restore folder to their new folder (share).
mdgm-ntgr
NETGEAR Employee Retired
Sep 02, 2014
Currently samba 4.0.x is used by ReadyNAS OS 6 all the way from ReadyNAS OS 6.0.0 through to the latest 6.1.9 RCs.

6.1.9-RC10 includes an update to samba 4.0.21 (currently the latest samba 4.0.x release).

A big update like samba 4.0.x to 4.1.x is not something we'd be likely to add in a minor 6.1.x update as samba 4.0.x is still being maintained so any important security fixes would be included in a samba 4.0.x update.
Roxy1
Apprentice
Oct 03, 2014
This is something I didn't even think to look for when purchasing my Duo. Even if I was looking for it, I question whether I would I have found out that not only can you not copy and paste folders in some of these products' GUIs(have to use command line or adjacent OS folder management), but you also get slow performance because of BTRFS.

And will a future firmware even apply to older products like the Duo 2?

EDIT: This happens to me with intra-share as well, not just between shares?
StephenB
Guru - Experienced User
Oct 03, 2014
Roxy - what product do you have?
Roxy1
Apprentice
Oct 03, 2014
Hey Stephen,

ReadyNas DUO V2

The first day I had it, I did a couple of intra-share copy/paste through Windows Explorer and it was instant, much like when doing it on a standard HDD/SSD.
However on the second day and furthering, it was treating it as a network transfer (speeds 20-30mb/s). Then earlier, I did an intra-transfer and it was instant.

Not sure what's going on. The only thing that varies is that the files I was moving myself were previously put there by me, via manually copying them from another drive(I didn't do a back up operation).
The video file that copied instantly, was put in the original folder by a P2P client and then I moved it myself afterwards.

Which raises the question, if the P2P client stores incomplete downloads in a separate share to the completed ones, what happens when it completes? I watched one and when it finished, it was moved instantly.
Why would that be any different? Does the P2P client have special low level folder permissions?

EDIT: Am doing it now. Moving completed downloads from one directory back to parent directory (intra-share) is instant. If I go over to a different parent folder in the SAME share and repeat the action (using different data of course), it's not instant.
GRR.
StephenB
Guru - Experienced User
Oct 03, 2014
So it sounds like sometimes the move is turning into a copy/delete? The decision on that is made by Windows, not the NAS. The P2P client wouldn't have any special privileges.

Generally if you are moving within the share Windows knows that it can do a move, and not a copy/delete. If you are moving across shares, you need to open the full data volume (\\nasname\c) using admin credentials. If you open the shares separately (\\nasname\share1, \\nasname\share2) then windows will treat the two shares as separate "drives" and do a copy/delete. Similarly if you open \\ipaddr\share1 and \\nasname\share1.
Roxy1
Apprentice
Oct 03, 2014
I did read your suggestion previously on how to mount data volume however it didn't not work. It says Windows cannot access \\NAS-BF-0E-E5\c
StephenB
Guru - Experienced User
Oct 03, 2014
Are you using admin credentials? Try opening a CMD window and entering
net use * /delete
net use z: \\NAS-BF-0E-E5\c /user:admin

That should map the nas data volume to the z drive.

What firmware are you running?

Forum Discussion

Copy/move between shares is sloooooooooooow...

42 Replies

Related Content

ReadyNAS NV+ 4.1.16 - Unable to copy, move, delete, or rename files or folders

Initiate a local copy or move on the server, then disconnect while the task runs to completion

File Move/Copy Leads to Lockup

Cannot Move/Copy folders

Backup (move/copy) share + snapshots to new ReadyNAS?

NETGEAR Academy

ProSupport for Business