× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

ReadyNAS 426 - Extremely fast internal copy!

AndreasBoozon
Follower

ReadyNAS 426 - Extremely fast internal copy!

I just bought a ReadyNAS 426 running OS 6.9.4.

 

Just to test it I put two WD Red 2TB disks into it.

 

Copying to and from the NAS gives expected results (constant 113 MB/s, which is what my network can handle I guess).

 

However, when copying files internally (I only have one volume), I get speeds of 15-20 GB/s!

(I repeat: 15-20 GB/s!!! That's over 100 Gb/s!)

And I have verified that the copied file is correct.

I managed to copy 8 files of 40 GB (each!) in less that half a minute.

How is this even physically possible?!

 

(The only explanation I have managed to come up with is that it does not copy the data, but only the pointers to the data blocks, and that all copies use the same data blocks. And copying the pointers does not take that long time. Otherwise I don't see how it would be physically possible...)

Does anybody (preferably from NetGear) know?

Message 1 of 2

Accepted Solutions
StephenB
Guru

Re: ReadyNAS 426 - Extremely fast internal copy!


@AndreasBoozon wrote:

The only explanation I have managed to come up with is that it does not copy the data, but only the pointers to the data blocks, and that all copies use the same data blocks. 

That is almost correct.  The data blocks weren't actually copied, but the new file metadata points to the same datablocks as the original file's metadata - sharing the data blocks.  This is taking advantage of the BTRFS copy-on-write (CoW) architecture.

 

If you do some more testing, you'll also find that the free space on the volume isn't changing.

 

If you were to alter a block in the middle of one of the files, then that file would become fragmented - sharing the unmodified blocks with the others, but requiring a new block to hold the change.  Defragging that modified file would eliminate the sharing of blocks - they would be duplicated by the file system as part of the defrag operation.

 

The snapshot feature of BTRFS also uses this concept.  A snapshot takes essentially no space in the beginning because all the data blocks are shared with the main share.  As time goes on (and files are deleted or modified), the snapshot begins to take up space in order to store the delta from the main share.  If you delete a file in the main share, it also doesn't change the free space (since the file is still in the snapshots).  If you modify it, the main copy again becomes fragmented (with the original unfragmented copy being in the snapshots).

 

Very cool idea.  But free space often doesn't turn out the way you'd expect.

View solution in original post

Message 2 of 2

All Replies
StephenB
Guru

Re: ReadyNAS 426 - Extremely fast internal copy!


@AndreasBoozon wrote:

The only explanation I have managed to come up with is that it does not copy the data, but only the pointers to the data blocks, and that all copies use the same data blocks. 

That is almost correct.  The data blocks weren't actually copied, but the new file metadata points to the same datablocks as the original file's metadata - sharing the data blocks.  This is taking advantage of the BTRFS copy-on-write (CoW) architecture.

 

If you do some more testing, you'll also find that the free space on the volume isn't changing.

 

If you were to alter a block in the middle of one of the files, then that file would become fragmented - sharing the unmodified blocks with the others, but requiring a new block to hold the change.  Defragging that modified file would eliminate the sharing of blocks - they would be duplicated by the file system as part of the defrag operation.

 

The snapshot feature of BTRFS also uses this concept.  A snapshot takes essentially no space in the beginning because all the data blocks are shared with the main share.  As time goes on (and files are deleted or modified), the snapshot begins to take up space in order to store the delta from the main share.  If you delete a file in the main share, it also doesn't change the free space (since the file is still in the snapshots).  If you modify it, the main copy again becomes fragmented (with the original unfragmented copy being in the snapshots).

 

Very cool idea.  But free space often doesn't turn out the way you'd expect.

Message 2 of 2
Top Contributors
Discussion stats
  • 1 reply
  • 690 views
  • 2 kudos
  • 2 in conversation
Announcements