× NETGEAR will be terminating ReadyCLOUD service by April 1st, 2023. For more details click here.
Reply

Enabling bit-rot protection for files on existing shares

ArjenR49
Aspirant

Enabling bit-rot protection for files on existing shares

My RN212 has two 2TB drives in RAID1. I have originally set up the shares - I don't use the home folders - without bit-rot protection.

I have now enabled it on some of the shares, but as I understand it is only enabled on new files.

Therefore I copied the folders on a share and pasted the folder with all its files in the same share. The copies get renamed automatically to "... (1)". I moved the original folders into a NoChecksum folder to be erased later, and renamed the copies back to the original names.

 

On a remote Raspberry Pi server I have an rsync job running every now and then to make off-site backups of the RN212 shares. These backups reside on two disks in a BTRFS configuration.

 

As the NoChecksum folder on the RN212 share is excluded in the rsync command, rsync detects nothing new to copy from the RN212 (unless files are deleted or written etc. on the RN212). As far as the remote server and rsync are concerned the folders and files are unchanged. This is what I hoped for because, if it were to copy all content as new, that would take a very long time, because the line between RN212 and Raspberry Pi is just 1 MB/s ...

 

However, has my copy operation on the RN212 as described actually enabled the bit-rot protection on the copied files, i.e. are they 'new files' as far as the checksum etc. is concerned?

Is there a command that will show if files have bit-rot protection and checksums enabled?

 

Netgear specifies somewhere that the way to enable bit-rot protection on an existing file requires copying a file within the same share giving the copy a new name and going from there ... For thousands of files this would be near impossible to do.

 

Thanks for any help and suggestions.

Arjen

Model: RN212|2 BAY Desktop ReadyNAS Storage
Message 1 of 3
StephenB
Guru

Re: Enabling bit-rot protection for files on existing shares


@ArjenR49 wrote:

My RN212 has two 2TB drives in RAID1. I have originally set up the shares - I don't use the home folders - without bit-rot protection.

 

I have now enabled it on some of the shares, but as I understand it is only enabled on new files.

Therefore I copied the folders on a share and pasted the folder with all its files in the same share. The copies get renamed automatically to "... (1)". I moved the original folders into a NoChecksum folder to be erased later, and renamed the copies back to the original names.

 

...

 

However, has my copy operation on the RN212 as described actually enabled the bit-rot protection on the copied files, i.e. are they 'new files' as far as the checksum etc. is concerned?

Is there a command that will show if files have bit-rot protection and checksums enabled?

 


It should have computed the checksums when you copied the files.  But I don't see any mechanism in btrfs to verify that the checksums are there - just mechanisms to verify that (when present) they are correct.  Note the checksums are on written data blocks, not on files.

 


@ArjenR49 wrote:

 

Netgear specifies somewhere that the way to enable bit-rot protection on an existing file requires copying a file within the same share giving the copy a new name and going from there ... For thousands of files this would be near impossible to do.

 


Not sure where Netgear says this.  But if you have enough space, you can create a new share with bit-rot protection on, and then use an RN212 backup job to copy the files to the new share.  Then delete the old share, and rename the new one to match the original.

 

Note there is an ssh command that will recreate all the btrfs checksums, though it is considered dangerous.  https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-check

 

 

Message 2 of 3
ArjenR49
Aspirant

Re: Enabling bit-rot protection for files on existing shares

https://kb.netgear.com/26091/Bit-rot-Protection-and-Copy-on-Write-COW-in-depth

quote:

Switching COW on to COW off on a file that has data

So if you have a file that has COW enabled, and you want to switch it to COW disabled (or vice-versa), the only way to do it:

  1. Disable COW on the share (or enable COW if you're doing the reverse operation)
  2. Copy the file to a new location in the share (it can be within the same folder), just make sure the new copy has a different name.
  3. Confirm the copy is good.
  4. Afterwards, delete the original file.
  5. Rename the new copy to the original name.

end of quote

 

It speaks of copying one file and giving a different name to the copy ...

Also, I was thinking that maybe the filesystem will just create a hardlink when copying the folders with the files in them, so the copy isn't actually a new file, but using the blocks that are already there ...

However, I really don't know much about that mechanism in Linux.

 

Unfortunately the btrfs command to check the checksums calls for unmounting the drives. I can do that on my Rapberry Pi NAS with its two BTRFS drives in tandem, since I essentially built it myself, but on the ReadyNAS ... I don't think I want to consider that on my skill level ...

I was bold enough to enable SSH and use RSync.

 

 

Recently I have changed ownership and group ownership on some shares and tried to get the permissions work as I need. I have had to study the way permissions work and still not totally sure.

What I did notice, is that the reset function in the GUI has an explanation that seems quite incorrect. Reset adjusts file permissions to be what they are set to be in the GUI, not to the defaults (guest:guest and so on) as I understand the help text.

After all, one can change permissions e.g. using FileZilla for a file on the NAS and pass by the GUI.

Also, if you happen to use a user account in FileZilla different from the owner account of the share for, say, SFTP'ing new files to a share, those files will not have the same permissions as files that were already there, possibly breaking rsync backup routines. It also changes directory owner and group etc.

Reset in the GUI seems to be an essential tool, which I find strange. Perhaps it is due to something I haven't understood. Masking, umask and what have you ....

 

 

 

Message 3 of 3
Top Contributors
Discussion stats
  • 2 replies
  • 1095 views
  • 0 kudos
  • 2 in conversation
Announcements