NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.

Forum Discussion

metapaso's avatar
metapaso
Apprentice
Sep 21, 2015
Solved

BTRFS and rsync --inplace what is the the default behavior and what is the optimal behavior?

Hi All,   I've just set up a few RSYNC backups from my new RN316.  I was poking around to see what options the built-in rsync backup uses.   1) From what I read elsewhere, I am lead to believe th...
  • StephenB's avatar
    Sep 21, 2015

    The NAS is not using this option now.

     

    Per the rsync documentation, this is what the option actually does:


    --inplaceThis option changes how rsync transfers a file when its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file.

    This has several effects:

     

    • Hard links are not broken. This means the new data will be visible through other hard links to the destination file. Moreover, attempts to copy differing source files onto a multiply-linked destination file will result in a "tug of war" with the destination data changing back and forth.
    • In-use binaries cannot be updated (either the OS will prevent this from happening, or binaries that attempt to swap-in their data will misbehave or crash).
    • The file's data will be in an inconsistent state during the transfer and will be left that way if the transfer is interrupted or if an update fails.
    • A file that rsync cannot write to cannot be updated. While a super user can update any file, a normal user needs to be granted write permission for the open of the file for writing to be successful.
    • The efficiency of rsync's delta-transfer algorithm may be reduced if some data in the destination file is overwritten before it can be copied to a position later in the file. This does not apply if you use --backup, since rsync is smart enough to use the backup file as the basis file for the transfer.

    WARNING: you should not use this option to update files that are being accessed by others, so be careful when choosing to use this for a copy.

    This option is useful for transferring large files with block-based changes or appended data, and also on systems that are disk bound, not network bound. It can also help keep a copy-on-write filesystem snapshot from diverging the entire contents of a file that only has minor changes.

    The option implies --partial (since an interrupted transfer does not delete the file), but conflicts with --partial-dir and --delay-updates. Prior to rsync 2.6.4 --inplace was also incompatible with --compare-dest and --link-dest.


     

    The section I underlined is a significant drawback, especially for people using rsync over ssh to update to a remote location.  So if Netgear changes this, hopefully they will make it an option.

     

    How this impacts CoW depends on another option, you'd also need to specify --no-whole-file if your goal is to minimize snapshot size.

     

    The combination should reduce snapshot size.  As is always the case with CoW, it also increases fragmentation of the main file. 

     

    metapaso wrote:

     

    3) If I convince myself that --inplace is the proper option for my setup...

     

     

    Well, the main risk I see is that a failed backup will leave you with an partially updated (inconsistent) file, instead of just leaving the older version of the file on the backup device.

     

    The potential benefit is only realized if the files are updated in a way that leaves most of the file blocks unchanged.  If you are inserting something new at the beginning of a file, then it likely won't help (since the old data shifts, so rsync will generally think it is new).  

     

    I don't think it would make much difference for my own shares - partly because there isn't a lot of churn in the first place (files are added, but not often modified).  And the modifications I make to the largest files (media files) are generally adding tagging - which is normally inserted at the beginning of the file.

     


    metapaso wrote:

     

    ... where are the locations of the rsync scripts that OS 6 uses so I can add this myself.  Yes I know this may break software support.

     

     


    At the moment, I think you'd have to use your own rsync commands.  AFAIK there is no way to modify any base script.

     

NETGEAR Academy

Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology! 

Join Us!

ProSupport for Business

Comprehensive support plans for maximum network uptime and business peace of mind.

 

Learn More