NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Excessive bandwidth using S3 backup feature
I have a ReadyNAS 316 (6.9.3) and have been using the built-in S3 backup utility for three of the shares on the NAS. These shares total 3TB of data, and all of that data is being successfully replica...
I checked with our Cloud service dev and it looks like he was able to get the information needed from Olicuk's extra details. There was some additional testing that we were checking for the calls/intervals in this case.
The calls are smaller in smaller datasets, but are significantly greater with larger data sets. When running a sync check every 30 seconds with data/capacity this size the return information is in the 100s of KB. We are looking into a configurable option so you are able to select the interval in the services that are more bandwidth aware.
Let me verify with the team a possible work around for the interval option. There are some cases where there is a setting that gets overwritten.
Thank you OOM-9 . So the NAS does a sync check every 30 secs, that's some workload? I guess the 70GB/mo equates to about 2.4GB/day or 100Mb/hour... or 1.65Mb/min.... so that's potentially two checks returning 800Kb each? For my needs a sync check every 30 mins, or even 2 hours would suffice, and even once per 24h would be ok. It would reduce bandwidth and request costs to almost certainly stay within the free tier..... unless I ever need to pull down the data archived. Though I also have a concern over what happens if I increase my S3 storage usage from the current 500GB, to archive all 3.6TB of data on my NAS. What would happen to the calls then as it sounds like they almost increase in size exponentially?
This has got me looking far more at the full costs of cloud storage and the various options on the ReadyNAS, about which I'll create a separate thread.
It seems like having our initial design of something similar to Dropbox sync type of behavior does not really fit the profile with these type of services. We are going to be bringing the interval option to these types of services (S3, Azure and Wasabi) that are more bandwidth aware. We will make it flexible enough in the cases you might want to sync/backup once a day (or more). Based on some of the calls that we were checking in our testing each call was estimated to be 273KB which adds up when we check every 30 seconds.
Since the option is not available in the UI yet, there will need to be a configuration change on in the CLI. Changing this configuration in the CLI will stay until there is a UI change to the service.
Example: Configuring the interval to every 24 hours and there was a new sync/backup destination added/edited. You will need to go back into the CLI to correct the interval configuration back to the 24 hours setting.
That information in mind... There are two option depending on how comfortable you are on the CLI. You can message me with Secure Diagnostics Mode ID to make the change if the current configuration is not going to change. You will need to mention the desired interval.
Alternatively, the following are the steps you would help extend the time of syncing to daily:
sed -i -e 's/"pollIntervalInSeconds": 30,/"pollIntervalInSeconds": 86400,/g' /etc/frontview/cloud-storage/s3.conf systemctl restart amazon-s3
This is chaning the "pollIntervalInSeconds" value and restarts the service.
(This option is available and works with S3, Azure and Wasabi with their respectable service name.)
Edit: Corrected the command that changes the config. Previous config change the Google Drive for hourly.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
