NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
ReadyNAS Backup Scheduling
I have multiple shares that are backed-up weekly from one NAS to another. Recently, this has become an issue because all the backups are configured to do a full followed by 3 incremental, which means...
db.sq3 has a 'Backup' table in it, which contains the same information as the backup_jobs.conf file. Couldn't find anything that records dates or anything in there.
I've started writing a script to do what I want as I can't see a way of doing it either through the GUI or in pure cron. It's obviously not finished, so don't be too critical 😀
#!/bin/bash
# Required backup pattern is defined below:
# F=Full, I=Incremental, S=Skip
# In order of Wk1, Wk2, Wk3, Wk4
job_1=("F" "I" "I" "I")
job_2=("F" "I" "I" "I")
job_3=("F" "I" "I" "I")
job_4=("F" "I" "I" "I")
job_5=("I" "F" "I" "I")
job_6=("I" "I" "F" "I")
job_7=("I" "I" "I" "F")
# Check if there is an existing timekeeper file
tspath="/data/Backup/timekeeper"
# If the timekeeper file exists...
if [ -f "$tspath" ]; then
# For debug
echo "Timekeeper file exists"
else
# But if it doesn't...
# For debug
echo "Timekeeper file does not exist"
echo "Initialising with this week as Wk1"
# ...assume that this is week 1, so get the week number...
wk1=`date +%U`
# ...and write that to the timekeeper file.
echo $wk1 > "$tspath"
# For debug
echo "First week = $wk1"
# Work out week numbers
# This is done by dividing the week number by 4 and recording the remainder. This allows us to work out which week of the 4 week cycle we're on, for instance:
# Say we're in week 38 when we first run, so that becomes cycle week 1. 38/4 = 9.50, so any week divided by 4 that has a 50 remainder is a cycle week 1.
# Which means that a week number with a remainder of 75 is cycle week 2, 00 is cycle week 3 and 25 is cycle week 4.
# Loop for a 4 week cycle
for ((i = 1 ; i < 5 ; i++)); do
# Bash can only do integer maths, so multiply week number by 100 before dividing by 4...
wkrem=$((${wk1}00/4))
# For debug
echo "Week $i remainder = ${wkrem: -2}"
# ...and write the remainder (either 00, 25, 50 or 75) for this week number to the timekeeper file.
echo ${wkrem: -2} >> "$tspath"
# Increment firstweek to work out the remainder for the next week.
wk1=$((++wk1))
done
fi
# Find out what week number number we're in
thisweek=`date +%U`
# Bash can only do integer maths, so multiply week number by 100 before dividing by 4.
thisweek=$((${thisweek}00/4))
# Just need the remainder to work out which week we're in.
tweek="${thisweek: -2}"
# Find out which line number in the timekeeper file the same remainder appears on.
thisweeknumber=$(grep -n $tweek timekeeper | cut -d: -f1)
# The first line of the timekeeper file is the first week number, so the cycle week number is actually one less
thisweeknumber=$((--thisweeknumber))
# For debug
echo "This is week $thisweeknumber"
# Decrement thisweeknumber again, as arrays indexes start at 0, not 1
thisweeknumber=$((--thisweeknumber))
# Generate this weeks jobs
echo "Job 1 = ${job_1[${thisweeknumber}]}"
echo "Job 2 = ${job_2[${thisweeknumber}]}"
echo "Job 3 = ${job_3[${thisweeknumber}]}"
echo "Job 4 = ${job_4[${thisweeknumber}]}"
echo "Job 5 = ${job_5[${thisweeknumber}]}"
echo "Job 6 = ${job_6[${thisweeknumber}]}"
echo "Job 7 = ${job_7[${thisweeknumber}]}"But any efficiencies that could be added without it making it too difficult to understand would be appreciated.
Again, the simplest approach is to create a full backup job and a purely incremental backup job in the web ui. Then create a custom frontview-backup file in /etc/cron.d that schedules both jobs as you wish. No need for a timekeeper script.
frontview-backup entries on one of my systems look like this:
5 00-23/24 * * mon,wed,fri root /frontview/bin/fvbackup 001 &> /dev/nullThis starts the backup at 5 minutes after the hour on monday, wednesday and friday (that particular NAS's power schedule tells it to turn on at 11 pm Sunday, Tues, and Thurs, and turn off at 1 am Monday, Weds, and Friday).
It's pretty easy to modify this to run the backup jobs on any schedule you want.
OK, I create two backups - one full (always) (001) and one full (never) (002), both for the same share.
How can I schedule 001 to run on the first Monday, then for the next three Mondays I want 002 to run instead, then repeat, using cron?
The first part seems easy:
1 0 * * 1 root /frontview/bin/fvbackup 001 &> /dev/nullI think this says run job 001 at 00:01 on every Monday. But that's not right, because I need to NOT run it on one of those Mondays...how?
The second part has the same problem, just inverted.
I just couldn't see how to do it, hence scripting.
daKlone wrote:
1 0 * * 1 root /frontview/bin/fvbackup 001 &> /dev/nullI think this says run job 001 at 00:01 on every Monday. But that's not right, because I need to NOT run it on one of those Mondays...how?
I'd set up the full backups to run on the specific week of the month, and the incremental backups to run on other weeks.
For instance, if you want the full backups to run on the first week, you can adapt the schedule to
1 0 1-7 * * root [ $(date +\%u) = 1 ] && /frontview/bin/fvbackup 001 &> /dev/nullCron will run this job on the first 7 days of the month, the date check added to the command itself results in the backup portion of the command only running on Monday.
While you might think that you can avoid the date check with 1 0 1-7 * 1, you can't. It's been a while since I played with this, but my recollection is that results in the cron job running on days 1-7 of the month and also on every Monday. You can add in months if you need more spacing between the full backups though.
Similarly, you can run incremental jobs using
1 0 8-31 * * root [ $(date +\%d) = 1 ] && /frontview/bin/fvbackup 001 &> /dev/null
daKlone wrote:
1 0 * * 1 root /frontview/bin/fvbackup 001 &> /dev/nullI don't think you started with a schedule created by the NAS web ui. I suggest you do that, and use the first two parameters you see in the NAS-created schedule. My sample was an excerpt of what I see ( 5 00-23/24 * * ) - not sure if the details depend on whether snapshots are running or not, but I am thinking the NAS more likely always uses these parameters for daily backups .
Nice work StephenB! 👍
If I'm reading it right then that almost works, but falls over when a month has five Mondays in it. You end up doing four incremental backups instead of three (plus an additional full backup of the last job).
An example of this is January 2023:
02/01/23 Monday - Full
09/01/23 Monday - Inc
16/01/23 Monday - Inc
23/01/23 Monday - Inc
30/01/23 Monday - Inc
Next year this would also happen in May, July & November so 30% of the time it doesn't work as intended.
Does that really matter? No, probably not, but I set myself the task of figuring out how to
make it as difficult as possibledo it.EDIT: It's probably obvious, but I forgot to mention that my script would be fired by a cron entry every Monday at 00:01.
daKlone wrote:
If I'm reading it right then that almost works, but falls over when a month has five Mondays in it. You end up doing four incremental backups instead of three (plus an additional full backup of the last job).
That is almost correct (there are four months each year with 5 weeks). No additional full backups though, just four months with 4 incremental ones.
But as you say doesn't really matter. And it's far simpler than what you are attempting.
Rewriting the backup schedule in cron.d is a little tricky when the system is running - you'll need to restart the crond daemon.
daKlone wrote:
EDIT: It's probably obvious, but I forgot to mention that my script would be fired by a cron entry every Monday at 00:01.
Your sample looked like it was triggering a backup job every Monday at 00:01. As I mentioned, I'd go with the parameters the NAS web ui uses instead - especially if you are putting the NAS on a power schedule.
Doh, must work on my communication skills!
When I said an additional Full, I meant that the last job - the one which was Full on week 4 (job_7) - would be repeated on week 5 thereby doing an additional Full.
job_1=("F" "I" "I" "I") job_2=("F" "I" "I" "I") job_3=("F" "I" "I" "I") job_4=("F" "I" "I" "I") job_5=("I" "F" "I" "I") job_6=("I" "I" "F" "I") job_7=("I" "I" "I" "F")That wasn't clear from my post, sorry.
Again, it was confusing when I mentioned that the timekeeper would be scheduled using cron. I didn't mean by the example entry that I'd previously given, the one with fvbackup in it, but a simple one something like this added in it's own file in cron.d (or directly in crontab even):
1 0 * * 1 /path/to/timekeeper.sh
That's all that would be needed, as the script will (when it's finished) do the rest including kicking off fvbackup with the appropriate backup definition number.
I'm going to carry on
torturing myselfworking on the script anyway, for education if nothing else.
daKlone wrote:
When I said an additional Full, I meant that the last job - the one which was Full on week 4 (job_7) - would be repeated on week 5 thereby doing an additional Full.
job_1=("F" "I" "I" "I") job_2=("F" "I" "I" "I") job_3=("F" "I" "I" "I") job_4=("F" "I" "I" "I") job_5=("I" "F" "I" "I") job_6=("I" "I" "F" "I") job_7=("I" "I" "I" "F")That wasn't clear from my post, sorry.
To clarify, my approach is using days of the month combined with a day of the week to drive the full backup jobs - the four date ranges being
- 1-7 (week 1)
- 8-14 (week 2)
- 21-27 (week 3)
- 22-28 (week 4)
No full backups scheduled for the 29-31, so a full backup would only be run once a month. If you did extend the fourth date range to 22-31, then you would sometimes get repeated full backups for the extended week 4.
You could run incremental backups similarly (running 3 incremental backups a month, and skipping the 4th in 5-week months), or you could extend the date range for incremental to 31.
Note I don't do this for backups, but I have made this mod for the 4 maintenance functions (disk test, balance, scrub, and defrag). It ensures that they start during the window when the NAS is powered up (per its power schedule). I made another mod to the power off script to make sure the NAS remains powered until the maintenance functions complete (which unfortunately wasn't done by Netgear).
daKlone wrote:
1 0 * * 1 /path/to/timekeeper.sh
That's all that would be needed, as the script will (when it's finished) do the rest including kicking off fvbackup with the appropriate backup definition number.
I use daily custom snapshots on my backup NAS to give retention (currently set to 3 weeks on most of my shares). If you do that, you want to make sure the snapshot schedule is set to either make the snapshot before the backup job starts, or after it finishes (not in the middle of the backup). Mine are set to make the snapshots before the backups start.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
