NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Cloud backup over Https/webdav...
Hello,
I looking for a backup via https (WebDav French Cloud Service) on Netgear NAS DUO but there is only rsync/ftp/nfs/http ...but no https ... So after read a looooot of technical documentations, comparaisons chart between cloud services etc...I tried to do myself.
So here is the scenario :
- try to install a webdav/https CLIENT on the Nas NETGEAR Duo (sparc)
- try to connect to the cloud service with this one
- try copy (manually ?..) data from NetGear Duo to the cloud
- try to write a script/cron to do this frequently.
So, there is differents webdav clients on Linux (http://www.webdav.org/)
After some studies, I decide to try to install davfs2 to mount my cloud on the NAS then, maybe, to use the original backup fonction (in the frontview of NAS) with this mount.
Let's go :
1) Enable Root on the nas Netgear (check the official add-on)
2) Install APT (official add-on in the Developper Network Forum)
USELESS) try to "apt-get install davfs2"
USELESS ?) be really furious that there isn't this package available
3) download the tar.gz of davfs2 :
http://savannah.nongnu.org/projects/davfs2
http://download.savannah.gnu.org/releases/davfs2/davfs2-1.4.6.tar.gz
Unzip/unpack this one
4 ) launch ./configure and check the error about gcc/libc6 dev & other
tools...
4a) apt-get install gcc, apt-get install libc6dev etc...with the dependancies of course...(little iterative loop between ./configure and apt-get install !...)
4b) re-launch ./configure
4c) check the error of the automatic detection of the configure script
4d) download config.guess and config.sub with the two url
4e) replace in the davfs/config directory config.sub and config.guess with the new version
4f) re-launch ./configure and check the new error about Neon Library (extremely important librairies for webdav/https...)
USELESS) try to "apt-get install libneon*" but there is only the 0.24 version available ...AND YOU MUST HAVE A 0.25
version at least !
So, let's go for another download/configure/make/make install the neon library then we'll back to davfs2 package.
4G) download the last Neon Library :
http://www.webdav.org/neon/
http://www.webdav.org/neon/neon-0.29.6.tar.gz
unzip/unpack this one
then run : ./configure --with-ssl=openssl
the last argument about SSL is important , else you won't be able to use https !
then "make" and "make install"
it's should be alright , back to the DavFS2 directory
4H) re-re-re-run ./configure in the davfs2 package directory
it should be OK !
5) run "make" and "make install"
it should be ok
and you can type this
try to mount -t davfs https://yourservice/yourfolder /localtestfolder ...but you should have a davfs2 user and group error.
6) Go to the FrontView of the Nas and create davfs2 user in the davfs2 group
7) try to mount -t davfs https://yourservice/yourfolder /localtestfolder
if all it's alright, you must type your login then pass of the cloud service then type "y" to accept the certs and ...Congratulations ! the first step is OK ! If you browse the mount point , you should see your cloud directory.
OOOKK....
Now, we'll try to use the backup function of the FrontView with this, so, to do this, we must create a new share with the front view then we'll mount the cloud ON THIS share.
So :
8) create the share in the front view (for example "Cloud")
9a) unmount /localtestfolder
9b) mount -t davfs https://yourservice/yourfolder /c/Cloud
10) configure a backup job between 2 shares , the destination will be "/c/Cloud"
Actually, I got some errors after 2 or 3 hours of backup jobs (about 3,2 Go transferred) but I don't know yet if it's an error due to mount.davfs , my cloud service, the backup job or other...
I just realise that the transfer has stop betweend 2 AM and 3 AM ..THIS NIGHT...WITH THE AUTOMATICALLY WINTER HOUR
UPDATE...ARGH. Maybe a relation...?
I try this morning an incremental backup...after less than one hour, the FrontView tell me that there is error ...but when I check with iftop or with top in ssh, I can see transfer is going on..and there is new files in my cloud.
So, it's not yet perfect but I hope it'll be usefull for some other people.
Now I'll look for about mount point syncrhonisations under linux to bypass the frontview backup...and check the /etc/fstab and other file to automatise the cloud mounting...
BUT THE BEST SOLUTION WILL BE THE OFFICIAL SUPPORT OF WEBDAV/HTTPS SUPPORT FOR BACKUP BY NETGEAR ! It'll be reeeeeaaaaally nice.
Voila
I looking for a backup via https (WebDav French Cloud Service) on Netgear NAS DUO but there is only rsync/ftp/nfs/http ...but no https ... So after read a looooot of technical documentations, comparaisons chart between cloud services etc...I tried to do myself.
So here is the scenario :
- try to install a webdav/https CLIENT on the Nas NETGEAR Duo (sparc)
- try to connect to the cloud service with this one
- try copy (manually ?..) data from NetGear Duo to the cloud
- try to write a script/cron to do this frequently.
So, there is differents webdav clients on Linux (http://www.webdav.org/)
After some studies, I decide to try to install davfs2 to mount my cloud on the NAS then, maybe, to use the original backup fonction (in the frontview of NAS) with this mount.
Let's go :
1) Enable Root on the nas Netgear (check the official add-on)
2) Install APT (official add-on in the Developper Network Forum)
USELESS) try to "apt-get install davfs2"
USELESS ?) be really furious that there isn't this package available
3) download the tar.gz of davfs2 :
http://savannah.nongnu.org/projects/davfs2
http://download.savannah.gnu.org/releases/davfs2/davfs2-1.4.6.tar.gz
Unzip/unpack this one
4 ) launch ./configure and check the error about gcc/libc6 dev & other
tools...
4a) apt-get install gcc, apt-get install libc6dev etc...with the dependancies of course...(little iterative loop between ./configure and apt-get install !...)
4b) re-launch ./configure
4c) check the error of the automatic detection of the configure script
4d) download config.guess and config.sub with the two url
4e) replace in the davfs/config directory config.sub and config.guess with the new version
4f) re-launch ./configure and check the new error about Neon Library (extremely important librairies for webdav/https...)
USELESS) try to "apt-get install libneon*" but there is only the 0.24 version available ...AND YOU MUST HAVE A 0.25
version at least !
So, let's go for another download/configure/make/make install the neon library then we'll back to davfs2 package.
4G) download the last Neon Library :
http://www.webdav.org/neon/
http://www.webdav.org/neon/neon-0.29.6.tar.gz
unzip/unpack this one
then run : ./configure --with-ssl=openssl
the last argument about SSL is important , else you won't be able to use https !
then "make" and "make install"
it's should be alright , back to the DavFS2 directory
4H) re-re-re-run ./configure in the davfs2 package directory
it should be OK !
5) run "make" and "make install"
it should be ok
and you can type this
try to mount -t davfs https://yourservice/yourfolder /localtestfolder ...but you should have a davfs2 user and group error.
6) Go to the FrontView of the Nas and create davfs2 user in the davfs2 group
7) try to mount -t davfs https://yourservice/yourfolder /localtestfolder
if all it's alright, you must type your login then pass of the cloud service then type "y" to accept the certs and ...Congratulations ! the first step is OK ! If you browse the mount point , you should see your cloud directory.
OOOKK....
Now, we'll try to use the backup function of the FrontView with this, so, to do this, we must create a new share with the front view then we'll mount the cloud ON THIS share.
So :
8) create the share in the front view (for example "Cloud")
9a) unmount /localtestfolder
9b) mount -t davfs https://yourservice/yourfolder /c/Cloud
10) configure a backup job between 2 shares , the destination will be "/c/Cloud"
Actually, I got some errors after 2 or 3 hours of backup jobs (about 3,2 Go transferred) but I don't know yet if it's an error due to mount.davfs , my cloud service, the backup job or other...
I just realise that the transfer has stop betweend 2 AM and 3 AM ..THIS NIGHT...WITH THE AUTOMATICALLY WINTER HOUR
UPDATE...ARGH. Maybe a relation...?
I try this morning an incremental backup...after less than one hour, the FrontView tell me that there is error ...but when I check with iftop or with top in ssh, I can see transfer is going on..and there is new files in my cloud.
So, it's not yet perfect but I hope it'll be usefull for some other people.
Now I'll look for about mount point syncrhonisations under linux to bypass the frontview backup...and check the /etc/fstab and other file to automatise the cloud mounting...
BUT THE BEST SOLUTION WILL BE THE OFFICIAL SUPPORT OF WEBDAV/HTTPS SUPPORT FOR BACKUP BY NETGEAR ! It'll be reeeeeaaaaally nice.
Voila
4 Replies
- So, after a day of test - with fiber optic (100 Mo / 50 Mo - Theoric data download/upload)
- rsync manually on the nas of 750 mo = 1h30 hour
- backup jobs (local share to davfs mount via rsync) with frontview of 750 Mo and with enable compression and co = 2h30 hour
- backup job (local share to davfs mount NOT VIA RSYNC) = 1H30
- copy command under Midnight Commander = about 1h30 hour
and
- directly upload from a WinXP computer to the Cloud (without NAS) of 350 Mo = 5-10 min
?....
So I got a real problem with the performance , for example, now i run a incremental backup (diff 4.4 Gb) and with "top" commande I can see that the load is extremely high :top - 09:41:50 up 2 days, 16:56, 1 user, load average: 3.22, 3.82, 3.46
Tasks: 84 total, 5 running, 79 sleeping, 0 stopped, 0 zombie
Cpu(s): 19.6% us, 8.0% sy, 8.9% ni, 62.4% id, 0.2% wa, 0.2% hi, 0.6% si
Mem: 226256k total, 215744k used, 10512k free, 10384k buffers
Swap: 255968k total, 1952k used, 254016k free, 44416k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6173 davfs2 25 0 20432 10m 2672 R 80.1 4.9 716:33.28 mount.davfs
22615 root 16 0 2976 1552 1216 R 10.4 0.7 0:00.16 top
17498 root 36 19 49568 34m 2032 R 7.0 15.7 12:50.13 minidlna
1 root 15 0 2000 880 768 S 0.0 0.4 0:06.94 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.07 ksoftirqd/0
Even before the install, I got bad perf (squeezebox server slow...), is it normal that the CPU Charge slow down the upload ?
or I should change some conf in the davfs2.conf ?
(or I should buy a new Netgear product with more mem & cpu ... :roll: ) - After the backup job, load is normal (under 0.5) ...
CPU is high during minidlna, squeezebox server, httpd and transmission process/inter-action. - Nice write-up!
I'm looking into jumping into the water with trying out backing up to a WebDAV cloud service (OnlineFileFolder, in my case), and I was really impressed that you shared your experience. Thank you for taking the extra time and effort.
Overall, this whole process is more complicated than it should be: I wish NetGear/ReadyNAS would implement official support for this. - Thanks for the great tutorial!
I'm trying to achieve the same here, this time with ovh's hubic service. I'm already lost on step 4 to install davfs2 on the Readynas. Could you explain a little bit more step 4G for someone who's not too familiar with the ssh commands?
I see that it's been about six month you posted your setup. I wanted to ask you how it's been working since then? Is it a workable solution to create a copy of a readynas share on the cloud?
Thanks, and let's hope Netgear adds official support for webdav one day...
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
