× NETGEAR will be terminating ReadyCLOUD service by July 1st, 2023. For more details click here.
Orbi WiFi 7 RBE973
Reply

Subversion : some progress

zfil
Aspirant

Subversion : some progress

Hello

I've managed to build a working subversion server :

wget http://subversion.tigris.org/downloads/ ... 5.0.tar.gz
wget http://subversion.tigris.org/downloads/ ... 5.0.tar.gz

First untar both previously downloaded tgz.

cd subversion-1.5.0/zlib
./configure && make test && make install
cd ..
./configure --build=sparc-linux --enable-all-static --with-ssl --without-serf --with-zlib=/usr/local
make
make check
make install

56 tests out of 56 must run successfully.

The key is to build static otherwise svn and svnsync command coredump ...


The problem now is that the subversion server is very very slow, it is not a cpu problem, more a networking I think (the cpu is idle).

I'm using svnserve (svnserve -d), it is my usual setup on other platform and I never experienced any slowness (but it wasn't svn 1.5).
Message 1 of 42
steverweber
Aspirant

Re: Subversion : some progress

Good job!! best SVN news in a long time.

If svnserve is slow perhaps, you could try using (http: https:) libapache2-svn.
I think apache does not use svnserve. I could be wrong :(.

Once the slow bug is solve..
It would be grate if the updated binaries could be added to apt-get for everyone to enjoy..

keep up the good work 🙂
Message 2 of 42
zfil
Aspirant

Re: Subversion : some progress

Okay

I think I've found the problem ...

The slowness is due to /dev/random that is VERY slow on this machine.

# dd if=/dev/random bs=1k count=1 >/dev/null
0+1 records in
0+1 records out
9 bytes transferred in 16.719820 seconds (1 bytes/sec)


Okay I can confirm this is really the problem ...
I've replaced /dev/random as /dev/urandom and now it's very fast.

I'm recompiling apr to use /dev/urandom so ...

🙂
Message 3 of 42
steverweber
Aspirant

Re: Subversion : some progress

Nice find!

I wounder if dev/random should be symbolic linked to urandom, so all dependent programs get a boost. Well until dev/random is fixed.
I don't see the less random dev/urandom as a large security problems.

Is their some ReadyNAS dev around that can add your bins as some sorta plug-in for us all to use?
don't get me wrong, cross compiling is fun... but it takes CPU cycles away from the video games and video watching.
Message 4 of 42
JimmyB1
Aspirant

Re: Subversion : some progress

Yes, could it be packaged in a nice addon (.bin) now the add-on API has been released?
Message 5 of 42
rocket15
Aspirant

Re: Subversion : some progress

Good news...

Did you use the following configure statement?
./configure --build=sparc-linux --enable-all-static --with-ssl --without-serf --with-zlib=/usr/local --with-devrandom=/dev/urandom
Message 6 of 42
samclement
Aspirant

Re: Subversion : some progress

Sounds pretty exciting... will you keep us updated on this thread?
Message 7 of 42
zfil
Aspirant

Re: Subversion : some progress

Yeah don't worry I'm a little busy but I will provide a tarball with the working binaries.

Then I will work on an addon version.

Philippe
Message 8 of 42
_mj
Aspirant
Aspirant

Re: Subversion : some progress

great to hear zfil, having a svn server running on the ReadyNAS seems like a perfect fit
Message 9 of 42
Dogberry1
Aspirant

Re: Subversion : some progress

zfil wrote:
The slowness is due to /dev/random that is VERY slow on this machine.

"When the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered. (Source: Linux Programmer's Manual, section 4)"

So apparently the environmental noise density contributing to the entropy pool is very slight on these boxes. Using /dev/urandom seems like a perfectly acceptable alternative here.

/* sidebar:
I believe it was Robert Coveyou who said "The generation of random numbers is far too important to be left to chance." But my favorite geekjoke about random numbers is in the form of a cartoon depicting a random-number generator function like this:

int getRandomNumber()
{
return 4; // chosen by fair dice roll
// guaranteed to be random
}

*/
Message 10 of 42
gromeen
Aspirant

Re: Subversion : some progress

Disclaimer: I'm new to ReadyNAS and linux.
I'm trying this with the latest subversion 1.5.1 files. This is what I've done so far:

1. started with a fresh factory default ReadyNAS (RAIDiator 4.1.3-T136 [1.00a146] in X-RAID)
2. downloaded EnableRootSSH and APT from here
3. installed those .bin files via FrontView, System>Update>Local
4. logged into my ReadyNAS via SSH (putty)
5. apt-get update
6. apt-get install nano (I'm not a vi wizard)
7. apt-get install gcc
8. wget http://subversion.tigris.org/downloads/subversion-1.5.1.tar.gz
9. wget http://subversion.tigris.org/downloads/subversion-deps-1.5.1.tar.gz
10. tar xvzf subversion-1.5.1.tar.gz
11. tar xvzf subversion-deps-1.5.1.tar.gz
12. rm *.gz (why keep the tarballs?)
13. cd subversion-1.5.1/zlib/
14. ./configure && make test && make install

This is where I get stuck. Here is the error dump:

Checking for gcc...
Building static library libz.a version 1.2.3 with gcc.
Checking for unistd.h... No.
Checking whether to use vs[n]printf() or s[n]printf()... using s[n]printf()
Checking for snprintf() in stdio.h... No.
WARNING: snprintf() not found, falling back to sprintf(). zlib
can build but will be open to possible buffer-overflow security
vulnerabilities.
Checking for return value of sprintf()... No.
WARNING: apparently sprintf() does not return a value. zlib
can build but will be open to possible string-format security
vulnerabilities.
Checking for errno.h... No.
Checking for mmap support... No.
gcc -O3 -DNO_snprintf -DHAS_sprintf_void -DNO_ERRNO_H -c -o example.o example.c
example.c:8:19: stdio.h: No such file or directory
example.c:12:22: string.h: No such file or directory
example.c:13:22: stdlib.h: No such file or directory
example.c: In function `test_compress':
example.c:67: error: `stderr' undeclared (first use in this function)
example.c:67: error: (Each undeclared identifier is reported only once
example.c:67: error: for each function it appears in.)
example.c: In function `test_gzio':
example.c:99: error: `NULL' undeclared (first use in this function)
example.c:100: error: `stderr' undeclared (first use in this function)
example.c: In function `test_deflate':
example.c:182: error: `stderr' undeclared (first use in this function)
example.c: In function `test_inflate':
example.c:225: error: `stderr' undeclared (first use in this function)
example.c: In function `test_large_deflate':
example.c:260: error: `stderr' undeclared (first use in this function)
example.c: In function `test_large_inflate':
example.c:320: error: `stderr' undeclared (first use in this function)
example.c: In function `test_flush':
example.c:357: error: `stderr' undeclared (first use in this function)
example.c: In function `test_sync':
example.c:399: error: `stderr' undeclared (first use in this function)
example.c: In function `test_dict_deflate':
example.c:438: error: `stderr' undeclared (first use in this function)
example.c: In function `test_dict_inflate':
example.c:480: error: `stderr' undeclared (first use in this function)
example.c: In function `main':
example.c:524: error: `stderr' undeclared (first use in this function)
make: *** [example.o] Error 1

Any ideas what I'm missing? It looks like I'm missing some important header files (stdio, string, stdlib). How do I get those? And where do I put them?

Also, if I can get the above working and continue on with the subversion installation, what are the steps needed to replace /dev/random with /dev/urandom in apr?

zfil? rocket15? Bueller? Anyone?

Much thanks!
Message 11 of 42
ianadavies
Aspirant

Re: Subversion : some progress

Yeah, I had this problem you have to install dev-libc or somthing , if you go to the how to's on the readynas home page and follow the instructions for setting up a development environment first you will be ok


Also I did not change the dev/urandom and it works fine for me it's not slow at all, I could not get it to work with https though once built i use ssh+svn instead. Let me know if it works with https for you
Message 12 of 42
gromeen
Aspirant

Re: Subversion : some progress

Okay, I've gotten farther, but still error out. These are the steps I've taken:

1. started with a fresh factory default ReadyNAS 1100 (RAIDiator 4.1.3-T136 [1.00a146] in X-RAID)
2. downloaded EnableRootSSH and APT from here
3. installed those .bin files via FrontView, System>Update>Local
4. logged into my ReadyNAS via SSH (putty)
5. apt-get update (see http://www.readynas.com/?p=145 for details - thanks ianadavies)
apt-get install nano
apt-get install libc6-dev
apt-get install gcc
apt-get install gdb
apt-get install libtag1-dev (this one gave me errors)
Reading Package Lists... Done
Building Dependency Tree... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libtag1-dev: Depends: libtag1 (= 1.4-2bpo2.netgear1) but 1.4-2bpo2.netgear2 is to be installed
E: Broken packages

apt-get install uuid-dev
6. wget http://subversion.tigris.org/downloads/ ... 5.1.tar.gz
7. wget http://subversion.tigris.org/downloads/ ... 5.1.tar.gz
8. tar xvzf subversion-1.5.1.tar.gz
9. tar xvzf subversion-deps-1.5.1.tar.gz
10. rm *.gz (why keep the tarballs?)
11. cd subversion-1.5.1/zlib/
12. ./configure && make test && make install (this is where I got stuck last time but ianadavies pointed me in the right direction)
13. cd ..
14. ./configure --build=sparc-linux --enable-all-static --with-ssl --without-serf --with-zlib=/usr/local

If I understand correctly, at this point my ReadyNAS started chugging along creating the proper configuration for subversion on my machine.

By the way, one of the status lines I got was
checking for entropy source... /dev/urandom
Does this mean that I don't need to worry about the /dev/random vs /dev/urandom issue mentioned in previous posts?

Eventually, however, the configuration erred out with the following (I've cut and pasted the last dozen lines of the 1000+ output lines):
configure: IPv6 support is enabled
checking for working AI_ADDRCONFIG... yes
checking for socklen_t... yes
checking for struct tm.tm_gmtoff... yes
checking for struct tm.__tm_gmtoff... no
configure: zlib not enabled
checking whether to enable ACL support in neon... yes
checking for pkg-config... no
checking for library containing RSA_new... not found
configure: error: could not find library containing RSA_new
configure failed for neon

There were a few other questionable configuration outputs but I'm not sure if they were actually errors or just standard occurrences. I can provide the entire output line dump if necessary.

Does anyone have any insights that might help with this?
Message 13 of 42
ianadavies
Aspirant

Re: Subversion : some progress

try

apt-get install libssl-dev ' I think I came accross this issue too.

See if that fixes it, and yes don't worry about the dev/urandom issue unless it becomes an issue once you start using it.


Cheers


Ian
Message 14 of 42
wishyou
Aspirant

Re: Subversion : some progress

I get a problem build apr which is a part of subversion 1.5.4
Anyone have a solution to this problem?


readynas:~/svn-install/subversion-1.5.4$ make
------ making all in apr
make[1]: Entering directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make[2]: Entering directory `/home/XXX/svn-install/subversion-1.5.4/apr'
/bin/sh /home/XXX/svn-install/subversion-1.5.4/apr/libtool --silent --mode=compile gcc -g -O2 -pthread -DHAVE_CONFIG_H -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -I./include -I/home/XXX/svn-install/subversion-1.5.4/apr/include/arch/unix -I./include/arch/unix -I/home/XXX/svn-install/subversion-1.5.4/apr/include -o passwd/apr_getpass.lo -c passwd/apr_getpass.c && touch passwd/apr_getpass.lo
passwd/apr_getpass.c:78: warning: static declaration for `getpass' follows non-static
passwd/apr_getpass.c: In function `getpass':
passwd/apr_getpass.c:81: error: `stderr' undeclared (first use in this function)
passwd/apr_getpass.c:81: error: (Each undeclared identifier is reported only once
passwd/apr_getpass.c:81: error: for each function it appears in.)
passwd/apr_getpass.c:82: error: `stdin' undeclared (first use in this function)
make[2]: *** [passwd/apr_getpass.lo] Error 1
make[2]: Leaving directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make: *** [external-all] Error 1
readynas:~/svn-install/subversion-1.5.4$


The zlib build worked fine so all libs should be in place...

Wish
Message 15 of 42
zfil
Aspirant

Re: Subversion : some progress

Hello

I don't understand why it is not working for you ...

I've done the usual stuff :

Download sources
wget http://subversion.tigris.org/downloads/subversion-1.5.4.tar.bz2
wget http://subversion.tigris.org/downloads/subversion-deps-1.5.4.tar.bz2


Build
First untar both previously downloaded tgz.
cd subversion-1.5.4/zlib
./configure && make test && make install
cd ..
./configure --build=sparc-linux --enable-all-static --with-ssl --without-serf --with-zlib=/usr/local --with-devrandom=/dev/urandom
make
make check

56 tests out of 56 must run successfully.
make install


And it is working very well :lol:

For the record I use svnserve (svn protocol), this build don't create the apache module but anyway for a NAS I think it is MUCH better to use a svnserve as it lightweight as faster.

Here the configuration steps I've done :

Create svn user and group
echo svn:!:100:102:svn:/c/home/svn:/bin/bash >> /etc/passwd
echo svn:x:102: >> /etc/group
mkdir -p /c/home/svn/repository
chown svn:svn /c/home/svn


Create repository
su - svn
svnadmin create /c/home/svn/repository
cd repository/conf/
vi passwd svnserve.conf

passwd and svnserve.conf must be configured for the authentication

Add lines to /etc/services like these (if they don't already exist):
svn           3690/tcp   # Subversion
svn 3690/udp # Subversion


Add this line to /etc/inetd.conf:
svn stream tcp nowait svn /usr/local/bin/svnserve svnserve -i -r /c/home/svn/repository


Restart inetd
killall -HUP inetd
Message 16 of 42
da9l
Aspirant

Re: Subversion : some progress

Hi!

Had the exact same problems. I finally made it make after:

Reinstalling APT addon (the latest seems to be dated 2008-10-31) in order to get the most recent and most correct sources.lists file.
Adding all libraries as described in the setting up development environment howto.

Running:
apt-get install python (the tests depends on python)
apt-get install coreutils (some stuff in configure is dependent on the uniq command)
apt-get install libtool (Might have something to do with our problem below)

Make sure that you do ./configure with all options again.

Regards, da9l.


wishyou wrote:
I get a problem build apr which is a part of subversion 1.5.4
Anyone have a solution to this problem?


readynas:~/svn-install/subversion-1.5.4$ make
------ making all in apr
make[1]: Entering directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make[2]: Entering directory `/home/XXX/svn-install/subversion-1.5.4/apr'
/bin/sh /home/XXX/svn-install/subversion-1.5.4/apr/libtool --silent --mode=compile gcc -g -O2 -pthread -DHAVE_CONFIG_H -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -I./include -I/home/XXX/svn-install/subversion-1.5.4/apr/include/arch/unix -I./include/arch/unix -I/home/XXX/svn-install/subversion-1.5.4/apr/include -o passwd/apr_getpass.lo -c passwd/apr_getpass.c && touch passwd/apr_getpass.lo
passwd/apr_getpass.c:78: warning: static declaration for `getpass' follows non-static
passwd/apr_getpass.c: In function `getpass':
passwd/apr_getpass.c:81: error: `stderr' undeclared (first use in this function)
passwd/apr_getpass.c:81: error: (Each undeclared identifier is reported only once
passwd/apr_getpass.c:81: error: for each function it appears in.)
passwd/apr_getpass.c:82: error: `stdin' undeclared (first use in this function)
make[2]: *** [passwd/apr_getpass.lo] Error 1
make[2]: Leaving directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/XXX/svn-install/subversion-1.5.4/apr'
make: *** [external-all] Error 1
readynas:~/svn-install/subversion-1.5.4$


The zlib build worked fine so all libs should be in place...

Wish
Message 17 of 42
stevepiercy
Aspirant

Re: Subversion : some progress

zfil wrote:
For the record I use svnserve (svn protocol), this build don't create the apache module but anyway for a NAS I think it is MUCH better to use a svnserve as it lightweight as faster.

Here the configuration steps I've done :

Create svn user and group
echo svn:!:100:102:svn:/c/home/svn:/bin/bash >> /etc/passwd


This step fails for me:
# echo svn:!:100:102:svn:/c/home/svn:/bin/bash >> /etc/passwd
-bash: :100: bad word specifier


Does that mean the numeric userid is not valid?

What do I need to do instead to set up a user named 'svn'?

Thank you!
Message 18 of 42
zfil
Aspirant

Re: Subversion : some progress

No it is
da9l wrote:
Does that mean the numeric userid is not valid?

What do I need to do instead to set up a user named 'svn'?


It is because ! in shell is used to lookup history command.

So with proper escaping that's give :

echo svn:\!:100:102:svn:/c/home/svn:/bin/bash >> /etc/passwd


Sorry for that
Message 19 of 42
stevepiercy
Aspirant

Re: Subversion : some progress

Thank you, zfil. That got me a few steps closer.

But now this command fails:
$ svnadmin create /c/home/svn/repository
svnadmin: Repository creation failed
svnadmin: Creating lock dir
svnadmin: Can't create directory '/c/home/svn/repository/locks': Permission denied


I'd greatly appreciate any further assistance. Thank you!
Message 20 of 42
zfil
Aspirant

Re: Subversion : some progress

It's a permission problem.

As root try a :
chown -R svn:svn /c/home/svn

Then make sure to su - svn first :
# su - svn
svn ~$ id
uid=100(svn) gid=102(svn) groups=102(svn)
svn ~$ pwd
/c/home/svn
svn ~$ ls -la
total 64
drwxr-xr-x 3 svn svn 16384 2008-11-09 18:23 .
drwxr-xr-x 5 admin admin 16384 2008-11-09 18:07 ..
-rw------- 1 svn svn 185 2008-12-07 15:27 .bash_history
drwxr-xr-x 6 svn svn 16384 2008-11-09 18:19 repository
svn ~$ svnadmin create /c/home/svn/repository

Then the svn create should work fine 🙂
Message 21 of 42
stevepiercy
Aspirant

Re: Subversion : some progress

Thank you! That was it.

So this step above:
mkdir -p /c/home/svn/repository
chown svn:svn /c/home/svn


should be:
mkdir -p /c/home/svn/repository
chown -R svn:svn /c/home/svn


Now the creation of the repository is fine.

But now I have problems with authenticating. I edited the two files as suggested, but perhaps I did not enter the proper information. Here they are:

For svnserve.conf, I simply uncommented all the options. I had no reasoning behind do so, so if you can advise something more intelligent, I'll go with it. Snipped (...) for brevity.
svn@nas-01-B5-6B:~/repository/conf$ cat svnserve.conf 
...
[general]
...
anon-access = read
auth-access = write
...
password-db = passwd
...
authz-db = authz
...
realm = My First Repository

[sasl]
...
use-sasl = true
...
min-encryption = 0
max-encryption = 256


Actual user and pass obscured:
svn@nas-01-B5-6B:~/repository/conf$ cat passwd
### This file is an example password file for svnserve.
### Its format is similar to that of svnserve.conf. As shown in the
### example below it contains one section labelled [users].
### The name and password for each user follow, one account per line.

[users]
# harry = harryssecret
# sally = sallyssecret
myuser = mypassword
svn@nas-01-B5-6B:~/repository/conf$


Thank you for any further help for this newbie.
Message 22 of 42
zfil
Aspirant

Re: Subversion : some progress

Okay you uncommented too much :wink:

Lets just do a basic setup, you can improve it later by reading the svn manual.

svnserve.conf
[general]
anon-access = read
auth-access = write
password-db = passwd
realm = Repository
[sasl]


Your passwd looks fine.
Message 23 of 42
stevepiercy
Aspirant

Re: Subversion : some progress

zfil wrote:
Okay you uncommented too much :wink:

Lets just do a basic setup, you can improve it later by reading the svn manual.

svnserve.conf
[general]
anon-access = read
auth-access = write
password-db = passwd
realm = Repository
[sasl]


Your passwd looks fine.


That worked. Thank you!

This was my first time setting up a SVN repository. If I have some spare time, I'll compile all the details and publish an article "SVN Setup (on ReadyNAS NV+) for Newbies".
Message 24 of 42
wishyou
Aspirant

Re: Subversion : some progress

da9l wrote:

Reinstalling APT addon (the latest seems to be dated 2008-10-31) in order to get the most recent and most correct sources.lists file.
Adding all libraries as described in the setting up development environment howto.

Running:
apt-get install python (the tests depends on python)
apt-get install coreutils (some stuff in configure is dependent on the uniq command)
apt-get install libtool (Might have something to do with our problem below)

Make sure that you do ./configure with all options again.


My god, you are good! 🙂
I've been banging my head against the wall for weeks over this...

Thank you!

Wish
Message 25 of 42
Top Contributors
Discussion stats
Announcements