Rsync slow as heck for everyone else???

cgreentx wrote:
I'm running rsync with a unix box on the same gigabit switch and it is ridiculously slow. So severely slow that it copied 200ish MB in about 3 hours. That's pretty harsh. Any ideas? Chris Green

Any network errors in the network tab? If you have a Windows PC, please follow the troubleshooting tips in the sticky on this board.

No network errors at all. I get 30MB/sec read and 20MB/sec writes on my windows box. I am getting 40kB/sec according to rsync! OUCH!!

Chris

FYI - I upped it to 512MB while troubleshooting other perf issues so it has plenty of RAM.

Chris

This place dries up at 5PM on friday... sheesh!! Anyone?

Chris Green

cgreentx wrote:
FYI - I upped it to 512MB while troubleshooting other perf issues so it has plenty of RAM. Chris

What are the avg sizes of your files and the number of files? Perhaps you can do a Download All Logs and forward that to logs@infrant.com, with a link back to this thread. Attn yoh-dah.

I anticipated your request and forwarded the logs last night with the link. There are 51,000+ files for a total of 244GB. This is an MP3 library that I am trying to sync for backups. Files range from 2MB to 6MB each. I am getting ~40 KB/sec on transfers of each file.

Chris

You do realize that both sides have to check every directory for changes.
Then any file that is different has to be block checked, for which part has changed. MP3
files probably don't ever change, so this part should not be an issue.

I suspect that 244GBs would take a while to do the directory scan. 51,000 files sounds
like a lot, if put in a single or few directories.

Did make a reasonable limit on each directory? (Like per Artist? Or per Album?)

I use "rsync" on Unix, (Solaris SPARC & x86, and Linux x86), for my backups to my ReadyNAS. Works okay, but my largest file system is 10GB. (I do it by file system, so
I can generate a nice log file.)

The directory scan take sa few minutes.. Then on each file it takes about 1-3 minutes to transfer it over gigabit ethernet. Its not the number of files that is the issue. The transfer rate hovers right at 40KB/sec which is ridiculously slow. I did this same rsync prior to getting the Infrant and I would max out the drive transfer rates.

Chris

Here are the "rsync" parameters I am using for my backups.
Note that I leave off the individual file printing, (which in your
case should make little difference).

rsync -x -aH --delete --stats SOURCE DESTINATION

If you don't have any hard links, leave off the "H". Makes
the directory scan faster.

Is anyone else from Infrant willing to help on this? I've tried calling for support several times but given up after 10 minutes holding each time. I need a solution here or I'm going to have to give up on the NAS completely. I appreciate any users trying to help here for sure, but I think we're at a point that Infrant needs to help find the software problem causeing this performance bottleneck.

Thanks!
Chris Green