Can you tell me who is 10.227.1.120 and who is 10.227.1.11? Do you capture it in Freebsd machine?
We found: 1. The tcp packets are all wrong checksum from 10.227.1.120. 2. Slower response from 10.227.1.120
1.11 is the NAS 1.120 is the FreeBSD client
Capture was done on the FreeBSD machine. I've tested on two other FreeBSD machines with different NICs (one of them being VMWare) and had the exact same result.
There is a consistant 100ms delay in freebsd side for each round. I guess it might be a delay ack thing in your freebsd machine, but I am not sure, you can find out by yourself.
There is a consistant 100ms delay in freebsd side for each round. I guess it might be a delay ack thing in your freebsd machine, but I am not sure, you can find out by yourself.
And you don't use Jumbo Frame, right?
I was using Jumbo Frame for a while, but disabled it in this testing. After this is working I will be replicating over the Internet so Jumbo Frames aren't really relevant here. Any ideas why all of my FreeBSD installations would do this with your rsyncd, but not with other machines?
You just need to do the same rsync test like your test with ReadyNAS, I think. And you need to use same machine - 10.227.1.120 for testing and capture, so that we can see what is different when it work with other rsync servers.
I fired up a quick rsyncd on the linux box and set up the tcp dump on the FreeBSD box per your request. I then performed a sync of the same file across from the rsyncd on the linux box.
receiving file list ... 1 file to consider CT4MGM.SF2 4174814 100% 12.56MB/s 0:00:00 (1, 100.0% of 1)
Number of files: 1 Number of files transferred: 1 Total file size: 4174814 bytes Total transferred file size: 4174814 bytes Literal data: 4174814 bytes Matched data: 0 bytes File list size: 29 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 98 Total bytes received: 4175437
sent 98 bytes received 4175437 bytes 1670214.00 bytes/sec total size is 4174814 speedup is 1.00
What's your exact rsync command that you're using?
rsync -rltv --progress --stats
-rlt is basically -a with a few options removed as I do not wish to transfer ownership, permissions, etc with this rsync process. I have also tried using a simple -av and -avz.
FYI -- I'ev done "smbmount" from the FreeBSD and Linux boxes and gotten full speed on both. This is clearly just limited to rsyncd with FreeBSD. FreeBSD 6.1 has rsync 2.6.6 from the base ports tree. For testing I updated the ports tree to the latest and that gave me rsync 2.6.8-2 which provided the same results.
We got the 2nd capture between freebsd and linux server. Unfortunately the rsync procedure was different from 1st capture between freebsd and readynas.
The 1st capture rsync transfer data, but 2nd capture rsync transfer module list.
But I can see there was also 100ms delay in 1.120 side once or twice. So I think if freebsd and the linux server rsync transfer data, it will be slow as well as freebsd and readynas.
You can compare the captures by yourself to see if I am right.
And to compare, you have to make a same test for rsync to transfer data instead of module lists.
As this issue is common across all of my test FreeBSD boxes and I'm not the first to bring up the issue on your forum I am assuming this is a general incompatibility between the linux rsyncd running on your NAS and FreeBSD.
I need to figure out pretty quickly if you think this is a problem we'll be able to fix on the NAS or not? Since rsync is really my only option for use with the NAS and my production servers I'll need to look for another solution if not.
To clarify, I am completely willing to help you guys work out a solution here if you're willing to help me... but if its all on my to find a way to make FreeBSD work better with the NAS then I need to move on to get a working solution sooner. Thoughts?
Here we will see if we can reproduce your problem with our freebsd machine.
Please keep me informed. I would prefer to keep this in place3 as changing would be a hassle. I was able to reproduce the same behavior using VMWare, so you should be able to do this virtual if you need to.
Any results from testing? Are you seeing the same behavior?
Can you check the value of "delayed_ack" for your FreeBSD system? If you can change that option to equal 0, should be able to solve this problem for now. Here is for your reference: sysctl net.inet.tcp.delayed_ack=0
It is sure the delayed ACK settings in FreeBSD or BSD systems (like MACs) will cause slow performance with ReadyNAS. And it seems that can cause similar problem with other systems.
We will try our best to find a reasonable and generic solution in our system, but BSD users have to turn delayed ACK off before we reach that target. And turn that off also will give you best performance with ReadyNAS, and also might improve BSD network performance in general.
How will this affect normal use of the boxes? These machines serve up web/email/ftp for mlutiple sites and I want to be careful about settings that may impact other applications.
Initial tests on the local LAN using my test box are promising. I would like to know what impact this setting can have on other stuff before it is moved to production and I am curious if you can find something to work around it on the NAS side.
