Reply

Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

ssawyer
Follower

Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

I'm documenting this here, because I didn't see anything about it in my forum searches, and I believe that it's likely to be a common problem with an easy fix.

I had just installed a new Netgear SRX5308 VPN/Firewall in our network (5 VLANs, about 70 users and around 200 devices total). Everything seemed to be working fine, but — randomly and intermittently — new connections would be slow or time out. So users would experience web pages loading slowly, or failing to load at all, but then working again on a second try, etc.

Netgear support couldn't help much with it, so I did more of my own diagnosis, and eventually found that it was related to DNS lookups slowing down or failing intermittently when going through the firewall; if I was outside the firewall, everything was fine.

I then found the solution myself: to uncheck the "Block UDP flood" on the "Attack Checks" configuration of the firewall settings. Since I deactivated it, everything has been working fine.

I then went back and looked in Netgear's documentation — apparently, the "Block UDP flood" option, which is enabled by default, triggers when it has 20 or more simultaneous UDP connections from a single LAN-side client. And of course, DNS works over UDP port 53, so we were seeing intermittency whenever we got to >20 DNS requests at the same time from a client. (And, in fact, the current manual acknowledges this in a note on p. 136 — which is in the firewall rules section and unfortunately not referenced in the Attack Checks section).

The reason I think this is likely a common problem: 20 simultaneous connections is WAY TOO LOW for modern browsers and network usage; an single average webpage can load material from its own server, 2-4 social networks, various sources for Javascript libraries, fonts, CSS, etc., and CDN servers for images — all of which require DNS lookups. You could easily get to 20 on a single page, even without accounting for stuff the computer is doing in the background that might involve DNS lookups or other uses of UDP.

There are, I think, a few options for how Netgear should fix this:
(a) set a separate policy for dealing with a flood of DNS UDP traffic on port 53 with a better (or configurable!) connections limit
(b) make the number of simultaneous UDP connections configurable overall
(c) MINIMALLY, make sure that when "Block UDP flood" is triggered, there's a clear log message about set up under default logging conditions — the place this got really painful was that I couldn't determine from the firewall's logs why the firewall had dropped my traffic; had I seen something about "Block UDP flood" in the logs, I would have been able to fix it myself without having to call.

But anyway: at least on the SRX5308, there's a simple setting, on by default, that causes DNS requests to be dropped under conditions that are fairly normal in modern networks. If you're having connection timeouts — especially if you can track them to failed or slow DNS lookups — try turning of "Block UDP flood" in the "Attack Checks" section of firewall policy.
Message 1 of 22
gkarasik
Guide

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Thanks very much for taking the time to post this.

GaryK
Message 2 of 22
MPN
Aspirant
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

This solved our problem as well. Our FVS318N kept having DNS problems, but when I switched to our FVS336G, no problems. Tried different firmware versions and a different FVS318N, nothing helped.
Turns out, the FVS318N has "Block UPD flood" on be default and the FVS336G has it off by default. No problems now.
Message 3 of 22
silvex
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

The wireless VPN FW SRXN3205 has the same issue and I did the same "fix".
Message 4 of 22
andyanderson
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Thank you Ssawyer for this information. I applied it to my three SRXN3205. I was about to remove all three of them from service because of this. I have contacted Netgear Support (if we really want to call them that) multiple times about this issue. Thank you again!
Message 5 of 22
giusiof
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Hi,
unfortunately this solved only some problems.
I updated to Firmware 4.2.1-2 and I'm experiencing intermittent connections yet.
I disabled any type of log on security and udp flood on attack checks , so this is the result
Thu Jan 3 18:14:31 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
<4
Thu Jan 3 18:11:26 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
<4
Thu Jan 3 18:11:26 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
Thu Jan 3 18:11:26 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
Thu Jan 3 18:11:26 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
Thu Jan 3 18:11:21 2013(TZi-) [SRX5308][Kernel][KERNEL] cvm_ipfwd_cache_flow: Failed to allocate flow info buffer
<4

Anyone has this problem?
Thanks
Message 6 of 22
volkuhl
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

same problem for me...
Message 7 of 22
SDM
Novice
Novice

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

same problem for me...
El mismo promema para nosotros...
SRX5308
Message 8 of 22
jzelos
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

NB: I've got a ticket open with netgear referencing the below thread for the "cvm_ipfwd_cache_flow" issue.

http://forum1.netgear.com/showthread.php?t=81403
Message 9 of 22
JellyBeanGreen
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Your the best, I spent hours upon hours trying to find an answer... I read your post, click, done!

Thanks Smiley Happy
Message 10 of 22
GDHuber
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Thank you so for this post, problem resolved.
I hope Netgear reads this and up-dates the firmware.
Message 11 of 22
NTGRCBU
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Thanks for bringing up this issue. We have verified this issue internally and will address this in upcoming firmware. In the mean time I'm glad to see that there is a relatively simple workaround, but regardless this will be addressed.
Message 12 of 22
BeerDrinker
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Excellent advice here, unchecking "Block UDP flood" on a FVS318N completely and totally cleared up numerous issues as described ... Intermittent Connections, Slowdowns, Timeouts.

I did some testing to set a rule to allow all outbound "DNS:UDP" from all lan users, that had no apparent effect with Block UDP flood enabled. That makes you wonder what UDP packets are triggering the flood detection threshold. Was seeing this with one VoIP ATA and one Win 7 client on the lan.
Message 13 of 22
BeerDrinker
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

I see the new firmware has implemented a suggestion from the first post, (b) make the number of simultaneous UDP connections configurable overall. At leats that's what I think the box next to the check box, with a default value of 25 seems to be.
Message 14 of 22
polecats
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Hi,

Have you tried the new 4.3.0-19 FW for SRX5308?

http://kb.netgear.com/app/answers/detail/a_id/23142

Results?
Message 15 of 22
SamirD
Prodigy

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

I'm really glad I saw this post. I was having very intermittent issues with the same thing, but also would have my fvs114 lock up completely every few days. After reviewing not only this setting, but my log settings in general, I think I'll have better luck now. Smiley Happy
Message 16 of 22
SamirD
Prodigy

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Well, so much for that--it locked up again today. Smiley Sad
Message 17 of 22
bobbroder
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Interestingly enough, I found this post AFTER I remedied the issue on my own - hours of work.

Now that they've made the number configurable, would someone please post the number of requests that would be adequate on a modern network. The default - 25 is not.

A request to Netgear - your search engine did not turn up this post when I searched for slow performance, Internet issues etc. Perhaps you could fix this?
Message 18 of 22
longmang
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

The other thing that seemed to help me was setting the DNS servers to be the same across all of the WAN ports, I use the Google DNS servers 8.8.8.8 and 8.8.4.4, but others like the OpenDNS servers.

This particularly helped me as I have a DNS server on the LAN side of the router using the DNS proxy facility of the SRX5308.
Message 19 of 22
jweber
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Just fixed a brand new FVS336G-300NAS with unchecking the UDP flood test.

Thank you!

Smiley Very Happy
Message 20 of 22
iToon
Novice

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

This sovled our slowed DNS perfromance on our FVS336G with Firmware 4.3.2.7 thanks for the information.
Message 21 of 22
sdahlstrom
Aspirant

Re: Easy Fix for Intermittent Connections, Slowdowns, Timeouts - DNS related, SRX5308

Remembering this thread. thank you so much, I've been looking for this solution...

 

Message 22 of 22
Top Contributors
Discussion stats
Announcements