UDP firewall rule goes to sleep after a few minutes and NTP traffic dies
A ProSafe FVS318Gv2 sits between the intranet and the DMZ, acting as a firewall. A firewall rule allows UDP 123 traffic (NTP/SNTP) for a workstation in the intranet to the time server in the DMZ.
When the NTP client starts synching with the time server on the other side of the firewall the polling intervals are short (from several seconds to about 2 minutes) and everything works normally. When the polling interval gets longer, say 3 to 4 minutes, the packets are no longer able to cross the firewall, as if the firewall rule had gone to sleep.
After about half a dozen requests without answer, the NTP client resets the polling interval to a few seconds and then the packets start to get through again. Hence the intervals get longer again, and again the firewall goes to sleep and starts dropping the packets. The dropped packets don't get logged though - there is no trace of these goings-on in the log file.
I discovered the details of this story by running a Wireshark on either side of the firewall. However, I don't think that the issue is specific to NTP. Similar things happen with ICMP echo, for example. The first ping after a period of silence times out, but subsequent pings get through.
There is low but steady traffic of TeamViewer/RAdmin/RDP type across the firewall at all times (monitoring a handful of machines), so the firewall is neither idle nor saturated. It is just the NTP and ICMP echo traffic that is infrequent. Normal frequency for NTP would be one request per hour, and ICMP echo averages about one packet per week. 😉
Sleeping sickness aside, the firewall works quite well and is a good fit for our needs. Is there anything that can be done to keep the thing from falling asleep on the job?
The firmware version is 4.3.1-11, in case it matters.
Model: FVS318Gv2|ProSafe gigabit 8 port VPN firewall
Not sure if this will help. Let us update the firmware of the FVS318Gv2 to the latest version which is v4.3.5-3 and you can download it here. Observe if the same problem occurs.
Re: UDP firewall rule goes to sleep after a few minutes and NTP traffic dies
Sorry for taking so long to reply - that tiny corona virus is surprisingly successful at throwing monkey wrenches into things. The bad news is that the standby/replacement firewall I ordered a week ago still hasn't arrived, which means I cannot risk bricking the firewall by trying to flash new firmware. The good news is that flashing new firmware no longer seems necessary.
I finally got my mitts on a pair of switches with port mirroring capability (by raiding my home network) and so I could sniff the complete traffic on either side of the firewall. The result is that the lost packets do make it out onto the wire on the near (intranet) side of the firewall, i.e. they aren't eaten by the network stack of the originating machine after having been seen by the Wireshark running there. The lost packets appear neither on the wire on the far (WAN) side of the firewall nor in the dropped packets log.
I also found out that both sides of the firewall are in the same broadcast domain because another programmer had installed a bypass to keep a legacy server functioning until a replacement can be rolled out. Obviously, this negates the security effect of the firewall but at least it keeps both kinds of clients running until the replacement is ready, i.e. the newer clients who are aware of the firewall and the old ones who need direct access.
After removing the bypass and rebooting the firewall the problem was completely gone, so it must have been caused by the fact that both ends of the firewall were connected to the same wire (broadcast domain). Hence the only thing I need to do is kick the programmer of the replacement server to make him work faster, in order to remove the need for the bypass.
The strange thing is that the sleepy firewall syndrome seems to affect only Windows 7 clients whereas packets from Windows 10 clients go through on the first try almost always, even after ages of inactivity. I.e. I haven't been able to provoke the glitch on Windows 10 clients at all. The few instances I observed on W10 clients were probably due to the boxen just having been plugged into different switches (meaning the switching mesh had to re-learn a different location for their MACs).
Inspecting the packets with Wireshark gave no indication of an appreciable difference between the Windows 7 packets and the Windows 10 ones - the packets were exactly the same on all levels, except for any contained sequence numbers and source IP/MAC naturally. Hence the difference probably lies in the things Wireshark cannot show, i.e. Ethernet framing and timings.
Anyway, the firewall works perfectly once the bypass has been removed, and so the bug hunt ends here. 😉
