Nighthawk RAX80 high latency

This started around 4 days ago, I start to experience random high latency and packet loss issue over 5G network. 2.4G is fine. It's not constant, but covers most of the time if I'm using the same SSID for 5G and 2.4G, no matter if Smart Connect is enabled.

I was able to work with a Charter technician to rule out everything else and narrow it down to the router (new connectors, new modem, new indoor cable, it only disappeared after we used a new router). I have tried the following:

• Firmware was the latest when I checked but don't know when was it auto updated. I tried downgrading to last release, not working.
• Checked for channel interference. No one was using my channel and I tried another unoccupied channel that's 2 channels apart from any other occupied ones, not helping.
• QoS was off, turning it on is not helping.
• Full reset. It works when first started and 5G speed is limited to 287 mbps. And it may be working after first or second reboot to set it into normal speed (during some tests, I didn't change any other settings than finishing initialization). But within 3 reboots without any additional setting will trigger the problem. So maybe it may go bad with 287 mbps as well, just not right after reset.
• Separating 5G and 2.4G SSIDs. For a time I thought it worked until it stopped. It took quite a while for the problem to start but sure it did. It seems that the problem covers less while SSIDs are separated, but it could be false feeling.

Although I ruled out firmware, but it still feels like some kind of software issue. But to be fair, I only had the new router for a few hours. While it didn't show during that time, it could be possible that the source of interference was not active at that time. Any ideas?

Attached photo is what it looks like to ping Google DNS while its acting up. Normal should be around 25ms in my area.