Charter Intermittent Slowdown

I have an R6400 router.

For the past week I've experienced intense slowdowns in internet speeds.  Rebooting the modem seems to always fix it.

Repairman came out and replaced everything from the box on the street to the ground box and even replaced it.  Also replaced the modem and ethernet cable between it and the router.  Router is about 4 months old.  He said filters on both sides of the line from the box to the modem had filters not used or needed anymore so removed them, 

Today I'm still experiencing the slowdowns until a modem reboot.

Charter's insisting it's my router since everytime I reboot the modem it fixes it, which sounds very farfetched.  

The only other thing I notice is when it's fast (130+ mbps) the server says it's coming from Olivette MO (I'm near St. Louis).  When it's slow (5-14 mpbs) it says Sugar Hill GA.  

Any ideas??