The router is new. Chromecast is connected to TV.
OK, how is the router connected? Did it replace an older router or are you now using the router instead of the modem your ISP provided?
Yes, the Chromecast is connected to the TV, where is it getting its signal from? What SSID?
Almost sounds like your modem is NOT bridged and it is putting out an SSID that Chromecast uses. Once you turn on the router that interfers with the modem (as it is not bridged) possibly.
Please list your wireless setting on the router and IF the modem is bridged or not?
Screen captures of the settings (with passphrase blacked out) would help here.