Amazon Fire Stick 4K

I just purchased an Amazon "FireTVstick 4K" and I'm having trouble setting it up on my tv. I have unplugged my modem, changed batteries in all remotes and I went to my online account under access control. I changed my selection to allow all new devices to connect.

I pressed the Firestick button to pair with my tv and it's still not working. I do not have a smart tv, but it does have HDMI. I did the same process to both of my tvs and there is still no connection with my tv and Firestick. 

I am at my wit's end. Please help!! Thank you very much!