2 PS4 in same house, same wifi NOT WORKING

My husband and I are NOT techies and all we want is to both we able to play against either other online on our respective PS4s in the same house - different rooms. We have xfinity, and just firgured our cheap little router they gave us was the problem. So we picked up this nighthawk router. We got the moderately priced option. Fearing the cheapest would be too weak, but we live in a small apartment so we didnt want the heavy duty one either.

We got home and hooked it up like it says in the booklet. It is up and running just fine. IT WONT LET US PLAY TOGETHER. It keeps kicking us saying we dont have internet, yet when we run the net work connection test, it looks to be connected at top speed. PLEASE help. This is the only reason we bought the router and it doesn't seem to be helping! Did we get in over our head or can this still be set up to work?