R8000P "Couldn't connect to this network" when security enabled

Be sure the ISP modem is configured for bridge mode. However not sure if this would correct the issue on the NG router.

ISP modem is configured as Modem only.

This would be a double NAT condition which isn't recommended. https://kb.netgear.com/30186/What-is-Double-NAT
https://kb.netgear.com/30187/How-to-fix-issues-with-Double-NAT
Couple of options,

1. Configure the modem for transparent bridge or modem only mode. Then use the NG router in router mode. You'll need to contact the ISP for help and information in regards to the modem being bridged correctly.

Modem is in modem only mode.  R8000P is in router mode.
2. If you can't bridge the modem, disable ALL wifi radios on the modem, configure the modems DMZ/ExposedHost or IP Pass-Through for the IP address the NG router gets from the modem. https://kb.netgear.com/25891/DMZ-on-NETGEAR-routers?_ga=2.190834155.1964120923.1641394007-133989108....
https://kb.netgear.com/24086/How-do-I-set-up-a-default-DMZ-server-on-my-Nighthawk-router

All modem wifi radios are already disabled.  Not seeing how the pass-through IP address would change anything of the problem with connecting with R8000P wireless.
3. Or disable all wifi radios on the modem and connect the NG router to the modem, LAN to LAN configure AP mode on the NG router.
https://kb.netgear.com/24104/How-do-I-change-my-Nighthawk-router-to-AP-mode-after-I-ve-already-run-s...

Try option 3 with the ISP modem to see if devices can connec thru the 8000 in AP mode. If still same problem, then send the R8000 back. 

Option 3 -- cannot connect through the R8000P in Access Point mode.

Yeah, I'm afraid I have to send it back.   

I helped a friend configure his two R8000Ps in a mesh configuration.   I was quite impressed with how it worked.   Now I am saddened to read all these stories of the wireless dropping or not connecting.  Thanks so much, Furry.