Creating local domains inside the network with the router

> [...] I would like to give them custom domains like
> `raspberrypi.local` or `raspberrypi.home` to make them better
> acessible. [...]

   You might create fewer headaches if you varied the front part of such
names, not the tail end.  So, say, "rpi-bedroom.local" and
"rpi-kitchen.local".

> [...] I'm using my hosts file on my windows pc at the moment but I
> want that all devices inside the network can communicate with the server
> over their domain names.

> [...] I'm using my hosts file on my windows pc at the moment [...]

   If you don't make (m)any changes, then copying a "hosts" file around
might be the easy way.

   Otherwise, you'd need your own DNS name server running on some
(always-available) system(s) or other.  That could be one (or both) of
the R-Pi systems.  A quick Web search (for: raspbian dns) led to what
looked like a plausible guide:

      https://www.ionos.com/digitalguide/server/configuration/how-to-make-your-raspberry-pi-into-a-dns-server/

   I'm sure that they'd be happy to sell you a real-world domain, but
(when I skimmed it quickly, it looked to me as if) that explanation
seems also to cover a strictly private domain.  In any case, BIND is one
of the better-known DNS server software kits.  Getting it configured may
require some thought, however.

> [...] I'm currently using the R8000P Nighthawk X6S as my router. Is it
> possible to achiev this with that router?

   So far as I know, Netgear consumer-grade routers do not offer a
user-configurable DNS server.