How to set up 2 routers with one modem

Ok so I know nothing.

I've got a modem/router supplied by my ISP.

This is in an outhouse building (why the designed it like that i don't know

From this place in one direction I have a wire going to the main home and a router there.

In the opposite direction we have now built an office in the garden for me to work from and I have a router there.

My question is how should I set this up correctly.

Do I need to stop the router combined modem from being a router?

Can I continue to have both routers connected direct to the modem?

Help because what is happening at the moment isn't working. We are getting alot of Internet dropping.

One router is a rbk30 the other rbk50

Thanks in advance.