iMap email issues

Hi there. I have the most frustrating issue. Around March, my Email began to play up. On and off during the day, my apple mail Client would stop sending and receiving email. If I plugged into the US Internet router directly, there is no issue. Now, the confusing part is I can get Gmail and Apple mail just fine. But, none of the email addresses from my own IMAP servers and domains. I get a message that mail cannot connect to server and a possible SSL Issue. Then at some point during the day, it might switch on for a while, then off. It’s random and really frustrating. Any thoughts?