Lagging out my internet

If wifi is unaffected, then it could mean two things :
=> It's an hardware problem on your wired section of the network
=> It's a problem with the device you notice lag on (xbox).
Indeed by design wifi is not isolated form the normal network (unless you decide so, and you can't usually if you have an all in one router). If they have the same IP addresses range, then they should be affected if something is messing your network that is non-physical. Just to make sure, you xbox/NAS has an address in 192.168.0.x or 192.168.1.x, or 172.16.y.x or 10.z.y.x ? and things in wifi have the same range too ?

If you get the xbox on wifi, is the problem still there ?
If you have a spare switch / router (don't buy one just for that, I'm sure you will find a friend that will lend you something for an hour or two), just hook it up behind your router (may have to get your router in bridge mode if you hook another router) and see if it's happening or not.
If it's still happening with those two tests, you can rule out hardware problem (not entirely but at 95%, to rule out the 5% remaining you would have to replace totally your router with a model that doesn't share anything with your actual one (another manufacturer, other internal components...).

Not sure how to rule out the possibility of a problem coming from xbox (or xbox live), unlikely since with similar network I have no problem but still possible (not in the same region of the world...).

What router have you got ? In general, when network problems occur (not including temporary failures from ISP) it's the network peripherals that are guilty... or an incompetent admin...
And I also forgot to say that any computer like system dislike to have to things with the same name on the network, I don't think you named your NAS and your xbox identically, but that's a possibility.

Is your NAS providing access to someone outside your network ? (ready cloud, website hosting ?...)