Yes but it's not always only just the bridge wings that crewmembers give their feedback from, ofcourse. And a skipper can issue orders no matter where he is, aswell.
And if the captain is in the deckchair at the pool, and uses his handheld to tell the helmsman to execute a certain manouvre, would the helmsman say 'no sir, I won't do that, cause you're sitting at the pool'?
Plus..not everyone wants to use the secondary/bridge wing controls with their mouse. Some people want to use the ship control unit, or have built their own controllers, or use joypads/sticks and the likes.
If you'd only be able to use the actual controls that are modeled there, it'd be restricted again. And you can already use the mouse in walkthrough anyway.
And many people have in the past requested this feature.. so the few people that think it's not realistic can just put the controller down, and not use it when they are walking about. Problem solved.
If that's a major issue for some people, and if it itches so badly that they simply can't ignore it, then maybe the problem isn't in the game itself.