Overflow from what? I'm slightly confused by what's happening here.
Taking a guess your cold water pressure is higher than the hot and with both on and the restriction of the hose water is being forced back up hot feed and causing overflow.
Anyhow outside taps very useful, just get one. Though...