Yes and no. Casual chat with a passenger is unlikely to be any distraction, whereas I almost never take a work call when driving (even on the hands free) because it could turn out to be a lengthy and / or technical call, in which case it absolutely would affect my concentration.
Self obsessed morons, are so fixated with their phones, that a ban on owning a PHONE would probably be a bigger deterrent to them than a driving ban!
I'd like to give every traffic cop a hammer, with license to smash up the phone, on the spot, of every knob spotted using it at the wheel.