'So' is certainly annoying, since it suggests the speaker hasn't been listening at all and is merely interrupting to continue from where he left off.
But that's not nearly as annoying as uptalk? Where any sentence uttered is instantly turned into... a question? Has the speaker suddenly become...