I think the Witch Doctor is a red herring here. Smithers is clearly a man of good character, and the Colonel is telling lies.
My suggestion, albeit libellous, is that the Colonel had a terrible and crippling virus caused through an unspeakable sexual engagement with an Impala.