A disadvantage yes, but not a morally unfair one. That's like saying Brighton are at a disadvantage when playing against Man City, because Man City's players are better. Yes they are. Should they be penalised for having better players? Obviously not.
Otherwise we need to also ban all athletes...