Personally I wouldn't overdo it on the anti Palace jokes as from what you've said only half the room is likely to understand and appreciate the rivalry. You risk the other the other half politely laughing but in reality not having a clue.
One of the classic pit falls of a best man's speech is...