I would rather drive then I get the freedom to go/leave when I want which with kids is kinda needed
100% agree. But unfortunatley Falmer is not going to be particularly travel-friendly with regard to parents with younger children. It's gonna be a mass stampede after the game to the bus or train queues and survival of the fittest. Not pleasant for small kids.
If I were you I'd take the kids to Crawley for the football until they are a bit older.