I normally drive to most games now from Crawley, I am in West Upper, leave at the final whistle and walk to Wild Park to pick up the car. I am normally home about an hour, maybe hour and 15 mins after Kickoff.
You can get fast trains from Brighton to Gatwick so it depends on what the schedule...