Did the same in the end - 50 mile drive instead of 90 including half a lap of the M25. Left Worthing around 3:30, parked up a short walk from Vauxhall Station and I was at Tottenham Hale around 6.
Coming back we were close to the front in the ground, so had to wait a while to get out of the...
Sizeable no-parking zone (not surprisingly) in play for Spurs home games - https://www.haringey.gov.uk/parking-roads-and-travel/parking/controlled-parking-zones-cpzs/tottenham-event-day-cpz/tottenham-event-day-cpz-map
I'm far too late to get a drive way now. From a quick search North Middlesex...