Actually in the early series of OF&H there was a Palace scarf on a coat rack in their hall. It could have been Grandad's. Their mum was a Charlton fan as explained at Rodney's wedding.
I had never noticed in all the times I had previously seen this photo that Lord Lucan seems to have been in...