Exactly. The people who voted for Brexit were those living in areas where immigration has not yet actually affected their lives - but they can travel to, or just watch on TV what has happened to the other parts of the UK where it has.
Hence in Germany the AFD are much stronger in Saxony and...