Was this a racist act? Maybe, maybe not.
Is this bloke racist? Maybe, maybe not.
The problem is the context and connotation in which this occurs. You don’t have to go back very far to a time when this almost definitely would have been at least partially racist, in the sense that you are...