i genuinely don't think so, i honestly cant see any hatred in "your holding hands". Its not nasty, its not derogatory in anyeway that I can see.
There is a danger, imo, that we over sanitise everything, where do we stop? Almost all humour has a but of the joke, all funny situations the same...
as a gay fan, this is something that is close to home. The example of we can see you holding hands is in no way homophobic, the retort of "too ugly to be gay" is a fun response and IS banter
Its a fine line between "banter" and hompohobia, and everyone will see it differently- personally not...