You are correct - racism is not confined to Caucasians - it is a product of oppression of one group by another. In its modern context it developed as a justification for the slave trade which was necessary for the development of capitalism. Without racism, capitalism will not have been capable...
Same thing happened in a game today between the Irish U21s and Kuwait - a Kuwaiti player made racist comments to a Irish player on the subs bench and the Irish team walk off the pitch (they were 3-0 up at the time).