Completely right... I have never had a problem talking with opposition fans wearing colours home or away. It's the scrotes young and old without colours, particularly wearing Stone Island you want to watch out for...
As others have pointed out, they are not likely to be season ticket holders but have joined the ticket scheme to get a fan number and thereby purchase an occasional match ticket. Easy peasy.