What you are effectively saying is that they are such a minority that the majority should be able to force them to be part of the Union whether they like it or not.
A union is supposed to be a voluntary arrangement, you can't have a union by force. Freedom means being part of something if you...