A fair point and I agree. However for the life to be sophisticated, intelligent and evolved enough to get anywhere near us extrapolated the already king odds. Within the billions of stars there maybe be a few planets that can sustain life. of those few they're may or may not be able to find us...
Given the number of stars out there i would argue that actually it is highly likely that life exists elsewhere. If they have the technology to get to us is another question.