I wouldn't be surprised if part of the reason they stay away is because they're scared. We supposedly captured one (probably more) of them, and it's not like we treat them like one of us. We kill animals, we kill bugs and insects, we even kill our own kind. There's a lot of things I could think which would justify them staying away from us. If we can do that, what makes you think aliens would want to associate themselves with that kind of behaviour and put themselves at such risk? I feel bad for them.