So I'm trying to get one of my villagers back, I believe I've moved out 8 villagers since. Anyway lately I have been seeing the villager in Main Street, I read that there are four villagers in Main Street, so what does this mean? Even when he leaves Main Street, will I still have to move out another four villagers?