If a villager move out from your town, they will go to your void. You can clear them by visiting a town that have a maximum villagers. If you dont clear your void, and you visited a town which only has 8 villagers, the villager will be drop to their town.