Yeaahhh, a ton of people are accidentally boxing up their favorite villagers. A girl in the group I'm in lost like, 3 in half a week. I'm really lucky that I have multiple towns and a second 3DS I can use for the cost of asking. o: Saves me a lot of stress and heartache.
I figure that if I have to kick out 16 villagers to get Julian back, I can kick out a couple more permanent villagers right after him , then move them back into my town in the same order they left, just in different places. When I was originally planning on which villagers I was going to look for and keep, Zucker, Octavian, Olivia, and Static were never on my list. So to get them in, I need to kick out certain villagers, which will leave an odd gap in my villager neighborhood. If I cycle out a couple more permanent villagers, I can end up with a nice, smooth line of houses.