Lol, try this if you haven't already, this helped me so much it's funny
TT 2 days forward ( make sure you have keep town beautiful ordinance if you don't want your flowers to wilt or weeds to erupt)
Talk To Your Snooty/Smug Villagers To See If They Mention Anyone Moving. If it's someone you need, talk to them and make sure they stay. If not, tell them to LEAVE. Repeat As many times as necessary, and make sure you talk to your snooty/smug villagers a lot to make sure no one is moving or of someone is moving.