Hit, push and pulled them into pitfalls. After 2 years, I decided to clean up my town and deciding to keep talking to all of my villagers so that the thinking bubble will show up. After that, I hit, push and pulled them into pitfalls and they showed rage and sadness. But no one decided to move at all.
I time traveled six days forward, saved and quit, traveled another day forward, run around town to see if anyone pings me, but sadly no one did. The time was around 12 to 1 PM.