If you want him to move, then be aware that talking to him might change his mind.
This. Seriously. Once he pings you to move.. DO NOT talk to them. Villagers can change their mind if they bring up the conversation and Cranky villagers LOVE doing this on you.
I wasnt aware of that, and I had wart Jr. stuck in my early days for MONTHS.