yeah, in another town they won't recognize you.
but in another person's dream town that has the same villager as you, they will sorta recognize you.
If you invite a villager back to your village with an amiibo card, they will also mention this seeming oddly familiar. But it doesn't go much farther than that.