Sometimes, when you see two villagers together, they're actually just greeting each other, most probably the reason why one walked away when you approached.
When villagers chat, you can actually see them nodding. And when you talk to either one of them, you should actually enter their conversation.