Grumble
Mayor of Moga Village
They can indeed ask for other fruits that isn't growing in your town. In my first town they did that and also mentioned 'This doesn't grow here in town, but I hope you can find one anyway'. About Durians and Bananas in particular I don't know. I think it's mostly the most common; apple, cherry, peach, orange, pears that they ask for.
I've also never had my villagers ask for any island fruit or persimmons. I think the game just wants you to visit other players' towns.