Opinions are subjective. At the same time, it's unfortunate that the way we perceive the world, which is often in terms of duality, means it's unavoidable to have contrasting, subjective opinions. If we think something is beautiful, then we are also capable of thinking that something is not beautiful, that is to say, ugly.
The more important question is, I believe, should anyone let someone else's opinion influence their mood and their enjoyment in the playing of a game? I would like to say, no, it should not be the case. But let's face it. We're weak. We're always looking for affirmation and approval from other individuals in our social/gaming/whatever circle. As such, we take other people's opinions way too seriously.
If someone says your villagers are ugly, stop and think, the opinion actually reflects more on the opinion holder's perspective and reality than on yours. And how does that actually affect your game? It doesn't. Your mood may be affected by the opinion, but your villagers are still the same villagers as they were a minute ago, they have not magically become ugly just because someone else proclaimed they are.
You can let someone else influence you, or you can be aware of the fact that you have a choice, and not let someone else influence you. The trick is in knowing what you like, what you enjoy, what you can do to keep enjoying what you do.
We all like some villagers better than others for a whole lot of different reasons. Because we're people. That's perfectly acceptable. A game is supposed to be a thing of fun, and if you enjoy your game for different reasons in different ways from your family/friends/forum posters, that's fine too.
Your game, your fun, your way. (As if I'm still making sense. Sorry for being long-winded.

)