Ankha is the big one for me. Like, I get the appeal of the Egyptian theme, but she's always had sort of a 'meh' design to me. Doesn't help that her mouth never seems to change its shape once when she talks, making her expressions look weird and unnatural. Ironically, I absolutely adore Lucky and completely understand the hype behind him.
Raymond is a weird one because I love him myself and I get the appeal behind his design and personality. What I do not understand is the sheer amount of extreme obsession with him. I swear, some people treat that poor cat like he's god and some of the things I've heard people do regarding him are just.... why? He's just an adore little virtual cat; there's no need to go insane over him.
As for the unpopular ones, I mentioned her in another thread but my favorite villager, Bea. Why does no one talk about her? Is it because she has a simple design? Villagers like Marshal, Beau, and Fauna have pretty simple designs and people love them (this is not meant to hate on those three; I have Marshal and Beau and I adore them both). Was she just overshadowed by some of the more popular Welcome Amiibo villagers like Ketchup and Dobie? I don't know, but I do she needs much more love.
She's the most adorable little beagle (fun fact, I believe she's the only villager where they explicitly mention her breed in a game - if you scan her card and visit her RV in New Leaf they'll be some flavor text which reads something along the lines of "Bea's bagels are the best baked by a beagle - sure, you can tell Goldie is based on a Golden Retriever, Portia is clearly supposed to be a Dalmatian, Shep is some kind of sheepdog, etc, but in Bea's case, they actually say what breed she is and I find that really neat), her house looks a cozy little bakery/restaurant and I love it, her catchphrase of 'bingo' sounds incredibly cute when she says it in-game, and as I found out, she looks absolutely adorable in sweaters. She's a good doggo and needs all the love.