I have noticed that if a villager is a bit more neutral on a piece of clothing, they'll display it in their house and then not wear it. However, you get the clothes replaced with a piece of furniture, then they'll wear it a little more.
For example, I have Coco funny glasses. She displayed it in her house after wearing it after I gave it to her. I gave her a plant, which replaced the glasses and made her put them on more frequently.
Same with Phoebe. I have her a bulldog jacket. She displayed it. I gave her a plant and now she wears the jacket more. I guess because it takes it out of the house as a piece of furniture and puts it back into their wardrobe.