Probably Agnes', though it's not THAT bad. My main problem is between Diana, Annalise, and Pinky I have too many purple houses. I like all three of them very much though so I can deal.
Cobb is another with an ugly sheet-metal shack. He moved out of my town a few weeks ago though, thankfully, that thing was an eyesore and his catchphrase drove me crazy "ladybro".
For me, it's Nana. I don't have a picture of it, but it looks like a little pink shack, not a proper house. Of course, I could be biased against it, considering that she parked her house right on my path, in front of The Roost. :I