I have all gold, except for 9 of them. A LOT of people think I'm a hacker or time traveler, because they find it hard to believe I've played 1,300 hours since launch (if you do the math, yes, I play this a good 6 hours a day everyday). I also bought 3 more copies for the extra town's weeds (getting me the 5,000 weed one recently), and the StreetPasses (only around 500). It's been tough getting these, but well worth it to show off my insane hard work.
Oh, to answer, yes, I also look at their badges, their start date, to see if they too are a legit player or not.