Nope it won't count against you. On the contrary, when you have a perfect town, you even have more of a chance with getting the famous mushroom, so that's a good thing
Usually anything that was already in the ground or on the ground would be alright. Like with fossils/seashells, those are okay to have and don't count as trash, and flowers, mushroom, and fruits (not in baskets) of course will not have a negative effect on your town score. Mainly natural stuff basically are alright.
right now since mushrooms spawn next to regular trees in november, its a good idea to store all the mushrooms you want to keep. otherwise, new ones including the mush furniture mushroom wont spawn.
yea i had this happen this year i did not get any because i had like 10 famous mushrooms on the ground after november though i am going to put em back.