not to be a sjw, but this is just predjudice, not racism. racism is tied to oppression, while us white people are still the oppressors and not the oppressed.
i think generally its justified and valid for poc to be wary and cautious around all white people, especially white they dont know. im not of the opinion that all non-poc are racist, just that we are part of a system of oppression and are brought up in generally racist societies. im a texan with very accepting and extremely un-racist parents, but absolutely disgusting racist grandparents on both sides. the difference between my parents and grandparents is their choice not to be part of racism that was absolutely ingrained in them by the gross southern environment they grew up in.
also, as someone whos lived in majority poc places his whole life and more than once was the only white person in my entire school and neighborhood, i find it very hard to believe they threatened you exclusively for being white. im not saying youre racist or you antagonized them, just i personally find that extremely unbelievable, especially with your admitted racist household. i also find it unfair for you to take a few bad examples of people disliking you for being of a race that has oppressed them for millennia and use it as a way to say all poc immediately find you racist.