I think it depends on the game!
If it's a story based game where the main character is actually a character, then I don't need to customize them at all. I don't want to pretend that I'm Ellie or Abby in the Last of Us 2 or Max in Life is Strange - I want them to be stand alone characters that I'm experiencing the story of. (That being said I'd never say no to getting to choose maybe an outfit or two for a character to wear!)
If it's a game where the character is meant to represent me (Animal Crossing, Pokemon, Monster Hunter Stories, Story of Seasons, etc) then it becomes more important to me. If the story is going to give me a relatively blank slate character and expect me to either self insert as the character or create my own character for the story, then I want to
actually be able to customize them.
It's frustrating in the Friends of Mineral Town remake in particular that I am supposed to be creating my little farmer to start my new life! And then... she's not really customizable. At all. I don't get to play around with her outfits (she's got a few to cycle through but that's it), or even really her hair from what I recall? Which absolutely takes me out of the game a little bit. For games like Pokemon half the fun (for me, anyways!!) is visiting the shops in new locations so I can dress up my character lol. When that's done poorly (Looking at you, Pokemon Shield!! Give me more dresses

) it can make me feel less motivated about playing the game.
