You cannot use cards in the board game but you DON'T need figures. The menu allows each player to choose a (human) villager character if you don't have an amiibo.
And you cannot use figures, in the mini games, just cards -- but you get three. So the Festival bundle is mostly self-contained -- you need 6 cards to play the camping game (like Mastermind), and I'm not sure if anything else requires more than three. You'll want at least 1 packs' worth of cards I think.
Having other cards definitely makes a difference, there is rock-paper-scissors in one game, dice rolling in another (I think) and the type of animal you have affects the Island Escape game. Plus every time you use a new card that character starts walking around the big plaza.