Stone. Definitely stone. There are items that take 30 stone for 1 item. Or 90 stones for 1 item and I don't go on tours a lot so I'm at the mercy of my 5 rocks a day.
I would have to say hardwood is probably the crafting material that I run out of the fastest. Stone would be in second place. The others seem easy enough to keep a good stock of.