I've only just started going into some more competitive pokemon builds (I just hatched a 6 IV Togepi about a week ago), so I don't know too much about it. Here's my impression of it in a nutshell.
IVs seem to be like the max potential a pokemon can reach in a certain area. If you've spoken to the purple-haired guy in Kiloude City in XY, he judges your Pokemon's potential, and what he says determines how many IVs each pokemon has, which ones it has (HP, Speed, Att. Special Att., etc), and how well those IVs work for the Pokemon. Whether this has to do with how well the IVs work for the Pokemon's nature, or all Pokemon of its kind, I'm not entirely sure. I've gotten "outstanding" potential from a Pokemon with only two IVs, and "relatively superior" from one with 4. I guess the key is to hope that you have the IVs that you DO want to max out in EV training.
EVs are a bit like ability points in MMOs. You only get a certain amount per pokemon, so you want to invest them in the stats you need the most. It's easier in X and Y with super training and reset bags, where you can set all your Pokemon's statistics back to base level, and then only train the ones you want trained. If you look at pokemon stats, they'll often have some added in places they won't really use. Like a special attacker having some points in attack (which wouldn't really do it much good). EV training helps you invest your points properly, so that none of your EVs go "wasted" so to speak. Where you decide to put the effort really depends on what kind of pokemon you want to build, as well as what the pokemon is generally suited for. If it has good IVs in a certain area, then it's more likely that it will reach is maximum potential. My first Togekiss has 4 IVs, but I invested into as much HP as it would allow, and it's only off the maximum possible HP for a Togekiss by 1. Very annoying to deal with, from someone else's point of view, haha.