Hmmm, he could have been saying I thought you'd be begging me to stay. That could go for either option though. If you didn't move the stick at all you would have automatically selected the stay/don't go option.
I suffer the same paranoia when any of mine ask about leaving, just try and talk to him tomorrow or later today/tonight and he will bring up how many days he has left before he leaves if he is indeed intending to go.
Try to relax! I'm exactly the same. I can be reasonable when it comes to other people's paranoia though haha