I guess people find it difficult because it isn't based on Romance languages (Spanish, French, Italian, etc.), so if you come from that background, word placement and sentence structure can seem backward or uncomfortable; I know I had some confusion when I was learning Spanish back in school.
Then there are spelling differences, based on whether you are dealing with British or American English, or even words meaning completely different things.