Pointers in and of themselves are not difficult to learn on their own, but when you're learning them alongside your first programming language, it's just adds to the difficulty I think.

I think a lot of noobs learning C struggle with pointers especially because there are no good error messages besides "segmentation fault" :D