Crazy, I hadn't appreciated that.
So:
Fn-Left = scroll to top
Fn-Right = scroll to bottom
Fn-Up = scroll page up
Fn-Down = scroll page down
⌘-Up = cursor to top
⌘-Down = cursor to bottom
⌘-Left = cursor to start of line
⌘-Right = cursor to end of line
And also option-left/right moves the cursor by word, and option-up/down moves (I think I almost never use it) by paragraph.