Register the mousemove event handler on window, then you will still get the events when the mouse moves out of the window/frame while dragging and it won't be that buggy.

Was about to comment the same. It's a common mistake/gotcha.

Possibly dumb question, but does that still hold inside p5js?

p5 is just a wrapper that adds the setup() and draw() functions, so yes