Other way around. ASCII classified control characters into blocks and newline ended up a bit-flip away from J, so the ADM-3A printed ‘­↓’ on the ‘­J’ key.

Yes, and ADM-3A also had "↑" on the "K" key, because \013 is vertical tabulation (VT), and "→" on the "L" because \014 is form feed (FF). It all makes perfect sense.

If terminfo database is to be believed, quite a lot of other terminals reused this control sequences too, for some reason.