One workaround I found as using iTerm2 instead of Terminal.app and then using a weird keyboard combo, I think it was Cmd + Option + mouse drag.