If encryption was absolutely required, I might try it over s_client/s_server or two stunnels, one in client=yes mode.

I'm assuming that would have different limits than you outlined above, although I don't think they do multi core.