Linear RNNs overcome both issues. All the RNNs I mentioned are linear RNNs.

I'll give them all a look. Thanks!