> I always thought that retransmitting the full chat every turn is a foundamental problem

So did many NLP experts who worked on LSTM, GRU, etc. But transformers empirically won out and here we are.

It will be interesting to see if future model architectures need to take this into account again.

If humans are any indication, one thing we're still missing is efficient movement of information from short term to long term memory. Fine tuning in principle does something akin to this, but not efficiently.