> Is there progress happening in that trajectory?

There was a recent Hackernews post which had a novel approach about making agents interact with GUI/computer-use

https://news.ycombinator.com/item?id=47125014: The First Fully General Computer Action Model : https://si.inc/posts/fdm1/

Hope this helps