This is all well and dandy for some usage scenarios but breaks in others, eg. scene graphs and GUI's.
A scene graph needs 2 mutable references, and has nothing to do with ownership. Same issue exists with GUI's. The pattern that Rust forces is to always request a reference, which incurs a performance penalty while retrieving the same reference again and again and again.
I am not familiar with scene graphs, but what is the problem with borrowing or refcounting? This article showed how you can have multiple mutable references in Rust, even multiple mutable references running in parallel threads.
Ref counting is for ownership, it doesnt convey intent. It kind of accidently works but is the wrong abstraction, especially in code bases where ownership is known.
If you're building data structures that have specific requirements and you know you'll implement it correctly, you can use raw pointers like `*mut T`. That's why they're in the language, for when you need to do something that the borrow checker can't verify and don't want the overhead of runtime borrow checks.
Branch predictor can predict these pretty damn well