Is this a good way to improve performance (frame rate, latency, CPU load) ?

Yea!

* Do video playback out of the browser. You can render a subset of frames, use a different pipeline for decode etc...

* Pull video from a different source. Join Google Meet on current computer, but stream from another host.