It's a damper on development of new WMs and DEs, too. I have ideas for my own desktop I'd like to explore at some point, and if I do it'll almost certainly be X11 based initially because it's so much more quick and easy to wrap one's head around and get the iteration loop up and running with.
I'm not anti-Wayland and I think X11 has enough issues that it's worth transitioning over to something better but this is a critical weakness in Wayland's design.
Check out Louvre. Wayland compositors may not be as hard as you thought.
Or build on River as this article suggests.
How is a WM not just a simple plugin/extension? Find a display server you like and write an extension for it!
That would suffice if I were only looking to build a WM, but my goal is a full (lean) DE.
The xserver may not be completely dead either:
https:/github.com/X11Libre/xserver