https://d2lang.com/ works well?

That’s what I have been using, but from glancing through this repository, it looks like the win here is that you can define where you want things and yet still have diagram as code

Yep, exactly! D2 is in the same family as Mermaid, i.e., automatic layout, with some escape hatches like grid placement or absolute positioning. Reladraw sits in a gap between automatic layout and absolute positioning.

D2 has some support for this. You can fix the position of certain elements and then it will redraw things around it.

I’ve been thinking about something similar to this but D2 wasn’t totally open source until now.

I suspect the subject line to this "Where you decide where to place things" is a step away from D2; I find that fussy in D2!