Layout is one of those things humans do so easily and intuitively, yet you couldn't write an easy algorithm for it. I wonder if there's potential to use gen ai to achieve human like results. Anyone has any thoughts on feasibility and complexity of such an approach?
I dabbled in this area, there are poster layout generation attempts that use gen ai to come up with an initial layout plan, and even feed the visualization back into the llm for iterative fine-tuning.
I was intrigued, but couldn't make it work reliably. Perhaps I forgot to add "make it look nice" to my prompts.
https://arxiv.org/abs/2412.04237v3
https://haoyuchen.com/POSTA
https://github.com/microsoft/LayoutGeneration/blob/main/Layo...
Thanks for sharing. Although diagrams have a unique requirement that the boxes are inter-connected, and those connections must also look "nice".
I think the hard part is more defining the different types of visually pleasing layouts that you want to support. What to align to what, what metrics to minimize, etc.
There is no single layout definition that looks good for all applications. What looks good for one type of graph data (flow charts, org charts, ASTs, family trees, etc, etc) may not look good for another.
Once you've reduced a desired type of layout to a set of layout rules, then I don't think implementing it is the bottleneck, and it seems this part could even be automated if wanted (e.g. evolve layout to best meet constraints).