fta: The biggest unsolved problem is making it work well on all kinds of music.
The wickedness comes from wanting something that works just as well for John Summit as the Grateful Dead as Mozart and Bad Bunny.
But it seems like you could cheat for installations where the type of music is known and go from there. The other cheat is to have a "tap" button, and to pull that data and go from there.
mental note: the thought "it can't be that hard" when obviously it is sent me down a rabbit hole for a couple of hours