I love this idea in theory, but in practice this would just mean coming up with some throwaway project and then paying them a full day to build on it. I've never had a job where there wasn't at least a few days of onboarding (often times a few weeks) before somebody could productively contribute to our applications. If it's a brand new project then sure, but how often does that happen? And when it does, do you want your completely unknown candidate doing it or one of your senior engineers who understands the big picture, how it will fit in, your company culture/values/etc?
My experience is LLMs can suggest some pretty good multi-day throwaway projects. I'd want to know can the candidate manage multiple agents, and effectively communicate/collaborate with humans.
Even something simple, like take 1 hr then explain to us this new repository in detail.
It doesn't need to be a day then hire. It can be done in stages.