My reaction to the first demo (recipe) is that it was slower than typing the same thing on your keyboard.
The second demo seems to be a wash: there's no time saved in saying "move this" versus "move crab". And an app-specific contextual menu would probably be faster.
The third demo doesn't seem to warrant the use of a pointer at all, since there is only one way to interpret the prompt.
None of this means that this approach will not be successful, but there's a reason why so many attempts to revolutionize user interfaces ended up going nowhere. Talking to your computer was always supposed to be the future, but in practice, it's slower and more finicky than typing.
In fact, the only new UI paradigm of the past 28+ years appears to have been touchscreens and swipe gestures on phones. But they are a matter of necessity. No one wants to finger-paint on a desktop screen.
Talking to your computer can only ever work for people in atomized work-from-home silos, surely. I can't really imagine living in a world where everybody is just muttering commands to the computer all the time.
Working near the Android Assistant team is like this. It's pretty chaotic.
This happens daily in radiology departments around the world
Aren’t radiologists dictating notes rather than issuing commands to the computer? From supporting them for a few years, I recall them having pretty good facility using the computer to zoom/filter/isolate parts of images, and most of the muttering was speech to text or a good old tape recorder for their writeups.
The dictation happens while reviewing images and the dictation software includes support for voice macros ranging from edits to adding information from the chart and other applications. Not quite the same as just recording.
> My reaction to the first demo (recipe) is that it was slower than typing the same thing on your keyboard.
For you and me, who have used keyboard in our lives for more than 1,000 or even 10,000 hours.
There was a brief period when typing slowed people down because they could write the same information down with pen&paper, and that period eventually passed.
But, you have been using your mouse and keyboard for many years, so you know well how to use them quickly. I think that you shouldn't expect to be able to be quick with a new input type when you only tried it for a few minutes