I tried talking to Claude today. What a nightmare. It constantly interrupts you. I don’t mind if Claude wants to spend ten seconds thinking about its reply, but at least let ME finish my thought. Without decent turn-taking, the AI seems impolite and it’s just an icky experience. I hope tech like this gets widely distributed soon because there are so many situations in which I would love to talk with a model. If only it worked.
Agreed. English is not my native language. And I do speak it well, it's just that sometimes I need a second to think mid-sentence. None of the live chat models out there handle this well. Claude just starts answering before I've even had the chance to finish a sentence.
English is my native language, and I still have this problem all the time with voice models.
Anthropic doesn't have any realtime multimodal audio models available, they just use STT and TTS models slapped on top of Claude. So they are currently the worst provider if you actually want to use voice communication.
It's unfortunate though, because Anthropic LLMs and ecosystem is the best IMHO. Tavus (we) and Anthropic should form a partnership.
I think Anthropic currently has a slight edge for coding, but this is changing constantly with every new model. For business applications, where tool calling and multi-modality matter a lot, OpenAI is and always has been superior. Only recently Google started to put some small dents in their moat. OpenAI also has the best platform, but less because it is good and more because Google and Anthropic are truly dismal in every regard when it comes to devx. I also feel like Google has accrued an edge in hard-core science, but that is just a personal feeling and I haven't seen any hard data on this yet.
I love Anthropic's models but their realtime voice is absolutely terrible. Every time I use it there is at least once that I curse at it for interrupting me.
My main use case for OpenAI/ChatGPT at this point is realtime voice chats.
OpenAI has done a pretty great job w/ realtime (their realtime API is pretty fantastic out of the box... not perfect, but pretty fantastic and dead simple setup). I can have what feels like a legitimate conversation with AI and it's downright magical feeling.
That said, the output is created by OpenAI models so it's... not my favorite.
I sometimes use ChatGPT realtime to think through/work through a problem/idea, have it create a detailed summary, then upload that summary to Claude to let 4.5 Opus rewrite/audit and come up with a better final output.
I use Claude Code for everything, and I love Anthropic's models. I don't know why, but it wasn't until reading this that I realized: I can use Sparrow-1 with Anthropic's models within CVI. Adding this to my todo list.
Agreed. I tried using Gemini's voice interface in their app. It went like this:
===
ME: "OK, so, I have a question about the economics of medicine. Uh..." [pauses to gather thoughts to ask question]
GEMINI: "Sure! Medical economics is the field of..."
===
And it's aggravated by the fact that all the LLMs love to give you page-long responses before it's your turn to talk again!
Am I not allowed to cut you off if you're ramble-y and incoherent?
Its rude if you're a human, and entirely unacceptable if you are a computer.
The one thing that really surprised me, the thing I learned that's affected my conversational abilities the most: turn taking in conversation is a negotiation: there are no set rules. There are protocols: - bids - holds / stays - implications (semantic / prosodic)
But then the actual flow of the conversation is deeply semantic in the best conversations, and the rules are very much a "dance" or a negotiation between partners.
That's an interesting way to think about it, I like that.
It also implies that being the person who has something to say but is unable to get into the conversation due to following the conversational semantics is akin to going to a dance in your nice clothes but not being able to find a dance partner.
Yeah, I can relate to that. Maybe it's also because you are too shy to ask someone to dance. I think I learned that lesson: just ask, and be unafraid to fail. Things tend to work themselves out. Much of this is experimentation. I think our models need to be open to that: which is one cool thing about Sparrow-1: it's a meta-in-context learner. This means that when it try's and fails, or you try and fail, it learns at runtime to adapt.