The conversation examples are impressive but this made me say holy shit out loud: "it runs locally on a MacBook Pro M4 Max or better. And we figured out how to train Dessa on only one NVIDIA GB200 NVL72."
If it really can replicate that low latency + that expressiveness on local devices, that's one heck of an accomplishment for "indie" AI.
Speech is in quite a weird spot when it comes to locally running it. Some of the best STT and TTS models run at faster-than-realtime speed on decent consumer hardware with quite small models (so also no huge RAM requirements), but the tooling and library ecosystem is much more fragmented, making it a lot harder to build actual working products around it. There is also a much more noticeable coverage / quality gap when it comes to languages other than English.
As a speech-to-speech model Dessa is of course in a different ballpark, but given the intelligence it exhibits, it seems to be working with not that many parameters. The developers also seem to be admitting that to some degree ("in fairness, LLMs also have to learn C++ and whatever the hell Angular is"). I'm wondering what the utility of this would be. Given that it lacks both broad world knowledge and tool calling integration in any way, that seems kind of like a dead-end.
Probably Dessa started those demo conversations with some context, so maybe it could receive documents and speak about them. If those were random conversations started with no context, the people at Block probably recorded a lot of uninteresting chat and cherry picked the few good ones.
What prevents them from adding tool integration?
How would those tool calls work? Would it say what it wants and another model/system act on that and 'read aloud' the document to it?
I guess that's somewhat how we do it, so maybe it would work.
It's a bit strange to call it a dead end when this was trained on just a single NVIDIA GB200 NVL72. LLMs couldn't do much back then, either.
> that's one heck of an accomplishment for "indie" AI.
It's a nice accomplishment, but I think we have very different definitions of "indie".
https://en.wikipedia.org/wiki/Block,_Inc.#
> one NVIDIA GB200 NVL72
That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).
Many AI shops are funded well into tens of millions if not hundreds, for early rounds.
Also the team size is really small (< 10???). So yes, indie might be a stretch but still very impressive, relatively speaking.
Now taking the other side, they didn’t have to worry about hosting/billing/subscriptions or any of that yet. It’s a research demo at this point.
> > one NVIDIA GB200 NVL72 That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).
that’s a good clarification, skimming some of the comments here I was thinking they trained it on a single GB200 gpu. THAT would be a huge win for indie AI
Edit: Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems. The claim of “we trained this on a single rack of gpus” is definitely much less impressive than “we trained this on a single gpu”
> Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems.
It's even more confusing than that :)
Think of GB200 as something like Apple's 'Ultra' chip. Not exactly, but similar. They fuse the chips together with a very fast interconnect (connection).Also a speech to speech model it can't do any tool use or you can't guide it much am I mistaken? How will it be used for agentic uses is the part I don't get. Edit: this is response from Argon if anyone was also curious: You are not mistaken about Dessa: because it was trained solely on raw audio with zero text data, it cannot read text system prompts or output code for tool calls on its own. To use it as an agent, developers must either steer it purely through spoken voice instructions or pair it as a conversational "voice front-end" alongside a separate text LLM that listens to the audio and executes the tools in the background.
> No, but I speak the language of binary load lifters, which is very similar to your moisture vaporators
> they didn’t have to worry about hosting/billing/subscriptions or any of that yet
Or marketing, which they kinda get for free. So much research goes unseen, there’s huge competition for attention, and most research is very limited in that regard
I would say that’s the biggest factor that makes the team “not indie”
At the same time, like you said, still amazing accomplishment, very cool model, and great work
Wait, seriously, a rack of 72 GPUs costs $4 million?
Hard to find concrete pricing [0], but apparently a pair can cost as much as $8.8M (depending on config)[1]
[0]: https://www.nvidia.com/en-us/data-center/gb200-nvl72/
[1]: https://www.tomshardware.com/tech-industry/artificial-intell...
This rack has 14 TB of ultra-fast VRAM.
oh, hacker news, where someone will act like a ten person shop with a seven million dollar computer might be indie
Perhaps boutique is a better term?
> it runs locally on a MacBook Pro M4 Max
Open weights or GTFO
I mean cartesia did one of their first demos on a Mac book pro, live.
Still extremely impressive but not unheard of.
Wetware can't stop getting mogged.
By 2030 we'll have this running on old pentiums.