let me get this straight, you are storing convo threads / context in DOs?
e.g. Deepgram (STT) via websocket -> DO -> LLM API -> TTS?
let me get this straight, you are storing convo threads / context in DOs?
e.g. Deepgram (STT) via websocket -> DO -> LLM API -> TTS?
Yes DO let you handle long lived websocket connections. I think this is unique to Cloudflare. AWS or Google Cloud don't seem to offer these things (statefulness basically).
Same with TTS: some like Deepgram and ElevenLabs let you stream the LLM text (or chunks per sentence) over their websocket API, making your Voice AI bot really really low latency.