Hi I am Sagar, We just open-sourced a complete framework to build an AI-powered telephony agent that can handle both inbound and outbound calls—using Python, SIP, and cloud LLMs like OpenAI or Gemini.
You can use it to create smart appointment bots, voice feedback collectors, or even enterprise IVR systems. It’s modular (plug in your SIP provider or AI model), production-ready, and extensible for real-time workflows.
Features include:
SIP & VoIP call handling (Twilio, Plivo, etc)
LLM-integrated AI agent (customizable prompt & tools)
FastAPI-based server for routing and control
Plugins for STT, TTS, sentiment analysis
Support for Agent2Agent and MCP protocols
GitHub Repo:https://github.com/videosdk-live/agents Full Blog: https://www.videosdk.live/blog/ai-telephony-agent-inbound-ou...
Would love feedback from anyone working with telephony, LLMs, or real-time automation!
I'm still getting to grips with all this AI stuff but I do know SIP n RTP rather well. Your project looks ideal for me to play with.
My initial project will be to replace my aging parent's and my home FreePBX IVR with something a bit more useful. At the moment it requests you press 1 to make the phones ring and that drops pretty much all unwanted calls but that will soon be automatically defeated.
I do also have some rather more commercial ideas but baby steps. Mind you a RPi based drop in box with a decent auto attendant for say £100 hardware, open source software and some fettling time seems a pretty good opportunity. You could go for somewhat cheaper hardware. The UK is being migrated away from copper A and B wires to what is called SOGEA and/or FTTP. Basically its going to be all VoIP for telephony from pretty much now on. The good old days of an analogue handset that is powered by the line will mostly go away within a few years. On the bright side our signalling was bloody weird and SIP is nearly eternal!