My experience is that pretty frequently the LLM just refuses to actually supply json conforming to the model and summarizes the input instead. Even with several retries configured I still couldn't rely on it. I only spent an afternoon with it though so it's possible I'm just doing it wrong (either in how I'm prompting or in how I'm configuring pydantic-ai).
How recently was that? I made a few improvements earlier this month: https://news.ycombinator.com/item?id=45058214
If the issue is still showing on the latest version, seeing the Pydantic model/schema would be very helpful.
It was about a month ago. I'll take another swing at it and make an issue if I can't overcome it.
Thanks for being a proactive kind of maintainer. The world is better because of people like you.