> As a CTO of an Ai first law firm I would like to add that most human lawyers are absolutely terrible
This is many people at many jobs. We continue to see this weird thing when AI comes into an industry that suddenly the product being produced by the people was perfect/amazing/whatever. Maybe that's true for the majority of HN who are lucky enough to work with experts in every field, but in the general population that's simply not the case.
Many doctors, lawyers, and programmers are not very good at their jobs - I've seen it first hand. AI gives the general population a way to steer around these people a bit, and maybe ask the right questions. Is it perfect? Nope, but it's often better than the people someone has access to.
The problem is that a layman has no way to know if what the llm told them is better than experts around them, or even just remotely correct. A wrong response and a correct, informative, helpful response look exactly the same, the LLM will express the same level of confidence and will defend them in both cases.
So, on one hand you can get an expert take that might be wrong, but is linked to an actual person, with a reputation and some level of ownership. On the other hand you have an over confident LLM that might be wrong and has no reputation, no ownership. How does that actually improve things compared to the older status quo?
Your argument that ownership and reputation generally ensures better answers than agents is absurd.
It is entirely clear that is it not the case when I compare agent generated apps with what contracted software teams have produced.
For medicine it is likely worse.
Doctors do care, but they have to give an advice based on their 15 year old knowledge - they simply can't read through 83 papers in a quick session.
I think it is a matter of time before we see the first insurance companies assign greater risk to human advice (legal, tech, medicine, etc.) than to agentic advice.
15 year old knowledge is fine in most cases. Human diseases and their treatments haven't really changed in that time, and those that have are pretty well known. You will get the "standard of care" as a starting point.
You are citing a legacy institutional system where updating a "standard of care" was a major institutional chance.
You likely have to change your idea about this. Heck, this view was wrong 6 months ago. Sticking with is becomes a hazard to patients.
> Your argument that ownership and reputation generally ensures better answers than agents is absurd.
That's not my argument... I didn't draw the conclusion that because of those 2 factors humans generally provide better answers. I personally have no idea if that's the case, and for sure wouldn't rely on my personal feelings to evaluate that
You clearly appear to setup that argument
> So, on one hand you can get an expert take that might be wrong, but is linked to an actual person, with a reputation and some level of ownership.
Regardless, I agree.
The newest studies still works on llms that are two years old. It doesn't appear that proper medical harnesses with frontier models has been evaluated.
My slight intuition is that we will already now see results that are much better than average human doctors.
This argument can be applied to anything LLM generated, and doesn’t pan out. An LLM can quickly respond with cases, interpretations, and standards in minutes as a defense to its claims. You can then independently research online if you believe those claims to be correct. With a lawyer you’re lucky to get a response back in less than a week and to have your name spelled correctly on the documents they return to you.
> This argument can be applied to anything LLM generated
That's the whole point, yes. LLMs are inherently unreliable. Both human experts and LLMs are unreliable in their own ways. The human expert has a reputation and some level of responsibility, the LLM doesn't
Remember the bell curve in grading? Yeah, most of the graduates from every school everywhere did NOT get straight A’s in their classes. On average, they are B students… at best. And half of them are worse than average.