The need for actual lawyers will persist I think from my own experience. I attempted drafting a contract with some points myself using AI, but after several edits I wasn't sure if it was correct. Sending it to an actual lawyer ended up in so many corrections I couldn't imagine the first time. One big thing was the overly excessive protective clauses which didn't make sense for reality or conflicted with another.
Its just like code I suppose, if you can read and understand and validate, you can use it to scale and otherwise it could end up being a vibe effort.
I'd correct this to "The need for actual GOOD lawyers will persist I think from my own experience."
I have worked with many lawyers in my career. The bad ones will look at a contract and tell you fifty things that can be improved. Meanwhile, you take the same contract to a really great lawyer and they tell you that's it's not worth fighting over the changes and to just keep it as is. It's kind of like how a beginner or intermediate software engineer might tell you how you can build out a k8s or auto-scaling fargate cluster for your project and a really senior engineer might look at the same thing and explain that it's not worth worrying about scalability right now.
I find that when I feed contracts to ChatGPT (which I do all the time) it will try to find 50 things wrong with it. For example, as a test, i fed chatgpt an employment offer from a US government agency. It came back with 1152 words and 14 recommended (some of them substantial) changes. Meanwhile, any half-decent employment attorney would probably laugh at me if I asked them to redline an employment offer from the State Department.
> Meanwhile, any half-decent employment attorney would probably laugh at me if I asked them to redline an employment offer from the State Department.
That might not be due to the contents but rather due to the fact that they won't negotiate I'd presume
"the hierarchy of knowledge", right. A bad or insecure speaker will ramble about insubstantial minutae whilst a good one will get straight to the key points that form an argument. Basically the confidence to prioritise information.
I’ve found that good lawyers are about getting things done. You could nitpick the terms of a contract, but would you rather buy the house or have a perfect contract? Once I realized that, my view of lawyers changed (for the better).
YAGNI isn't just for software.
In my recent experience, it seems like LLMs may even increase need/demand for actual lawyers (more ideas + more words = more lawyer time). I expect lawyers will generally adopt the technologies that will benefit them, rather than the technologies that would increase standardization and trust
The same has happened with modern finance. Despite more technology that has power to drive more transparent/efficient markets, we have less perfect information sharing and a larger group of middlemen capturing a growing share of profits of the economy
I think you're onto something, but it'll be more of a reshaping of practitioner distribution to match the mode of work (perhaps fewer employment lawyers, many more litigators/ip litigators, etc...)
Not trying to be dismissive here but a lawyer will always edit your proposal. I do think lawyers will persist but not as many. And they will work very differently, much like we already use claude / codex to code - a big part of the contract probably won't be read by the lawyer.
I think the argument goes, even if 90% of a lawyers markups aren't useful, the 10% that are, are basically what you're paying them for.
I would rather argue they SPECIFICALLY will read everything. However, they'll likely often just be like "I'd phrase this differently but that works, too".
Have you watched Nathan for you? There is a lawyer that didn’t read on tv. And almost got in trouble.
LLMs are the first genuinely useful legal tech since the Internet. I'm pretty shocked, though, at the delta between how competent Claude is on code versus legal work. It's good for research and data organization, but terrible for drafting. I wonder if this is a structural problem with the lack of feedback loops. In law, there's no compiler to check for logical or continuity errors in your brief, and there's no unit tests to check for correctness or performance.
Even without that, I think it'll be extremely valuable to clients to allow them to answer simple questions without a lawyer, figure out the lay of the land so they can supervise their counsel, etc.
You should know - for coding they make terrible mistakes as well.
But programmers have this concept of a "code review" where another person looks at the code to look for problems.
We use this same technique with our LLMs. Most mistakes are caught by having a second LLM look at it. Doesn't even need to be a different model - just make sure it has a different context.
> Doesn't even need to be a different model - just make sure it has a different context.
I find that the number of issues discovered is noticeably higher if you do use different models though. I'm doing some very finicky things (formal semantics) and find that there's value in review panels as large as 5 different models from different families. It gets even more profitable if you set it up as a truly agentic panel where after writing their own separate reviews they get to see what others have said, and adjust their positions or defend them etc. Some models are not that good by themselves but can be surprisingly good at finding flaws in others' reasoning (Grok for example).
Programming has a long standing culture of accepting the code to be somewhat wrong, so we have various tests, linters, reviews and error handling. Also in programming there are many ways to do something and it's the end result that matters most.
Not so in other knowledge work. There's no test harness for a contract and error is non-recoverable. Likewise in finance. There are specific ways of doing things and these ways are many times set in regulations. LLMs can assist all day, sure. But replacing the human, in highly regulated, zero tolerance for error environment?
The highly regulated, zero tolerance for error environment is a huge problem because anyone doing these jobs is more like a small sample size LORA than a general model.
It doesn't matter how smart someone is, they need specialized training to be good at these jobs. Specialized training in the area the company specializes in.
There is a category error in all this that is hard to think about because of the normal discourse and ordinary language. We say people work in "finance" but no one works in just "finance". They work at a company that has a specialization within "finance", inside a hierarchy that has specialization on top of specialization.
What we really need is exactly what we don't have and aren't going to get. A type of LORA that generalizes the task specific intelligence needed from a very small sample size and that in practice makes so many less mistakes in a highly regulated, zero tolerance for error environment that it is irresponsible to not use the model.
I have worked in this type of environment for 3 years and I have made zero mistakes in 3 years. The people that make even a small number of mistakes get fired.
Any real automation in this area is going to be incredibly slow and piecemeal over a long period of time because even an amazing model would need a long time to prove itself against what the human standards for error rates are.
Even the ensemble average error rate on a large number of tasks in space would not be good enough. It needs to be an average error rate over time.
You could say the Salvatorian Clause in contracts is like exception handling: a "catch (all)": even if some clauses in this contract are illegal, the remaining contract stays in place.
Logically, this actually doesn't make sense strictly speaking because the sentence creates a paradox: doesn't it make clear whether it includes itself or not, and each reading ends up in trouble. There is a "tradition" in law around the world to accept the only benign reading of such clauses, which I always found funny given that in all other ways lawyers adopt the most adversarial mindset imaginable.
I don't understand. I just provided advice about getting better output. Are you trying to reply to someone else?
You made some points worthy of expansion:
>> You should know - for coding they make terrible mistakes as well.
>> But programmers have this concept of a "code review" where another person looks at the code to look for problems.
> LLMs are the first genuinely useful legal tech since the Internet
That is an incredible statement that could not be further from the truth. Large scale adoption of email, searchable document databases like Westlaw, LexisNexis, PACER, etc.. , OCR Software, electronic signatures, and tons more have had a much more defineably positive impact on the legal profession since the internet came about.
But emphasising "tech" advancements, that really only covers OCR.
I think "email" and "westlaw" fairly count as "the Internet." LLMs might be bigger than either of those.
The internet came well before both of those. And none of the example I gave have lead to lawyers regularly being fined in court because of "hallucinations"
I reckon law and medicine are really premier use case for llms since those areas are all about having vast knowledge(knowing all about obsure cases in law or about an very rare disease the average doctor wouldn't have heard about)
I've heard this argument (basically, you can just have good legal skills and adapt your practice to more areas of law) but we're still pretty far from a layperson confidently navigating court (you could say the same about code).
That's exactly it. You basically can't set up deterministic regression tests, which makes development of (legal capabilities) even more abstract and messy.
I have found that it’s useful generally speaking to get the intent of contracts and red lines, but actual drafting I agree is where I lose all confidence. My guess is that the significance of the difference between using a word like “and“ or “or“ can be so meaningful that that level of nuance can often be lost. But I know nothing I’m not in the space, I just pay too much money for lawyers.
Law is certainly a less verifiable domain
I want to take a step back and observe that you're saying expensive specialists will continue to be required if two parties want to make an agreement. That seems like something worth fixing rather than just accepting.
> I want to take a step back and observe that you're saying expensive specialists will continue to be required if two parties want to make an agreement.
A lot of contract-lawyering involves getting ahead of when things go south and the two previously agreeing parties find themselves in disagreement.
Same for SWE. AIs will churn out happy path vibe slop and fail when something unexpected happens by swallowing errors or having a bunch of terrible "fallbacks".
The problem is those two people are in conflict not collaboration.
If the expensive person is still better at getting what _you_ want from a contract then it’s always worth doing that.
There’s a reason lawyers are so expensive to begin with - they do very complicated stuff with very expensive consequences if done badly.
It's probably somewhat of a zero sum game. Both sides lawyers have AI tools which probably just keeps them on a level playing field rather than actually making anything cheaper or faster.
I spoke to a very very high up corporate lawyer specialising in tech - so knows Ai is real. And they said AI contracts are the bane of their existence. They’d rather you arrive with no idea what you want than a contract partially written by AI. They said it usually takes longer with an AI starting point than starting from scratch.
Millions if not billions of agreements are made every day without the involvement of lawyers. Lawyers are only needed for particularly complex situations and even there they are generally only "needed" in the sense that parties feel they would add value.
Yeah, and you can already get fill in the blank legal forms for a lot of basic situations, including leases, wills, employment agreements, etc.
The interesting question is whether AI can fill a niche these can’t — or, alternatively, help a layperson evaluate a contract they’re asked to sign better than they could themselves.
In my experience, those expensive specialists help me protect against things me and the counterparty cannot even imagine when making an agreement. Do you mean that this would be fixed by AI or by making simpler agreements?
It can imagine what yourself report that you cannot :)
Friction is inherent to a low-trust transaction. An agreement between two business entities is kind of the epitome of that.
How?
Contract law has evolved because transfering the ownership of things is fraught with difficulty. Ownership as a concept itself is notoriously hard to define.
Consider housing - when you buy a house you own that land. Except if its an apartment, you just own the internal walls. Or when the government aquires your land through eminent domain. Or mineral rights - which don't belong to you, you just own the top 5 feet of topsoil. Or if you neglect a property and some squats in it. Or if you were given a fraudulent payment, and they take back the property.
Just getting people to agree is notoriously difficult when people have different often conflicting desires.
Most people dont appreciate that the world is a complex place. Nothing is ever simple, sure concepts can be extremely simplified but the nuances and details are lost. These details are the important parts, just ask a surgeon or rocket scientist etc.
Also, unlike SW engineers, lawyers control their industry and will quickly circle the wagons to protect themselves. Paralegals are probably screwed, but lawyers will do fine. I'm considering law as my post-ai-SWE-apocolypse career path.
I’m thinking asking the same lines, but also looking into research / lab chemist so I can hang out with my best friend. If im going to change careers so drastically, I might as well enjoy the company.
Human professionals put their reputation and finances at risk when performing their work. This risk functions as a guarantee.
> Human professionals put their reputation and finances at risk when performing their work. This risk functions as a guarantee.
Do you know how difficult and costly it is to sue an attorney for malpractice?
Yes. But this option still exists in addition to reputation which I mentioned.
With AI models this option does not exist. I can't sue OpenAI because the contract that I got ChatGPT to draft carelessly exposes me to some type of financial loss. I can't contact the state bar and have Open AI banned from drafting up contracts or doing any other work that lawyers are licenced to perform. Even if I could it would not compare to a lawyer loosing their license to practice for life. If I tell a friend or acquaintance about my negative experience they will still likely use OpenAI for professional work and chalk up my experience to a skill issue or premature use of the technology.
Also note the use of the word "professionals". I was referring to doctors, engineers, accountants etc too.
You wrote:
> This risk functions as a guarantee.
Except it doesn't, particularly as far as lawyers are concerned. There are almost a million and a half active attorneys in the US. Just 500-600 are disbarred every year, and almost always for money-related violations, not bad advice. Public discipline is also relatively uncommon (about a quarter of 1% of attorneys every year) and usually related to issues like financial impropriety, neglect, undisclosed conflicts of interest and failure to communicate, not disagreement over the quality of advice.
If a lawyer drafts an agreement that causes a loss, a bar complaint will not help you recover your money. You'd need to sue for malpractice. This costs a lot of money and takes a lot of time. Because you're going up against a lawyer, expect the defense to be vigorous, especially, ironically, if you hired an attorney at a good firm. The full weight of the firm will back the attorney and you might find it difficult to find an attorney, as many won't want to be involved in suing a fellow professional in their market.
I don't disagree with your fundamental premise about the value of human responsibility, but these professions have been structured to protect their members and the ABA in particular basically functions like a cartel today.
Do they?
Not being snarky here but I've encountered some really crappy lawyers and they seem to be doing great ($$$).
It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages. There should be a safe harbor for that.
If you don't have the resources to defend yourself then it's probably not wise to publish things like this but you can tell others of your experience.
Also for those things you could sue for damages or lodge a complaint for malpractice.
Assuming you are in the US, there is a safe harbour called "opinion based on disclosed facts" for that. You could also fairly argue that the conduct of a lawyer is a matter of public concern
> It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages.
That's pretty much what bar associations are for, filing a complaint against an attorney at the bar seems like a pretty standard procedure.
>There should be a safe harbor for that.
A safe harbor would be the opposite, it would be immunity for the lawyer.
Even if there were it would just be another court case
I was in a similar situation and tried a different approach.
I had start with asking for a contact with some details I provided. But then I ask the model to be an experienced corporate lawyer and ask me a series of questions to gather the details it needed and then write the contract. The result covered a lot of details that were highly relevant but were absent from the original attempt.
The key insight is that you can lean on the model to cover your unknown unknowns.
Your comment is a great example of the phenomenon where people think AI is an expert in areas that are not because they have no way to determine if the seemingly good looking result is genius, gibberish, or somewhere in between.
Here's another techniqe that gets you (even) more out of LLMs: run each prompt several times and compare the various outputs. It is not uncommon for models to contradict their own advice, and also you will get additional insights not included in previous runs.
This is due to the fact that LLMs are statistical processes that rely on pseudo random numbers in chosing what to say and how to say it to a substantial degree.
The point was that you can go further than "write me a contact that says x,y and z". I actually believe the process I went through with the AI was a more custom, tailored experience than any interaction I have had with actual lawyers, who'd usually apply a template. The AI asked me a bunch of questions, many of which were relevant, and led to follow up questions. I could ask my own questions, make corrections, etc. and all along, they made sense to me. This part of the experience was great.
There could still be more unknowns that I did not encounter and by definition don't know about.
And the final output could be wrong, wording, exact legal terminology etc.
Can we say working with an actual lawyer is guaranteed to be better, more correct? Lawyers are people and they also make mistakes, wake up on the wrong side of the bed, hate their job etc, just like the rest of us.
> The result covered a lot of details that were highly relevant but were absent from the original attempt
Another one suckered by the plausibility engines.
Did you validate the output with a lawyer?
No admittedly not. I can try it and report back.
Yes the probabilistic smudging machine does not make many redlines with confidence, unless A) directed or B) it's a painfully out-of-distribution argument.
> The need for actual lawyers will persist..
I agree. But, we will need a lot fewer of them. My small company called our lawyer a lot more before the advent of AI. Mostly to clarify a few things or review simpler contracts. Now, for simple things, we do not call them at all.
Their work will get cheaper so it will become more accessible for people, increasing demand over all would be my guess to some degree at least.
I anticipate an effect on society that troubles me:
(1) Rich people will use human expertise and "the rest of us" will use AI models to get by. This could happen in law, but also in medicine (in particular in societies that - like in the US - do not have universal healthcare).
(2) A positive effect in both domains may be that access to knowledge will be broader and cheaper.
I hope I will be wrong about (1) - I would not like to live in such a two-class society.
(1) already exists. A more banal example is rich people get their kids private tutors. Poor kids lean more on public resources, wiki, the youtube and the internet. Economic forces will always dictate this kind of change.
> One big thing was the overly excessive protective clauses which didn't make sense for reality or conflicted with another.
I'm currently negotiating with someone who I feel is clearly using AI and this is EXACTLY what has been happening
> The need for actual lawyers will persist I think
But will their glamorous salaries persist? That is the question that matters.
AI doesn't need to wipe out lawyers. If they just depress salaries enough, virtually nobody is going to want to be a lawyer anymore.
Biglaw salary will probably keep going up for at least a decade. Some high street lawyers could be in trouble. Same story as before.
The salaries are a result of the insane hours they have to do. Biglaw needs to pay those salaries to keep getting young associates in the door because burnout is a real problem in the industry. If salaries were lower and hours/stresses were lower, you would probably still find people wanting to be lawyers. A different kind of person, maybe.
(Not that I think AI will necessarily have that effect. We just don't know yet)
Salaries are about scarcity and leverage, they have little to do with working hours. Nurses pull insane hours and they still don't get paid as much as doctors.
Very few people, if any, are going to bust ass in law school or medical school if they're not going to get rich off it. Better to do literally anything else.
In Biglaw, the scarcity is driven by the hours and stress. If it was a chilled 9-5 job you would certainly have more people willing to do it, even if it paid less.
It’s a professional organization that only allows so many people to pass the bar per year so I don’t see that changing anytime soon… if anything if they need fewer lawyers they will just pass fewer people to keep their wages high
> The need for actual lawyers will persist I think from my own experience
This doesn’t replace lawyers, but paralegals surely will be affected. A good enough model could shrink the number of paralegals needed in a firm.
A lot of paralegal work has already been eliminated by intake automation. OCR and text-to-speech kicked things off.
As costs for legal work drop we might expect demand for that work to increase. For example, it may be the case that legal help becomes accessible to entire segments of the population that had no access previously.
The best writer I know of was an associate attorney. He didn't have the technical background nor the in depth computer related knowledge, and relying on the information I fed him. But man the briefs he filed to the court were amazingly good. Reading them I would have been convinced his side was right if I were in the jury.
Law LLM will surely help competent lawyers in their fields with greater sources of knowledge not in their core area of expertise.
I don't want to imagine a world where juries were responsible for reading briefs and making decisions.
Sure as long as they stop making things up.
you can't in imagine the same way you couldn't imagine generating the text from your initial prompt. when will the hubris end?
some variant of "oh i am capable of generating text i know nothing about, but it helps to have a human i can rely on to tell me more about the text i know nothing about."
pick up a bachelors degree worth of books to humble yourself
And to add to your last point yes it should also be like code: where the costs of development are coming down. Legal costs should trend down if law firms are effectively leveraging LLMs to accellerate their procedures (though I won't hold my breath)
I got the exact opposite. A contract from a lawyer passed through ai found so many flaws.
> The need for actual lawyers will persist I think from my own experience.
The outcome of a case shouldn't depend on someone's fallible ability to recall facts or convince other people or point their index finger*
Law should generally be deterministic. One's CHA stat should have no bearing on justice.
There should still be human judges, but the middleman between the judge and petitioner could easily be removed, and have generally been seen as leeches since forever anyway.
Though, like how the USA opts to remain in the Stone Age with regard to tax filing because of lobbying by tax software companies, this faction of society will flail the hardest before they admit they're obsolete.
----
* What's a lawyer's favorite programming language? Objection C.
The ability to clearly and effectively communicate, and the ability to know and recall the facts of a case is always going to be important. It'd probably be a more fair system if judges and jurors couldn't see the attorneys or even the people they're representing. Now that many cases are handled over video calls it'd be pretty easy to make that change.
A lot of communication is conveyed through more than the words themselves though and even though it's often misinterpreted people put a lot of value in that information. I suspect that even if we took away the ability to see the people involved judges and jurors would still find bias in the tone/quality of people's voices. Even if everything was reduced to text they'd find bias based on word choices, phrasing, spelling, and grammar.
Yep, just yesterday, Nike removed one leach middlemen called retailers and sell directly through their app. Turns out working out very well for them too.
If you have been following this story for the past couple of years, this has been disastrous for Nike.
I mean the law is somewhat deterministic, an amazing lawyer can't just win any case.
An Amazing Lawyer not winning an unwinnable case = a good thing.
A Mediocre Lawyer not winning a winnable case = not a good thing.
Say a starving child steals a loaf of bread. The law says all theft is at least 1 year in prison. But the judge can see that the child is actually destitute and it was his first time. The judge could shorten the sentence to 1 week; that's fine, there should be room for interpretation and leniency.
But it should NOT depend on whether that child had a persuasive lawyer or not.
A good "AI Lawyer" would simply behave as how ALL lawyers SHOULD: Simply present the facts, the laws that apply to those, offer suggestions for the verdict, and the possible long-term consequences for each possible verdict, without injecting dramatization, exaggeration, or attempts to pull at heartstrings etc.
The law is much more interpretation and argument based than logic routing based than you seem to believe.
Lawyers sure. Paralegals and assistants are cooked in the way typist and secretaries were back in the day.
Nope. Paralegals and assistants do the type of thing GP hired a lawyer for (and lawyers do, as well).
Lawyers can practice, can represent you in court, etc. that's not going away soon.
Lawyers have that lovely regulatory wall, for better or worse.
The typical pattern is called “deskilling”. It doesn’t usually mean a skilled profession will disappear overnight. Instead, the job might be done by less expensive folks like paralegals.
An example is in the banking industry, where making a loan used to require deep analysis of a person’s credit worthiness. Now they use an algorithm (credit scores) which means someone with less experience can do it.
If law follows the same pattern, a job done by someone making $500/hour might be done by someone making $50/hour.
If you think paralegals + AI can do the job of good lawyers you don't understand the profession at all.
i don't think that's the point anybody is making. "good lawyers" is not the profession at risk here. a top tier courtroom lawyer won't be replaced by AI.
but how much of the revenue of the average law firm comes from that, vs the day-to-day "we need this relatively routine contract reviewed" sort of work?
Yes. And it would be the continuation of a trend that’s already been going on for two decades. When I started out as an associate at a fancy Wall Street firm, we had 30 people working on reviewing documents for this matter even that level of firm today would not be able to justify doing that anymore. We’re going to move from a pyramid model to more of a diamond shaped model with many fewer associated attorneys doing the legal research and information marshaling.
> Instead, the job might be done by less expensive folks like paralegals.
So who is going to define your strategy, represent you in court, adapt the strategy to changing circumstances, negotiate with counter-parties on a mutually accepted settlement? A paralegal? An AI model? Please
Very little of corporate law is in court. The mass of things is “does this contract look good?” And “what do we need to change in our business when regulation X comes into force?”
Yes hence why only 25% of the situations I presented were in court.
Former attorney here. A good paralegal is worth 3-4 junior attorneys. A good paralegal with AI should probably still have a senior attorney oversee them but this will absolutely hollow out firms.
He just gad better prompt
"law" and those who pratice "law" have managed to place itself largly in a monopoly. This is a huge drain on the economy. Law is expensive and irrational.
It's worth noting that the announcement seems to aim the product at lawyers or students rather than laymen.
Our entire world is about reducing the friction of conducting a transaction
How to fix the lack of trust which is needed to conduct a transaction
Contracts affix a moment of trust in time, so even when trust is lost between parties the contract has guardrails for behavior and consequences for acting outside those rails
There are many transactions that are completely inaccessible to most people because the need for a contract or lawyer is too high
LLM use allows for those inaccessible transactions to decrease, and get people further along to the point where a lawyer is accessible as well, and economically viable to use
> One big thing was the overly excessive protective clauses which didn't make sense for reality or conflicted with another.
>Its just like code I suppose, if you can read and understand and validate
I share this, when I asked an attorney for a contract, they were much simpler when compared to self redacted or AI redacted contract, it was as if I were rediscovering lots of concepts and I HAD to put them in paper, even if they were defaults and were essentially no-ops.
Comparable to:
> Mkdir $PWD/path
instead of
> mkdir path
or conditions so rare that they don't merit including.
> x = 0
> if x!=0:
> raise Exception("Bit flip by solar rays or otherwise")
>The need for actual lawyers will persist I think from my own experience
At the very least because the attorney monopoly assigns a 5ish year of training as PoW and natural personhood as Identity as protection for Sybil Attacks.
If anything, the exclusive right of attorneys to represent clients in court, file motions, and enjoy professional secret, is more valuable, as it's not something AI can ever compete at. (Barring a wild recognition of machines as humans, or an overturn of the impossibility of companies to appear in court.)
What model did you use?
[dead]