They're currently already relying on overworked, underpaid interns who draft those documents. The lawyer is checking it anyway. Now the lawyer and his intern have time to check it.
I suggest we do not repeat the myth and urban legend that LLMs are good for legal document review.
I had a couple of real use cases used for real clients who were hyped about LLMs to be used for document review and trying to save salary, for Engish language documents.
We've found Kira, Luminance and similar due diligence project management stuff as useful being a timesaver if done right. But not LLMs.
Due to longer context windows, it is possible to ask LLMs the usual hazy questions that people ask in a due diligence review (many of which can be answered dozens of different ways by human lawyers). Is there a most favoured nation provision in the contract, is there a financial cap limiting the liability of the seller or the buyer, governing law etc.
Considering risks of uploading such documents into ChatGPT, you are stuck with Copilot M365 etc. or some outrageously expensive "legal specific" LLMs that I cannot test.
Just to be curious with Copilot I've asked five rather simple questions for three different agreements (where we had the golden answer), and the results were quite unequal, but mostly useless - in one contract, it incorrectly reported for all questions that these cannot be answered based on the contract (while the answers were clearly included in the document), in an another, two questions were answered correctly, two questions not answered precisely (just governing law being US instead of the correct answer being Michigan, even after reprompting to give the state level answer, not "USA") and hallucinated one answer incorrectly. In the third one, three answeres were hallucinated incorrectly, answered one correctly and one provision was not found.
Of course, it's better to have a LEGAL specific benchmark for this, but 75% hallucination in complex questions is not something that helps your workflow (https://hai.stanford.edu/news/hallucinating-law-legal-mistak...)
I don't recommend at least LLMs to anyone for legal document reviews, even for the English language.
I have no idea what type of law you're talking about here, but (given the context of the thread) I can guarantee you major firms working on M&As are most definitely not using underpaid interns to draft those documents. They are overpaid qualified solicitors.
They're currently already relying on overworked, underpaid interns who draft those documents. The lawyer is checking it anyway. Now the lawyer and his intern have time to check it.
I suggest we do not repeat the myth and urban legend that LLMs are good for legal document review. I had a couple of real use cases used for real clients who were hyped about LLMs to be used for document review and trying to save salary, for Engish language documents. We've found Kira, Luminance and similar due diligence project management stuff as useful being a timesaver if done right. But not LLMs. Due to longer context windows, it is possible to ask LLMs the usual hazy questions that people ask in a due diligence review (many of which can be answered dozens of different ways by human lawyers). Is there a most favoured nation provision in the contract, is there a financial cap limiting the liability of the seller or the buyer, governing law etc. Considering risks of uploading such documents into ChatGPT, you are stuck with Copilot M365 etc. or some outrageously expensive "legal specific" LLMs that I cannot test. Just to be curious with Copilot I've asked five rather simple questions for three different agreements (where we had the golden answer), and the results were quite unequal, but mostly useless - in one contract, it incorrectly reported for all questions that these cannot be answered based on the contract (while the answers were clearly included in the document), in an another, two questions were answered correctly, two questions not answered precisely (just governing law being US instead of the correct answer being Michigan, even after reprompting to give the state level answer, not "USA") and hallucinated one answer incorrectly. In the third one, three answeres were hallucinated incorrectly, answered one correctly and one provision was not found. Of course, it's better to have a LEGAL specific benchmark for this, but 75% hallucination in complex questions is not something that helps your workflow (https://hai.stanford.edu/news/hallucinating-law-legal-mistak...) I don't recommend at least LLMs to anyone for legal document reviews, even for the English language.
I'm not talking about reviewing, only drafting. Every word should be checked. A terrible idea relying on the advice of an LLM.
I have no idea what type of law you're talking about here, but (given the context of the thread) I can guarantee you major firms working on M&As are most definitely not using underpaid interns to draft those documents. They are overpaid qualified solicitors.
Apologies I mean candidate attorneys when I say interns. Those overpaid qualified attorneys, read it and sign off on it.