At AI Summit, Attendees Ponder Potential, Limits of AI in Legal Space
September 19, 2024
Approximately 50 lawyers, judges, law professors, and technology experts gathered at the D.C. Bar headquarters on September 16 for the Bar’s second annual AI Summit, exploring how the legal profession can maximize use of AI technology while recognizing the ethical issues surrounding it.
Hosted by D.C. Bar President Shaun Snyder, the summit began with a demonstration of the use of AI legal workflow and research platform Vincent AI by Ed Walters, chief strategy officer of vLex and cofounder of Fastcase.
During his presentation, Walters posed a hypothetical query on Vincent AI, a product of vLex: “Can the defendant be prosecuted for a felony of murder when the victim was killed as an accomplice during the course of a robbery?” In response, Vincent AI produced a list of summarized answers with hyperlinks to actual case documents as well as direct quotes from the cases.
Walters also demonstrated the retrieval-augmented generation (RAG) process to optimize output and enhance accuracy and reliability. The RAG process breaks down the research task into two parts: (1) retrieving the documents that are likely to help answer the question, and then (2) tasking the AI platform to answer the question using only those retrieved documents. With the RAG technique, the platform produces answers grounded in authoritative law, Walters said.
For lawyers to get the best use out of AI platforms, Walters said they should be “intelligent consumer[s],” which translates to not trusting artificial intelligence completely. “The two most important answers for legal generative AI to be able to give are (1) I don’t know, and (2) the law is not what you wish it to be. And if your AI tool can’t do those two things, I have a problem with using it,” he added.
Training for the Future
But how should the legal profession go about training lawyers and law students on generative AI tools? One attendee said that with so much information coming out of the AI space, legal professionals want quick, interactive, on-demand training. What they often receive, however, are long PowerPoint presentations and CLE programs on AI.
Three planks of generative AI training could be useful for lawyers: experimental learning, on-demand training, and a curriculum that covers the bases and supervision of generative AI, as well as the business and legal information regarding the technology.
The summit also addressed the technological gap between young practitioners and senior lawyers, the latter often being late adopters of new technology. Senior lawyers supervising young attorneys using generative AI tools need to understand the technology to properly guide them, one attendee recommended, adding that she would like to see more CLEs on emerging technology for senior lawyers.
Clients have mixed opinions so far on lawyers using generative AI, possibly driven by concerns about confidentiality, data security, and billing. Usage of generative AI technology is not necessarily going to reduce billing hours, one participant pointed out. Instead, AI is going to cause those billable hours to be reallocated.
Despite the technology’s flaws, it appears that the legal profession is beginning to embrace AI, with three in four lawyers planning to use it in their work, according to a Wolters Kluwer 2023 Future Ready Lawyer Report.
Use With Caution
The AI Summit also explored opportunities and obstacles associated with AI use in law schools. A law professor who specializes in technology, access to justice, and legal ethics issues mentioned the recent preview release of Open AI o1, a new series of AI models. Built to take more time thinking, Open AI o1 could tackle complex problems in math and science, and even challenge hypotheses and complete New York Times crossword puzzles.
To test OpenAI o1’s legal reasoning and analysis skills, the professor said she used one of her evidence quizzes involving character and habits. The result: OpenAI o1 took approximately three minutes to answer the 10-question quiz, producing correct answers with citations.
Beginning in their first year of law school, students should learn how to use generative AI tools, spot their limits, and understand the underlying ethics issues, the professor said. However, law schools should double down on traditional and fundamental legal skills such as research, writing, and analysis, she added.
Another law professor stressed the need for transparency with law students about what generative AI is, how it can and cannot be used in law school, why student policies on AI usage exist, and how those policies will lead them to career success.
While there is still no concrete consensus among law schools on how to integrate generative AI into the classroom, there is general agreement that students need to develop “lawyer intelligence” first before they can effectively use artificial intelligence.
AI instruction and usage should be restricted during students’ first year in law school so they can better learn how to think like traditional lawyers, the professor said, warning that if students delve into generative AI for legal writing too early and without structure, they risk learning to depend too much on the technology.
Generative AI was described as an auto-predictive tool, akin to the one used in text messaging, that strives to produce consistently average outputs to please its users but often get things wrong. In the hands of a novice, AI will generally produce a low-C or D level work, but when used by a skilled professional, it has incredible potential, the professor said.
Snyder doesn’t see an endpoint in learning generative AI. “There’s so much opportunity here,” said Snyder, who has identified the future of the profession as one of his areas of focus during his term. “There’s a lot of good that it can do; there’s a lot that it can help the [legal] profession with.”
“But the question is, how do we make sure our [Bar] members understand that. This will be a continuing discussion,” Snyder added.
