Law Professors Favor AI Responses Over Peers in Legal Reasoning Study

Researchers from Stanford University conducted a study involving 16 law professors from 14 U.S. law schools, including prominent institutions like Stanford and Yale. The study examined their preferences in legal reasoning by comparing AI-generated responses to those created by their peers. Professors generated a total of 40 contract law questions, allowing for a thorough evaluation of how modern large language models (LLMs) perform in legal contexts.

The results showed a significant preference for AI answers, with Google’s Gemini 2.5 Pro winning 75.92% of matchups against human-written responses, and NotebookLM winning 74.75% of the time. This indicates that AI-generated responses were favored in about three-quarters of the evaluations, suggesting a potential alignment with established professional standards.

The researchers noted that the findings challenge traditional views on AI's capabilities in educational settings. They expressed, “Large language models (LLMs) are increasingly promoted as educational tutors, yet most evaluations focus on domains with a single ground truth.” The legal field, which often requires nuanced judgment and sound reasoning, offers a distinctive environment to measure AI performance.

Notably, the AI models demonstrated superior performance across various categories, including recall questions related to case law and policy discussions. Such results compel educators to reconsider the integration of AI within legal studies and encourage a dialogue about its implications for the future of legal education.

As AI continues to evolve, the legal community may need to monitor how these technologies impact learning, teaching methodologies, and professional evaluations. The question of whether AI can truly enhance, rather than replace, human judgment within law remains vital as the landscape of legal education transforms.