AI & The Law: Changing Practice, Claude Constitution, & New Rights, w/ Kevin & Alan of Scaling Laws
Timestamps are as accurate as they can be but may be slightly off. We encourage you to listen to the full context.
Podcast Summary
In this fascinating deep dive into the intersection of AI and law, Kevin Frazier and Alan Rosenstein explore how artificial intelligence is rapidly transforming both legal practice and the broader legal system. The conversation reveals that frontier AI models are already outperforming the median lawyer in raw intellectual horsepower, while 70% of top law firms have licensed tools like Harvey. (00:42) Despite this adoption, day-to-day usage remains surprisingly low due to billable hour incentives and institutional resistance.
- Main themes: The discussion covers two distinct areas - the law of AI (how we regulate and govern artificial intelligence) and AI in law (how AI is transforming legal practice itself), examining everything from "secret cyborg" lawyers quietly outperforming peers to speculative visions of complete contingent contracts and outcome-oriented legislation.
Speakers
Kevin Frazier
Kevin Frazier is a senior fellow at the Abundance Institute and director of the AI Innovation and Law Program at the University of Texas School of Law. He co-hosts the Scaling Laws podcast and conducts extensive research on bringing practicing lawyers to campus to understand how AI is being adopted in legal practice. (03:43)
Alan Rosenstein
Alan Rosenstein is an associate professor of law at the University of Minnesota and senior editor at Lawfare. He co-hosts the Scaling Laws podcast with Kevin Frazier and has been actively presenting research on AI in legal scholarship to law school faculties across the country. He describes himself as spending "a horrifying" amount of money each month testing various AI models as part of his professional obligation to understand their capabilities. (03:43)
Key Takeaways
AI Models Already Outperform Most Lawyers
Alan Rosenstein makes the striking assertion that frontier AI models like Claude and GPT-4 are "certainly better than the median lawyer" in terms of raw intellectual horsepower. (10:32) While they still face limitations like hallucinations and lack of access to specialized databases, these are "trivially solvable problems" that will be resolved in the coming years. The models excel at pattern matching across standardized contexts, which comprises the vast majority of legal work, similar to programming and medical work.
The Billable Hour Creates Perverse Incentives Against AI Adoption
Despite 70% of major law firms licensing AI tools like Harvey, actual usage remains surprisingly low. (34:53) Kevin Frazier explains this paradox: "The underlying incentive of practicing attorneys is to spend as much time as possible on any given task within the band that's acceptable to your client because we have the billable hour." This creates a fundamental misalignment where efficiency gains from AI actually hurt lawyer compensation, leading to symbolic adoption without real implementation.
Legal Deserts Present Massive Opportunity for AI-Powered Legal Services
Kevin Frazier introduces the concept of "legal deserts" - areas with about one lawyer per 1,000 residents where people lack access to basic legal services like lease reviews, business formation, or divorce proceedings. (20:43) AI could dramatically expand access to legal services for underserved populations, with studies showing that even minimal legal counsel significantly improves outcomes in landlord-tenant disputes. This represents enormous latent demand that AI could help fulfill.
Secret Cyborgs Are Gaining Competitive Advantages
Frazier reports hearing about "secret cyborgs" - lawyers who are AI-savvy but aren't telling their superiors about their sophisticated use of AI tools. (35:38) Meanwhile, firms are beginning to prefer candidates who are "AI whizzes from middle-ranked law schools" over "the number one graduating student from Harvard" with no AI experience. This suggests a significant shift in what skills will be valued in the legal profession going forward.
Complete Contingent Contracts Could Transform Legal Agreements
Alan Rosenstein envisions a future where AI enables "complete contingent contracts" that address every possible scenario before signing. (25:28) Currently impossible due to time and cost constraints, such comprehensive agreements would be socially optimal but require sophisticated AI agents that can negotiate at inference speeds. This could create "orders of magnitude more legal demand" as parties can anticipate and address far more contingencies than humanly possible today.
Statistics & Facts
- 70% of top 100 US law firms are using Harvey according to Harvey's own statistics. (34:53) Kevin Frazier notes this represents symbolic technological adoption, but actual day-to-day usage remains low due to lack of training and billable hour disincentives.
- 95% of all litigation occurs in state courts, not federal courts. (51:59) Kevin Frazier emphasizes this statistic to highlight how most people's experience with the legal system involves notoriously inefficient state court processes where delays can last months or years.
- Legal deserts are defined as areas with about one lawyer for every 1,000 residents. (20:43) Kevin Frazier explains these areas lack adequate access to basic legal services, creating tremendous latent demand that AI-powered tools could potentially address.
