The 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI)
Timestamps are as accurate as they can be but may be slightly off. We encourage you to listen to the full context.
Podcast Summary
Edwin Chen, founder and CEO of Surge AI, shares the extraordinary story of building the fastest company to reach $1 billion in revenue in just four years with fewer than 100 people, completely bootstrapped. (05:05) Surge AI is the leading AI data company that teaches AI models what's good and what's bad, powering training at every frontier AI lab. Edwin discusses his contrarian approach to company building that rejects Silicon Valley's "pivot and blitzscale" playbook, his concerns about AI labs optimizing for the wrong metrics, and why he believes we're still a decade away from AGI.
- Main themes include the critical importance of high-quality data in AI training, the dangers of optimizing AI for engagement rather than truth, and how small elite teams can outperform massive organizations through focus and principled decision-making.
Speakers
Edwin Chen
Edwin is the founder and CEO of Surge AI, the leading AI data company that has reached over $1 billion in revenue with under 100 employees while remaining completely bootstrapped. Before founding Surge, Edwin was a research scientist at Google, Facebook, and Twitter, and studied mathematics, computer science, and linguistics at MIT.
Lenny Rachitsky
Lenny is the host of Lenny's Podcast and author of Lenny's Newsletter, focusing on product management, growth, and building successful companies. He interviews leading entrepreneurs and executives to share insights for ambitious professionals.
Key Takeaways
Build Small Elite Teams Over Large Organizations
Edwin argues that 90% of people at big tech companies could be fired and the company would move faster because the best people wouldn't have distractions. (06:05) Surge succeeded by building a super small, super elite team focused on one mission. This approach allows for higher quality output, better decision-making, and faster execution. The key is hiring people who are genuinely passionate about the work rather than those just looking to add a hot company to their resume.
Quality in AI Data Requires Deep Understanding, Not Just Scale
Most people think you can just throw bodies at a data problem to get good results, but Edwin explains this is completely wrong. (09:55) Using the example of training a model to write poetry about the moon, he illustrates the difference between checking boxes (8 lines, contains "moon") versus creating Nobel Prize-winning poetry with subtle imagery and emotional impact. This requires building sophisticated technology to measure thousands of signals about workers' backgrounds, expertise, and actual performance on tasks.
Taste and Human Judgment Shape AI Model Success
The values and taste of companies building AI models fundamentally shape how those models behave. (28:13) Edwin gives the example of asking Claude to help write an email - do you want a model that continues refining for 50 iterations, or one that says your email is good enough and you should move on? Different companies will make different choices based on their principles, leading to increasingly differentiated AI models rather than commoditized ones.
Reject Silicon Valley's Pivot and Blitzscale Playbook
Edwin advocates for a contrarian approach to company building: don't pivot, don't blitzscale, and don't hire people who just want to add a hot company to their resume. (29:07) Instead, build the one thing only you could build based on your unique insights and expertise. He criticizes founders who pivot from crypto to NFTs to AI, noting there's no consistency or mission - they're just chasing valuations. Taking big risks on something you believe in is better than constantly pivoting for quick gains.
Focus on Real-World Performance Over Academic Benchmarks
Edwin doesn't trust AI benchmarks because they often have wrong answers and are full of messiness that people don't realize. (18:03) More importantly, benchmarks have well-defined objective answers that are easier for models to game compared to real-world ambiguity. While models can win IMO gold medals, they still struggle with parsing PDFs. Surge measures progress through human evaluations where expert annotators have deep conversations with models across different domains rather than relying on superficial benchmark scores.
Statistics & Facts
- Surge AI hit over $1 billion in revenue last year with under 100 people, completely bootstrapped, making them the fastest company in history to reach this milestone. (05:05) Edwin believes we'll see companies with even more extreme ratios like $100 million per employee in the next few years as AI continues improving efficiency.
- Edwin estimates that within the next 1-2 years, AI models will automate 80% of the average L6 software engineer's job, but it will take several more years each to reach 98%, then 99%, then 99.9% performance. (22:40) This is why he believes we're closer to a decade away from AGI rather than just a few years.
- Edwin mentions that GPT-3 launched in 2020, and he started Surge just a month later, recognizing the need for a completely new data solution to train advanced AI models. (54:03) This timing allowed Surge to position itself perfectly for the AI boom that followed.
