⚡ Inside Google Labs: The AI Coding Agent You Haven't Heard About — Jed Borovik, Google
Timestamps are as accurate as they can be but may be slightly off. We encourage you to listen to the full context.
Podcast Summary
In this episode of Latent Space, Jed Borovik, Product Lead at Google Labs, shares his journey from working on Google Search to spearheading Jules, one of the most ambitious autonomous coding agent projects in tech. (02:47) Borovik discusses how discovering Stable Diffusion sparked his transition into AI, leading him to view AI coding tools as "a new brush for software engineering" rather than a replacement for developers. The conversation explores Jules' unique approach to autonomous coding agents that run on their own infrastructure, (06:41) allowing developers to deploy agents for hours or even days at a time. We dive deep into Google Labs' mission at the intersection of DeepMind's model development and product innovation, the evolution from complex agent scaffolding to simpler approaches as models improve, and the shift from embeddings-based RAG to attention-based code understanding.
- Main themes: The evolution of coding agents from complex scaffolding to simpler approaches, the transition from RAG-based to attention-based code understanding, and Google's positioning in building autonomous AI-powered development tools for the future of software engineering.
Speakers
Jed Borovik
Jed Borovik is a Product Lead at Google Labs, where he spearheads the development of Jules, Google's autonomous coding agent platform. With nine years at Google, he previously worked on Google Search, focusing on news and freshness algorithms. Borovik studied at NYU and has been deeply involved in the New York tech scene, including organizing hackathons and serving as an MC for AI Engineer events. His journey into AI began with Stable Diffusion, which inspired his transition from search engineering to building the future of AI-powered software development.
Key Takeaways
Agent Scaffolding Simplifies as Models Improve
Borovik reveals a counterintuitive trend in AI agent development: as models become more capable, the complex scaffolding around them becomes unnecessary. (09:49) When Google Labs started building Jules with early Gemini models, they implemented incredibly complex agent systems with multiple sub-agents for different tasks - code editors, reviewers, and specialized personas. However, as model quality improved, this elaborate scaffolding became more of a "crutch" than a helpful tool. The lesson for AI engineers is to resist over-engineering agent frameworks and instead focus on building for where models are heading, not where they currently are. This approach allows for easier maintenance, better performance improvements, and more reliable systems as the underlying models continue to evolve.
RAG is Giving Way to Attention-Based Search
One of the most significant technical shifts Borovik discusses is moving away from traditional embeddings-based RAG (Retrieval-Augmented Generation) toward attention-based search mechanisms. (11:30) The fundamental problem with RAG is that chunks with arbitrary boundaries will inevitably miss important context that spans those boundaries. While embeddings-based retrieval was the standard approach when building coding agents, Jules has found that simply "throwing attention at it" with tools like grep proves more effective and scalable. This insight suggests that maintaining complex embedding systems and chunk optimization may be unnecessary overhead as context windows expand and attention mechanisms improve.
Autonomous Means Having Its Own Infrastructure
Jules differentiates itself by running on dedicated infrastructure rather than locally or within developer workspaces. (06:41) This architectural decision enables truly long-running agents that can work for hours or days without interrupting the developer's workflow. Borovik shares stories of users running single Jules sessions for over thirty days - something that surprised even the Google Labs team. The practical implication is that for coding agents to reach their full potential, they need their own compute environment, APIs for integration, and the ability to operate independently. This approach enables use cases like automatic code updates triggered by external events and seamless integration with CI/CD pipelines.
Context Management is the New Frontier
Managing context windows approaching 2 million tokens presents unique challenges that coding agents push to the limit. (25:04) Unlike other AI applications, coding agents consume massive amounts of context through file reading, command outputs, and extended conversations. Borovik emphasizes that coding agents are special in requiring both extensive context and long-running sessions, making them an ideal testing ground for context compression and management techniques. Teams building coding agents should invest heavily in context management strategies, whether through summarization, sub-agent handoffs, or external memory systems, as this capability often determines the agent's practical utility.
Software Engineering Will Become More Strategic
Rather than replacing software engineers, Borovik argues that AI coding tools will shift the profession toward higher-value work. (32:54) Using the analogy of electricity versus farming, he suggests software has elastic demand - as development becomes more efficient, demand for software increases rather than decreases. The key insight is that commoditized coding work can be delegated to agents, freeing engineers to focus on strategic, novel, and deep problem-solving work. Engineers should position themselves for this transition by developing skills in specification, verification, and architectural thinking rather than trying to compete with AI on routine coding tasks.
Statistics & Facts
- Jules sessions can run for up to 30 days, which initially surprised the Google Labs team who only designed their data storage to last 30 days before sessions become locked. (25:05)
- Jules can handle context windows of up to 2 million tokens, pushing the limits of what's possible with current language models for coding applications. (25:32)
- The AI Engineer Code Summit has a highly selective acceptance rate of approximately 1 in 23 applicants, making it one of the most exclusive AI engineering conferences. (19:29)
