🚧 📱

Mobile experience coming soon

Mobile development is in progress. Until it is complete, please use your desktop or laptop.

Thanks!

← Back
AI Engineer June 27, 2025

CI in the Era of AI: From Unit Tests to Stochastic Evals — Nathan Sobo, Zed

Summary

This talk focuses on the empirical testing approach used by Zed, an AI-enabled code editor built from scratch in Rust. The speaker highlights their rigorous, deterministic testing methods, even for complex concurrent scenarios, and discusses the new challenges and the need to embrace stochastic behavior introduced by integrating LLM-powered features like Agentic Editing. The practical takeaway is the necessity of adapting testing strategies to accommodate the inherent non-determinism of AI models to build reliable products.

View original episode ↗