Summary

Jeff Huber from Chroma discusses the importance of measuring and evaluating AI data inputs, particularly in retrieval systems using "fast evals" - a method of creating query and document pairs to systematically test and improve performance. The presentation emphasizes the management principle that "you can only manage what you measure," and introduces a practical approach for AI practitioners to assess their embedding models and chunking strategies. The key takeaway is that systematic improvement becomes easier when you implement straightforward, fast evaluation methods that allow you to understand and optimize your AI system's input retrieval process.