From Mixture of Experts to Mixture of Agents with Super Fast Inference - Daniel Kim & Daria Soboleva
Summary
This workshop explores the concept of Mixture of Experts (MoE) architectures for improving LLMs, referencing its role in scaling models like ChatGPT. The practical takeaway is to build a "Mixture of Agents" by replacing experts with agents, culminating in a hands-on development session.