Mixture of Experts (MoE) — AI Glossary

Mixture of experts (MoE) is a neural network architecture that divides a model into many specialized sub-networks called "experts" and uses a routing mechanism to activate only a small subset of them for each input. This design allows models to have trillions of total parameters while only using a fraction during each prediction, dramatically reducing computational cost without sacrificing capability.