Mixture of Experts

Mixture of Experts (MOE)

Model divides into specialized “experts”; router activates relevant subset per input.

From 10-AI-Concepts

• Huge params, but efficient (speed/cost).
• Scales intelligence without cost explosion.