← Back to library

▶ quest log

Mixture of Experts: Why a Model Card Shows Two Numbers

AI ExplainedBeginner@romippatelsaved AUG 29

◇ Could this help me?

Yes. The explanation is accurate. Mixture of experts does route each token through a learned router to a small subset of experts, and the stored-versus-active split is exactly what a card like 35B A3B encodes. It is worth internalizing because it corrects the common assumption that a big parameter count means a big serving cost, and it names the memory caveat that most short explanations leave out. This is a general practice.

✦ Walkthrough+10 xp / step
0 / 7 steps · adopt for +40
$ suggested_prompt+40xp

I am choosing between a dense model and a mixture of experts model for a workload I will describe. For each candidate I name, tell me the total stored parameters, the active parameters per token, the memory needed to load the full model, and what that implies for my hardware and cost per token. Flag any case where the mixture of experts option would not actually be cheaper for me in practice.

More from AI Explained

Enter world →