How Large a Model Can Your Computer Run

The last row, Qwen3.8-Max, stays red no matter what you pick. That is not a bug in the list. It is the model from the third lesson of this chapter whose weights have been announced as coming but have not been released yet: run 2.4 trillion parameters through the formula above and you need 1560 GB, more than even the most expensive machine on the list can offer. It stays in the table because "the weights are open" and "you can run it" are two different things, and seeing that number lands better than reading the words "very large."

The result above is not looked up in a table. It is one multiplication:

The factor is the part that needs explaining. To load the weights alone, INT4 takes 0.5 bytes per parameter, so an 8B model needs only 4 GB. But a running model also needs extra room to hold the intermediate state that piles up during a conversation, the KV Cache covered in Part 2. That overhead is already baked into the factor, so do not multiply in another safety margin on top of it, or you will end up concluding that no machine can run anything.

Quantization means storing each parameter with fewer bits. Fewer bits means a smaller footprint, and the price is lost precision.

Switch the precision from FP16 to INT4 in the calculator above and watch which models become runnable. Quantization is the single most effective way to lower the bar for local deployment: an 8 GB card cannot handle an 8B model at FP16, but at INT4 it has room to spare.

Everyone can recite the line "fewer bits means some loss," but at which step does the loss actually happen? Quantization does exactly one thing: it rounds weights that used to be continuous onto a finite set of levels. The bit width decides how many levels there are to work with — 4 bits gives you 16, 8 bits gives you 256. The fewer the levels, the further each weight has to move to reach one.

Two things are worth pausing on at INT4. First, the weights with the smallest magnitudes round straight onto 0, so those parameters stop doing anything at all in the quantized file; a model holds an enormous number of such small weights, and while none of them matters on its own, together they carry a lot of the fine detail. Second, there are fewer dots on the ruler — no dot went missing, several weights that used to be different got squeezed onto the same level. Sixteen levels cannot hold that many distinct values, so those weights have to share one number, and whatever set them apart is gone.

Why should nudging the weights a little affect the quality of an answer? Because every word the model writes is the highest-scoring option out of a whole pool of candidates. Most of the time the front-runner is far enough ahead that a nudge changes nothing, but every so often the top two are almost tied, and then the smallest perturbation is enough to swap them.

Put simply: NVIDIA competes on speed, Apple on capacity. If you want to run models above 30B, a Mac with plenty of memory is often more realistic than a consumer GPU; if you are after response speed, a discrete GPU suits you better.

Entries in the results list with an A in the name are MoE models, such as Qwen3-30B-A3B, meaning 30B parameters in total with 3B actually activated each pass. These come with a trap that is easy to fall into:

That makes MoE a good fit when VRAM is plentiful but you want quick responses — a high-memory Mac, for example. The reverse also holds: if VRAM is tight, a smaller dense model gives you more for the same footprint.

Every number above is an estimate. Actual usage is also shaped by these factors:

So the calculator gives you a feasibility judgment, not a precise budget. When the verdict is "tight," plan as if it will not run.

Now that you know what you can run, the next lesson gets it installed. Two tools — Ollama on the command line and LM Studio with a GUI — and ten minutes to a working setup.