How Much VRAM You Need for a Local AI Model: GPU and Memory Guide
How much VRAM you actually need for a local 7B, 14B, or 30B model, whether a GPU is mandatory, and whether regular system RAM is enough. A practical breakdown, no marketing.
Before running a local model, almost everyone hits the same question: will my hardware handle it, and do I even need a graphics card. The answer isn't as scary as forum threads that push you to buy an expensive card make it sound. Let's go through it honestly: how much memory different models need, and when you can skip the GPU entirely.
What it all comes down to
What matters for a local model isn't so much hardware "power" as the memory it fits into. The whole model loads into memory and runs there. If it fits, it works; if it doesn't, it either won't start or will crawl painfully.
Memory comes in two kinds, and both work. Video memory (VRAM) on a graphics card is the fast option. Regular system memory (RAM) is slower, but still workable. More on that below — numbers first.
How much memory for which model
Rough guidelines for popular sizes at a convenient quantization (there's a separate breakdown of what quantization and Q4_K_M mean). The numbers are approximate — they depend on the specific model and version:
| Model size | Memory needed (Q4_K_M) | What it's good for |
|---|---|---|
| ~7–8B | 6–8 GB | Everyday tasks, chat, working with documents |
| ~13–14B | 10–12 GB | More complex tasks, code |
| ~30B+ | 20 GB and up | Close to cloud quality, but needs serious hardware |
Practical takeaway: a 7–8B model runs on almost any modern laptop, while a 30B one already needs a strong GPU or a lot of RAM.
Do you even need a graphics card
A common myth is that local AI won't run without a discrete GPU. Not true. A model can run on a regular CPU, using system RAM. The difference is speed: on a GPU, answers come faster; on a CPU, slower — but it works.
So you don't have to buy anything to get started. With 16 GB of RAM, a mid-sized model will run fine without a GPU — just not instantly. Want it faster and bigger? That's when you start looking at a graphics card with enough VRAM.
Don't rush to buy hardware "for AI." Run a local model first on what you already have, and see if the speed works for you. Often the laptop you already own is enough for everyday tasks, and an upgrade only makes sense if you want to run large models fast.
What to look for when choosing a GPU
If you've decided to buy a card specifically for local models, the main spec is VRAM size, not abstract gaming performance. The model has to fit into VRAM whole, so 12 GB on a card is more useful for this than a faster card with 8 GB. The more VRAM, the bigger a model you can run at a comfortable speed.
How Doka makes this easier
To skip the guesswork, Doka checks your hardware itself on first launch and picks a model that's sure to fit. It also decides whether to run on the GPU or the CPU. So you can get started without knowing any specs at all: the app picks something sensible, and you can switch to a bigger model later if you want.
Want to check before installing anything? There's a model picker calculator: pick your GPU from the list or set your memory with a slider, and it names a model that fits right away.
The bottom line on hardware
For a local model, what matters most is the memory it fits into: 6–8 GB for a small one, 20+ for a large one. A GPU speeds things up but isn't required — it also runs on a CPU and system RAM, just slower. Start with the hardware you already have, and plan an upgrade based on what you actually need. Download Doka for free, and it'll pick the model for you.