Can NVIDIA GeForce RTX 3080 10GB run Codestral 22B?
22B parameter Code model on 10GB GDDR6X
Barely — requires CPU/RAM offloading
~1-3 tok/s (offload)
SpeedVery slow — expect 1-3 tokens/sec
QualityQuality is fine but the speed makes it impractical for interactive use
VRAM Requirements
Codestral 22B is a 22B parameter model. At full precision (FP16), it requires 44GB of VRAM. Your NVIDIA GeForce RTX 3080 10GB only has 10GB — not enough even at maximum compression.
FP16 (Full Precision)44GB (need 34GB more)
Maximum quality, no quantization
Q8 (8-bit)22GB (need 12GB more)
Near-lossless, ~50% size reduction
Q4 (4-bit)13GB (need 3GB more)
Good quality, ~75% size reduction
Your GPU VRAM: 10GB GDDR6X at 760 GB/s bandwidth
Recommended system RAM: 32GB DDR5 (2x GPU VRAM minimum for model overflow)
Recommended system RAM: 32GB DDR5 (2x GPU VRAM minimum for model overflow)
How to Set It Up
Step 1: Install Ollama
curl -fsSL https://ollama.com/install.sh | shOllama is the easiest way to run local LLMs. Works on Linux, macOS, and Windows.
Step 2: Download and run Codestral 22B
ollama run codestral:22b:q4_K_MThis downloads the model (~22GB). First run takes a few minutes.
Step 3: Verify GPU is being used
nvidia-smiCheck that VRAM usage increases when the model loads. You should see ~13GB used.
NVIDIA GeForce RTX 3080 10GB Specs
VRAM10GB GDDR6X
Memory Bandwidth760 GB/s
TDP320W
CUDA Cores8,704
Street Price~$450
AI Rating4/10
Other GPUs That Run Codestral 22B
Other Code Models on NVIDIA GeForce RTX 3080 10GB
About Codestral 22B
Top code completion model. Q4 fits on 16GB GPUs.
Category: Code · Parameters: 22B · CUDA required: No (runs via llama.cpp/GGUF)