FreeLLMGPUs › Data center
Best free AI models for the A100 80 GB
A100 80 GB: 80 GB of video memory, 2039 GB/s. These are the free models it runs, ranked by Arena votes, with the speed you can expect.
Voted models that fit fully on the GPU
| Model | Arena score | Size | 4-bit file | Tokens/s |
|---|---|---|---|---|
| Gemma 4 31B |
1443 | 31.3 B | 20 GB | 70 |
| Qwen3.8-27B alibaba |
1441 | 27.8 B | 18 GB | 84 |
| Gemma 4 26B A4B google · MoE, 4 B active |
1434 | 25.8 B | 17 GB | 214 |
| Qwen3.5-122B-A10B alibaba · MoE, 10 B active |
1417 | 125.1 B | 77 GB | 148 |
| Qwen3.5-27B alibaba |
1409 | 27.8 B | 18 GB | 84 |
| glm-4.6v zai · MoE, 14.3 B active |
1376 | 107.7 B | 66 GB | 108 |
| gemma-3-27b-it |
1358 | 27.4 B | 18 GB | 82 |
| glm-4.5v zai · MoE, 14.3 B active |
1333 | 107.7 B | 66 GB | 108 |
Popular small models
Official small models with no Arena votes yet, ordered by downloads on Hugging Face (popularity, not quality).
| Model | Downloads/month | Size | 4-bit file | Tokens/s |
|---|---|---|---|---|
| Qwen3-0.6B Qwen |
29.5M | 0.8 B | 1.7 GB | 528 |
| Qwen3-4B Qwen |
11.8M | 4 B | 3.6 GB | 305 |
| Qwen3-8B Qwen |
8.9M | 8.2 B | 6.2 GB | 205 |
| Qwen3-14B Qwen |
8.4M | 14.8 B | 10 GB | 134 |
| Qwen3.5-4B Qwen |
7.8M | 4.7 B | 4.0 GB | 338 |
| Qwen3.5-9B Qwen |
7.5M | 9.7 B | 7.1 GB | 206 |
Estimates at 4-bit (Q4_K_M) with an 8K context, calibrated on 44 real llama.cpp tests (typical error about ±10%). Tokens per second: about 10 is reading speed.
Questions
What is the best free AI model for the A100 80 GB?
Gemma 4 31B (Arena score 1443) is the highest-voted open model that fits entirely in 80 GB of video memory at 4-bit, writing about 70 tokens per second.
How big a model can the A100 80 GB run?
About 123 billion parameters for a dense model at 4-bit with an 8K context, fully on the GPU. Mixture-of-experts models can be bigger, and with 64 GB of system RAM larger models still run, more slowly.
Is the A100 80 GB good for local LLMs?
It has 80 GB of memory and 2039 GB/s of bandwidth. Memory decides which models fit; bandwidth decides how fast they write. With 24 GB or more it runs 27–32 B models, the sweet spot of quality for home use.
Similar cards
- H100 SXM 80 GB · 3350 GB/s
- RTX PRO 6000 Blackwell 96 GB · 1792 GB/s
- Mac mini M4 Pro 64 GB 64 GB · 273 GB/s
- RTX 6000 Ada 48 GB · 960 GB/s
- RTX A6000 48 GB · 768 GB/s
- RTX 5090 32 GB · 1792 GB/s