Vote in Free Arena

All models › microsoft

Aion-1.0-Instruct-2.3B

microsoft · released 18 Nov 2026

Download New · waiting for Arena votes
Arena score—Not voted on Arena yet
Size2.3 Bparameters
4-bit download1.4 GBweights only, Q4_K_M
Max context32Ktokens, from the model config

Where to use it for free

  • Run it in one command with whichllm (free, open source): uvx whichllm@latest plan "Aion-1.0-Instruct-2.3B" shows the GPU you need, uvx whichllm@latest run "Aion-1.0-Instruct-2.3B gguf" downloads it and starts a chat
  • Download and run it at home: official weights · GGUF files for Ollama and LM Studio

New: waiting for Arena votes

Released on 18 Nov 2026. Arena has not voted on this model yet. As soon as it does, the model moves into the ranking automatically, usually the next night. Until then you can still check its size, memory needs and speed below.

Speed on popular hardware

HardwareGPU memoryFits?Speed
RTX 3060 12 GB12 GBFits in GPU101 tok/s
RTX 4060 Ti 16 GB16 GBFits in GPU82 tok/s
RTX 309024 GBFits in GPU225 tok/s
RTX 409024 GBFits in GPU239 tok/s
RTX 509032 GBFits in GPU358 tok/s
RX 7900 XTX24 GBFits in GPU230 tok/s
2 × RTX 309048 GBFits in GPU219 tok/s
4 × RTX 309096 GBFits in GPU207 tok/s
Mac M4 Max 128 GB96 GBFits in GPU106 tok/s
Ryzen AI Max+ 395 128 GB96 GBFits in GPU74 tok/s
8 × H100 SXM640 GBFits in GPU422 tok/s

Estimate at 4-bit with 32K context, 64 GB of DDR5 for offload. Speed ≈ memory bandwidth × 78% ÷ data read per token, plus a fixed cost per token, calibrated on 44 real llama.cpp tests; typical error about ±10%. Compare all GPUs.

GPU memory needed

Level8K context32K contextQuality vs full
Q3_K_M2.2 GB2.9 GB+10.5% noticeable loss
Q4_K_M2.4 GB3.1 GB+2.8% small loss
Q5_K_M2.6 GB3.4 GB+0.9% very small loss
Q6_K2.9 GB3.6 GB+0.3% near-lossless
Q8_03.5 GB4.2 GB≈ 0% same as full
F165.6 GB6.3 GB≈ 0% same as full

Weights + context cache (from the model's architecture) + runtime. Quality: increase in perplexity against the full model, from llama.cpp tests, scaled by model size. Simulate your own setup.

Share this page:XLinkedInRedditWhatsApp

Press / or Ctrl+K to search from any page.