Aion-1.0-Instruct-2.3B: How to Run It Free
Aion-1.0-Instruct-2.3B is a free, on-device 2.3 B parameter model with a 32 K token context. Learn its capabilities, benchmarks, and who can benefit from…
What the video says
Microsoft released a 2.3 billion-parameter model that runs on-device. It can handle a 32 K token context, enough for multi-page documents.
Aion-1.0-Instruct-2.3B is Microsoft’s small language model for the Windows ecosystem. The model runs in PyTorch, ONNX and GGUF formats and is released under the MIT license on Hugging Face. It supports English, German, Spanish, Portuguese, Italian, French, Chinese and Japanese, and is built for on-device text generation tasks like summarization, rewriting and chat.
According to the model card, Aion is the foundational Small Language Model for Windows, optimized for responsive, private AI without constant cloud calls. Positional encoding combines RoPE with YaRN scaling, enabling the long 32 K context. Microsoft released the unquantized version first, promising later quantized releases.
The model card lists several benchmark scores. On the MMLU knowledge test the model scored 55.15 percent, while on the multilingual MMMLU it reached 45.51 percent. In math, GSM8K gave 74.91 percent and GSMPlus 49.56 percent. Instruction following measured by IFEval was 83.90 percent, and translation quality (chrF++) was 43.18 percent. These numbers reflect a small model’s performance compared with larger systems.
Because it is only 2.3 B parameters, Aion loads quickly and fits on modest hardware, a RTX 4060 can hold it fully in GPU memory. Token throughput ranges from 135 tokens per second on an RTX 3060 up to 436 tokens per second on an RTX 5090. The model excels at short-form generation and on-device privacy. However, the card warns that small models often miss fine details, struggle with multi-step reasoning and can generate inaccurate statements. Safety alignment is present but not as robust as larger models.
Aion-1.0-Instruct-2.3B is aimed at people who run AI at home and want a free, open model that works offline. Its long context makes it useful for document summarization, email thread analysis and other multi-page tasks. If you need a model that respects privacy and runs on a mid-range GPU, this is a solid choice. Keep in mind its reasoning limits and verify outputs for critical applications.
What do you need to run it at home? At 4-bit it takes about 4 gigabytes of memory. The smallest home machine that holds it fully is an RTX 4060. On an RTX 3060 12 GB with 32 gigabytes of RAM it writes about 135 tokens per second. On an RTX 4090 with 64 gigabytes of RAM it writes about 303 tokens per second.
There is no free API for it yet. The weights are open: you can download them from Hugging Face under the MIT license. Arena has not ranked it yet: we add the score as soon as the votes arrive.
Compare Aion 1.0 Instruct 2.3B with every free model, and check what your own GPU can run, on FreeLLMArena.
The data
- Size: 2.3B parameters · context 32K · memory at 4-bit about 4 GB.
- RTX 3060 12 GB + 32 GB RAM: about 135 tokens/s
- RTX 4090 + 64 GB RAM: about 303 tokens/s
- RTX 5090 + 64 GB RAM: about 436 tokens/s
- Mac M4 Max 128 GB: about 129 tokens/s
- NVIDIA DGX Spark 128 GB: about 106 tokens/s
- Ryzen AI Max+ 395 128 GB: about 100 tokens/s
- Mac Studio M3 Ultra 512 GB: about 158 tokens/s
- Model card: https://huggingface.co/microsoft/Aion-1.0-Instruct-2.3B
- Every free model, compared: https://freellmarena.com
Hardware
Memory at 4-bit with an 8K context, from our simulator. To check your own setup, use the Run at home simulator.
- Aion-1.0-Instruct-2.3B: about 4 GB at 4-bit · smallest home machine that holds it: RTX 4060 (8 GB) · Amazon ↗
- RTX 3060 12 GB + 32 GB RAM: Aion-1.0-Instruct-2.3B, about 135 tokens/s (fully on the GPU) · Amazon ↗
- RTX 4090 (24 GB) + 64 GB RAM: Aion-1.0-Instruct-2.3B, about 303 tokens/s (fully on the GPU) · Amazon ↗
- RTX 5090 (32 GB) + 64 GB RAM: Aion-1.0-Instruct-2.3B, about 436 tokens/s (fully on the GPU) · Amazon ↗
- Mac M4 Max 128 GB: Aion-1.0-Instruct-2.3B, about 129 tokens/s (fully on the GPU) · Amazon ↗
- NVIDIA DGX Spark 128 GB: Aion-1.0-Instruct-2.3B, about 106 tokens/s (fully on the GPU) · Amazon ↗
- Ryzen AI Max+ 395 128 GB: Aion-1.0-Instruct-2.3B, about 100 tokens/s (fully on the GPU) · Amazon ↗
- Mac Studio M3 Ultra 512 GB: Aion-1.0-Instruct-2.3B, about 158 tokens/s (fully on the GPU) · Amazon ↗
As an Amazon Associate, FreeLLMArena earns from qualifying purchases; the links do not change the numbers.
Every number in this post comes from the FreeLLMArena data on 18 Nov 2026: Arena votes, Hugging Face sizes and our speed simulator. Download the video and its subtitles.
Every free AI model, ranked
Sizes, memory, speed on your GPU and where to use each model for free. Updated every night.