FreeLLMTogether AI launches Together Compiler, boosting inference speed up to 2.6×
The new agent‑driven compiler re‑optimizes models in under 24 hours, delivering up to 2.6× higher TPS on frontier open‑source LLMs.
Together AI announced the Together Compiler, an agent‑driven loop that continuously re‑optimizes the inference forward pass as traffic patterns change.
In less than 24 hours the system achieved up to 2.6× higher throughput per second (TPS) compared with the best InferenceX configurations on models such as Kimi K3, GLM‑5.3, MiniMax M3.1‑Flash and DeepSeek v4.1‑Flash.
The compiler set new records on the SemiAnalysis InferenceX benchmark, reaching 1,111 TPS on MiniMax M3.1‑Flash and 1,205 TPS on DeepSeek v4.1‑Flash, and outperformed open‑source megakernels by 1.7× on Llama 1B batch‑size‑1 workloads.
Together Compiler’s agents run on persistent engines, eliminating the minutes‑long restarts typical of prior systems and delivering 252× higher experiment throughput.
The platform serves trillions of tokens daily and adapts to diverse hardware (H100, B200, B300, NVL72, Groq, etc.) and traffic shapes, from batch size to prompt length variations.
What it means for you
For AI users, the compiler means faster response times and lower costs when running popular free models such as Kimi K3 and GLM‑5.3, which rank at the top of FreeLLMArena. Developers of local or self‑hosted models can benefit from the continual optimization loop without manual tuning, making it easier to keep performance high as workloads evolve.
On FreeLLMArena
From our rankings on 7 Nov 2026, based on Arena's blind votes:
- Kimi K3 (Max): #1 among free chat models
Sources
- Together AI Releases Together Compiler: Agents Make Inference Up to 2.6x Faster, MarkTechPost, 5 Nov 2026
- Together Compiler: Continual Compilation for AI inference, together.ai (official), 4 Nov 2026
Written automatically from the sources above, then checked by our code: every number and name must appear in them, and no sentence may copy a source. Spotted a mistake? Report it.
Which AI can you use for free today?
Every free AI model, ranked by blind human votes, with the memory and speed it needs on your own GPU.