FreeLLMTogether AI launches Together Compiler, boosting inference speed up to 2.6×
The new agent‑driven compiler re‑optimizes models in under 24 hours, delivering up to 2.6× higher TPS on frontier open‑source LLMs.
Together AI announced the Together Compiler, an agent‑driven loop that continuously re‑optimizes the inference forward pass as traffic patterns change.
In less than 24 hours the system achieved up to 2.6× higher throughput per second (TPS) compared with the best InferenceX configurations on models such as Kimi K3, GLM‑5.3, MiniMax M3.1‑Flash and DeepSeek v4.1‑Flash.
The compiler set new records on the SemiAnalysis InferenceX benchmark, reaching 1,111 TPS on MiniMax M3.1‑Flash and 1,205 TPS on DeepSeek v4.1‑Flash, and outperformed open‑source megakernels by 1.7× on Llama 1B batch‑size‑1 workloads.
Together Compiler’s agents run on persistent engines, eliminating the minutes‑long restarts typical of prior systems and delivering 252× higher experiment throughput.
The platform serves trillions of tokens daily and adapts to diverse hardware (H100, B200, B300, NVL72, Groq, etc.) and traffic shapes, from batch size to prompt length variations.
Ce que cela signifie pour vous
For AI users, the compiler means faster response times and lower costs when running popular free models such as Kimi K3 and GLM‑5.3, which rank at the top of FreeLLMArena. Developers of local or self‑hosted models can benefit from the continual optimization loop without manual tuning, making it easier to keep performance high as workloads evolve.
Sur FreeLLMArena
From our rankings on 7 Nov 2026, based on Arena's blind votes:
- Kimi K3 (Max): #1 among free chat models
Sources
- Together AI Releases Together Compiler: Agents Make Inference Up to 2.6x Faster, MarkTechPost, 5 Nov 2026
- Together Compiler: Continual Compilation for AI inference, together.ai (official), 4 Nov 2026
Rédigé automatiquement à partir des sources ci‑dessus, puis vérifié par notre code : chaque nombre et chaque nom doivent y figurer, et aucune phrase ne doit copier une source. Une erreur ? Signalez‑la.
Quelle IA pouvez‑vous utiliser gratuitement aujourd’hui ?
Tous les modèles IA gratuits, classés par votes humains anonymes, avec la mémoire et la vitesse dont ils ont besoin sur votre propre GPU.