FreeLLMQwen3.8-27B scores 64.4% on new Co-Design Bench multimodal coding benchmark
The model ranks fourth overall, second on repository editing, and leads text‑only multimodal editing with 77.9% strict completion.
Hugging Face released Co-Design Bench, a benchmark that evaluates coding agents on real user requests involving multimodal inputs and application revisions.
Tasks are split into Multimodal Editing (100 tasks) and Repository Editing (173 tasks), requiring agents to understand intent, edit code, and satisfy detailed criteria.
Across ten model endpoints tested, the strongest, gpt‑6‑astra, passed 69.9% of criteria per task but fully satisfied only 26.4% of tasks.
Qwen3.8‑27B achieved a 64.4% overall strict completion rate, placing fourth overall, and scored 64.5% on Repository Editing (second on that track). It topped the text‑only Multimodal Editing subset with 77.9%.
Was das für dich bedeutet
The benchmark highlights how current coding agents still struggle to fully satisfy complex, multimodal editing requests, even for top‑performing models. Users seeking free or locally run models should note that while Qwen3.8‑27B shows strong text‑only performance, full task completion remains limited, underscoring the need for continued research and more capable open‑source alternatives.
Quellen
- Qwen paper: Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models, Hugging Face Papers (official), 25 Oct 2026
Automatisch aus den obigen Quellen geschrieben, dann von unserem Code geprüft: Jede Zahl und jeder Name muss darin vorkommen, und kein Satz darf eine Quelle kopieren. Einen Fehler entdeckt? Melde ihn.
Welche KI kannst du heute kostenlos nutzen?
Jedes kostenlose KI‑Modell, bewertet durch blinde menschliche Stimmen, mit dem Speicher und der Geschwindigkeit, die es auf deiner eigenen GPU benötigt.