FreeLLMQwen3.8-27B scores 64.4% on new Co-Design Bench multimodal coding benchmark
The model ranks fourth overall, second on repository editing, and leads text‑only multimodal editing with 77.9% strict completion.
Hugging Face released Co-Design Bench, a benchmark that evaluates coding agents on real user requests involving multimodal inputs and application revisions.
Tasks are split into Multimodal Editing (100 tasks) and Repository Editing (173 tasks), requiring agents to understand intent, edit code, and satisfy detailed criteria.
Across ten model endpoints tested, the strongest, gpt‑6‑astra, passed 69.9% of criteria per task but fully satisfied only 26.4% of tasks.
Qwen3.8‑27B achieved a 64.4% overall strict completion rate, placing fourth overall, and scored 64.5% on Repository Editing (second on that track). It topped the text‑only Multimodal Editing subset with 77.9%.
Cosa significa per te
The benchmark highlights how current coding agents still struggle to fully satisfy complex, multimodal editing requests, even for top‑performing models. Users seeking free or locally run models should note that while Qwen3.8‑27B shows strong text‑only performance, full task completion remains limited, underscoring the need for continued research and more capable open‑source alternatives.
Fonti
- Qwen paper: Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models, Hugging Face Papers (official), 25 Oct 2026
Scritto automaticamente dalle fonti sopra, poi controllato dal nostro codice: ogni numero e nome deve comparire, e nessuna frase può copiare una fonte. Hai trovato un errore? Segnalalo.
Quale AI puoi usare gratis oggi?
Tutti i modelli AI gratuiti, classificati da voti umani anonimi, con la memoria e la velocità necessarie sulla tua GPU.