FreeLLMQwen3.8-27B scores 64.4% on new Co-Design Bench multimodal coding benchmark
The model ranks fourth overall, second on repository editing, and leads text‑only multimodal editing with 77.9% strict completion.
Hugging Face released Co-Design Bench, a benchmark that evaluates coding agents on real user requests involving multimodal inputs and application revisions.
Tasks are split into Multimodal Editing (100 tasks) and Repository Editing (173 tasks), requiring agents to understand intent, edit code, and satisfy detailed criteria.
Across ten model endpoints tested, the strongest, gpt‑6‑astra, passed 69.9% of criteria per task but fully satisfied only 26.4% of tasks.
Qwen3.8‑27B achieved a 64.4% overall strict completion rate, placing fourth overall, and scored 64.5% on Repository Editing (second on that track). It topped the text‑only Multimodal Editing subset with 77.9%.
O que isso significa para você
The benchmark highlights how current coding agents still struggle to fully satisfy complex, multimodal editing requests, even for top‑performing models. Users seeking free or locally run models should note that while Qwen3.8‑27B shows strong text‑only performance, full task completion remains limited, underscoring the need for continued research and more capable open‑source alternatives.
Fontes
- Qwen paper: Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models, Hugging Face Papers (official), 25 Oct 2026
Escrito automaticamente a partir das fontes acima, depois verificado pelo nosso código: todo número e nome deve aparecer nelas, e nenhuma frase pode copiar uma fonte. Encontrou um erro? Reporte‑o.
Qual IA você pode usar grátis hoje?
Todo modelo de IA gratuito, classificado por votos humanos cegos, com a memória e velocidade que ele precisa na sua própria GPU.