FreeLLMQwen3.8-27B scores 64.4% on new Co-Design Bench multimodal coding benchmark
The model ranks fourth overall, second on repository editing, and leads text‑only multimodal editing with 77.9% strict completion.
Hugging Face released Co-Design Bench, a benchmark that evaluates coding agents on real user requests involving multimodal inputs and application revisions.
Tasks are split into Multimodal Editing (100 tasks) and Repository Editing (173 tasks), requiring agents to understand intent, edit code, and satisfy detailed criteria.
Across ten model endpoints tested, the strongest, gpt‑6‑astra, passed 69.9% of criteria per task but fully satisfied only 26.4% of tasks.
Qwen3.8‑27B achieved a 64.4% overall strict completion rate, placing fourth overall, and scored 64.5% on Repository Editing (second on that track). It topped the text‑only Multimodal Editing subset with 77.9%.
あなたにとっての意味
The benchmark highlights how current coding agents still struggle to fully satisfy complex, multimodal editing requests, even for top‑performing models. Users seeking free or locally run models should note that while Qwen3.8‑27B shows strong text‑only performance, full task completion remains limited, underscoring the need for continued research and more capable open‑source alternatives.
ソース
- Qwen paper: Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models, Hugging Face Papers (official), 25 Oct 2026
上記のソースから自動的に生成し、当社のコードでチェックしました：すべての数値と名前が含まれ、文がソースをコピーしてはいけません。間違いを見つけましたか？報告する。
今日無料で使えるAIはどれですか？
すべての無料AIモデルをブラインドヒューマン投票で順位付けし、あなたのGPUで必要なメモリと速度を示します。