granite-switch-4.1-30b: Neuheiten, nötige Hardware, so nutzt du es kostenlos

Granite Switch 4.1 combines chat, decision-making and speech-to-text in a single 30 B model. It lets you switch tasks with a control token, no extra server needed.

Granite Switch 4.1-30B is a modular large language model built by IBM-Granite. The base has thirty billion parameters and can handle up to one hundred twenty-eight thousand tokens in a single prompt. It is released under the Apache 2.0 license, so you can download the weights from Hugging Face and run them locally.

According to the model card, Granite Switch composes a base model with two adapters, decision and guardian-core, plus an audio encoder. You activate an adapter by inserting a special control token in the chat template. The speech-to-text component is a 470 million-parameter non-autoregressive conformer that runs on the same endpoint, so you don’t need a separate ASR server. The decision adapter produces a calibrated probability distribution over option labels in a single forward pass.

The model card reports results on the public Decision Index fast subset, which contains 1,068 rows. The overall score is 50.1 out of 100. Per-area scores are 58.6 for language, 49.8 for retrieval, 76.5 for tools, 32.4 for knowledge, 26.4 for arts, and 58.6 for language again. Hosted and local serving numbers differ by about 0.6 points, confirming the endpoint behaves consistently.

A key strength is the ability to run chat, decision making and speech-to-text on a single vLLM server, simplifying deployment. The speech encoder is the fastest model on the OpenASR leaderboard for English, though it only outputs lower-case, unpunctuated text and is not jointly trained with the LLM. The decision adapter can handle up to 128 option tokens per request; larger sets require splitting into multiple calls.

If you run AI models at home and want a single model that can chat, make structured decisions and transcribe English speech, Granite Switch 4.1-30B is a good fit. It runs on a RTX 3090 with 4-bit quantization, and on higher-end GPUs like RTX 4090 you can reach about 37 tokens per second. Its Apache license lets you experiment without cost.

What do you need to run it at home? At 4-bit it takes about 21 gigabytes of memory. The smallest home machine that holds it fully is an RTX 3090. On an RTX 4090 with 64 gigabytes of RAM it writes about 37 tokens per second. On an RTX 5090 with 64 gigabytes of RAM it writes about 64 tokens per second.

Es gibt noch keine kostenlose API dafür. Die Gewichte sind offen: Du kannst sie von Hugging Face unter der Apache 2.0-Lizenz herunterladen. Arena hat es noch nicht bewertet: Wir fügen die Punktzahl hinzu, sobald die Stimmen eintreffen.

Compare granite switch 4.1 30b with every free model, and check what your own GPU can run, on FreeLLMArena.