FreeLLMPrime Intellect unveils Prime Mega MoE, a trainable fused MoE kernel with 3.11× faster forward pass
The new Prime Mega MoE kernel speeds Qwen3 MoE training by up to 34% and adds backward‑pass support for DeepSeek’s Mega MoE.
Prime Intellect announced Prime Mega MoE, a trainable version of DeepSeek’s fused Mega MoE kernel.
The kernel delivers a 3.11× faster forward pass and improves Qwen3 MoE training steps by up to 34% in the prime‑rl benchmark.
Prime Mega MoE adds backward‑pass capability, enabling full training of models that use the Mega MoE architecture.
The release follows ongoing work in DeepSeek’s DeepGEMM library, which provides high‑performance GPU kernels for GEMM, fused MoE, and other primitives used in large language models.
DeepGEMM’s recent updates include Mega MoE kernels, FP8/FP4 GEMM support, and optimizations for NVIDIA SM90/SM100 GPUs.
What it means for you
For users of free or locally hosted models, the underlying DeepGEMM library offers a lightweight, open‑source path to high‑performance GPU kernels without proprietary dependencies.
Sources
- Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass, MarkTechPost, 29 Oct 2026
- GitHub - deepseek-ai/DeepGEMM: DeepGEMM: clean and efficient BLAS kernel library on GPU · GitHub, github.com (official), 29 Oct 2026
Written automatically from the sources above, then checked by our code: every number and name must appear in them, and no sentence may copy a source. Spotted a mistake? Report it.
Which AI can you use for free today?
Every free AI model, ranked by blind human votes, with the memory and speed it needs on your own GPU.