FreeLLMStanford releases Memorilla: 16-token memory module for frozen LLMs
Memorilla compresses whole document collections into 16 tokens, boosting accuracy on benchmarks while cutting token usage by over 100×.
Stanford researchers have open‑sourced Memorilla, a latent memory system that encodes an entire document collection into a fixed set of 16 memory tokens for a frozen LLM.
The module uses a 194‑million‑parameter trainable memory network paired with a frozen Qwen3‑Embedding‑4B encoder and Qwen3‑8B decoder, running on Linux with CUDA GPUs and Python 3.11.
On the FactKG benchmark, a frozen Qwen3‑8B with Memorilla achieved 84.33% accuracy, surpassing the 80.43% score when the model sees every document and using 108× fewer tokens.
Across eight benchmarks Memorilla averaged 49.68 points, the highest reported score, and improved a TextWorld agent’s success rate from 18.6% to 24.2% with just eight memory tokens.
Training required about 19 hours on four NVIDIA H100 80 GB GPUs; inference needs a single 80 GB GPU, with response times of 0.74 s for 512 documents versus 19.9 s for full‑context prompting.
Cosa significa per te
Memorilla lets developers keep frozen, high‑performing LLMs while dramatically reducing context length, making retrieval‑augmented applications faster and cheaper. For users of free or locally hosted models, the approach offers a way to handle massive document stores without scaling hardware or token costs.
Fonti
- Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs, MarkTechPost, 14 Nov 2026
- GitHub - snap-stanford/memorilla: 🦍 Memorilla: Latent Semantic Memory for LLMs · GitHub, github.com (official), 14 Nov 2026
Scritto automaticamente dalle fonti sopra, poi controllato dal nostro codice: ogni numero e nome deve comparire, e nessuna frase può copiare una fonte. Hai trovato un errore? Segnalalo.
Quale AI puoi usare gratis oggi?
Tutti i modelli AI gratuiti, classificati da voti umani anonimi, con la memoria e la velocità necessarie sulla tua GPU.