FreeLLMNVIDIA Snapshot cuts GPU inference startup to seconds, up to 36× faster
Open‑source Snapshot restores GPU workloads in 3.6‑42.7 seconds, delivering 8‑36× speedups over cold starts on tested B200 and HGX systems.
Restoring a snapshot brings an inference replica online in 3.6 to 42.7 seconds, which is 8 to 36 times faster than a cold start that can take minutes.
Creating a checkpoint takes between 15 seconds and 5.5 minutes, and the system can handle checkpoints up to 1 TB across eight GPUs.
Partners reported that GLM‑5.3 pod readiness dropped from roughly eight minutes to 31 seconds on an HGX B300, and vLLM startup fell from two‑8.5 minutes to six‑34 seconds on the same hardware.
Snapshot works only on nodes with the same GPU model and driver; it skips weight loading, compilation and CUDA‑graph capture, but still requires the underlying provisioning and container loading steps.
Was das für dich bedeutet
For AI operators, the ability to restart replicas in seconds reduces scaling latency and improves fault tolerance. The tool targets Kubernetes clusters with compatible GPUs, so it does not directly benefit single‑home or CPU‑only setups, but the speed gains may inspire similar checkpoint approaches for free or locally hosted models.
Quellen
- Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot, NVIDIA Developer (official), 10 Nov 2026
Automatisch aus den obigen Quellen geschrieben, dann von unserem Code geprüft: Jede Zahl und jeder Name muss darin vorkommen, und kein Satz darf eine Quelle kopieren. Einen Fehler entdeckt? Melde ihn.
Welche KI kannst du heute kostenlos nutzen?
Jedes kostenlose KI‑Modell, bewertet durch blinde menschliche Stimmen, mit dem Speicher und der Geschwindigkeit, die es auf deiner eigenen GPU benötigt.