FreeLLMNVIDIA Snapshot cuts GPU inference startup to seconds, up to 36× faster
Open‑source Snapshot restores GPU workloads in 3.6‑42.7 seconds, delivering 8‑36× speedups over cold starts on tested B200 and HGX systems.
Restoring a snapshot brings an inference replica online in 3.6 to 42.7 seconds, which is 8 to 36 times faster than a cold start that can take minutes.
Creating a checkpoint takes between 15 seconds and 5.5 minutes, and the system can handle checkpoints up to 1 TB across eight GPUs.
Partners reported that GLM‑5.3 pod readiness dropped from roughly eight minutes to 31 seconds on an HGX B300, and vLLM startup fell from two‑8.5 minutes to six‑34 seconds on the same hardware.
Snapshot works only on nodes with the same GPU model and driver; it skips weight loading, compilation and CUDA‑graph capture, but still requires the underlying provisioning and container loading steps.
What it means for you
For AI operators, the ability to restart replicas in seconds reduces scaling latency and improves fault tolerance. The tool targets Kubernetes clusters with compatible GPUs, so it does not directly benefit single‑home or CPU‑only setups, but the speed gains may inspire similar checkpoint approaches for free or locally hosted models.
Sources
- Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot, NVIDIA Developer (official), 10 Nov 2026
Written automatically from the sources above, then checked by our code: every number and name must appear in them, and no sentence may copy a source. Spotted a mistake? Report it.
Which AI can you use for free today?
Every free AI model, ranked by blind human votes, with the memory and speed it needs on your own GPU.