FreeLLMNVIDIA Snapshot cuts GPU inference startup to seconds, up to 36× faster
Open‑source Snapshot restores GPU workloads in 3.6‑42.7 seconds, delivering 8‑36× speedups over cold starts on tested B200 and HGX systems.
Restoring a snapshot brings an inference replica online in 3.6 to 42.7 seconds, which is 8 to 36 times faster than a cold start that can take minutes.
Creating a checkpoint takes between 15 seconds and 5.5 minutes, and the system can handle checkpoints up to 1 TB across eight GPUs.
Partners reported that GLM‑5.3 pod readiness dropped from roughly eight minutes to 31 seconds on an HGX B300, and vLLM startup fell from two‑8.5 minutes to six‑34 seconds on the same hardware.
Snapshot works only on nodes with the same GPU model and driver; it skips weight loading, compilation and CUDA‑graph capture, but still requires the underlying provisioning and container loading steps.
Ce que cela signifie pour vous
For AI operators, the ability to restart replicas in seconds reduces scaling latency and improves fault tolerance. The tool targets Kubernetes clusters with compatible GPUs, so it does not directly benefit single‑home or CPU‑only setups, but the speed gains may inspire similar checkpoint approaches for free or locally hosted models.
Sources
- Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot, NVIDIA Developer (official), 10 Nov 2026
Rédigé automatiquement à partir des sources ci‑dessus, puis vérifié par notre code : chaque nombre et chaque nom doivent y figurer, et aucune phrase ne doit copier une source. Une erreur ? Signalez‑la.
Quelle IA pouvez‑vous utiliser gratuitement aujourd’hui ?
Tous les modèles IA gratuits, classés par votes humains anonymes, avec la mémoire et la vitesse dont ils ont besoin sur votre propre GPU.