FreeLLMNVIDIA Snapshot cuts GPU inference startup to seconds, up to 36× faster
Open‑source Snapshot restores GPU workloads in 3.6‑42.7 seconds, delivering 8‑36× speedups over cold starts on tested B200 and HGX systems.
Restoring a snapshot brings an inference replica online in 3.6 to 42.7 seconds, which is 8 to 36 times faster than a cold start that can take minutes.
Creating a checkpoint takes between 15 seconds and 5.5 minutes, and the system can handle checkpoints up to 1 TB across eight GPUs.
Partners reported that GLM‑5.3 pod readiness dropped from roughly eight minutes to 31 seconds on an HGX B300, and vLLM startup fell from two‑8.5 minutes to six‑34 seconds on the same hardware.
Snapshot works only on nodes with the same GPU model and driver; it skips weight loading, compilation and CUDA‑graph capture, but still requires the underlying provisioning and container loading steps.
Qué significa para ti
For AI operators, the ability to restart replicas in seconds reduces scaling latency and improves fault tolerance. The tool targets Kubernetes clusters with compatible GPUs, so it does not directly benefit single‑home or CPU‑only setups, but the speed gains may inspire similar checkpoint approaches for free or locally hosted models.
Fuentes
- Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot, NVIDIA Developer (official), 10 Nov 2026
Escrito automáticamente a partir de las fuentes anteriores, luego revisado por nuestro código: cada número y nombre debe aparecer en ellas, y ninguna oración puede copiar una fuente. ¿Detectaste un error? Infórmalo.
¿Qué IA puedes usar gratis hoy?
Todos los modelos de IA gratuitos, clasificados por votos humanos ciegos, con la memoria y velocidad que necesita tu GPU.