FreeLLMNVIDIA DGX Spark 64GB runs Nemotron‑3.5 Lightning on edge with TensorRT Edge‑LLM
Developers can prototype, fine‑tune and test the Nemotron‑3.5 Lightning 30B model on the DGX Spark using a few CLI commands and serve it locally via an OpenAI‑compatible API.
NVIDIA announced that the DGX Spark 64GB system now supports edge AI development with TensorRT Edge‑LLM, a lightweight C++ inference runtime.
Using the trtedge CLI, developers can download the Nemotron‑3.5 Lightning 30B model (about 21 GB) and build optimized TensorRT engines on the Spark platform.
The runtime also lets users launch a local OpenAI‑compatible API server, enabling rapid application prototyping on the Spark before deployment to Jetson Thor, DRIVE AGX Thor or IGX Thor devices.
Prerequisites include a DGX Spark with CUDA 13, TensorRT 10.16.1.11 or newer, and Python 3.10‑3.12; the tutorial was validated with TensorRT Edge‑LLM 0.11.1.
O que isso significa para você
It also offers a path for users of free or locally hosted models to test and benchmark their applications before moving to production hardware.
Fontes
- Run AI Models for Edge Application Development with Just a Few Commands, NVIDIA Developer (official), 30 Oct 2026
Escrito automaticamente a partir das fontes acima, depois verificado pelo nosso código: todo número e nome deve aparecer nelas, e nenhuma frase pode copiar uma fonte. Encontrou um erro? Reporte‑o.
Qual IA você pode usar grátis hoje?
Todo modelo de IA gratuito, classificado por votos humanos cegos, com a memória e velocidade que ele precisa na sua própria GPU.