FreeLLMNVIDIA DGX Spark 64GB runs Nemotron‑3.5 Lightning on edge with TensorRT Edge‑LLM
Developers can prototype, fine‑tune and test the Nemotron‑3.5 Lightning 30B model on the DGX Spark using a few CLI commands and serve it locally via an OpenAI‑compatible API.
NVIDIA announced that the DGX Spark 64GB system now supports edge AI development with TensorRT Edge‑LLM, a lightweight C++ inference runtime.
Using the trtedge CLI, developers can download the Nemotron‑3.5 Lightning 30B model (about 21 GB) and build optimized TensorRT engines on the Spark platform.
The runtime also lets users launch a local OpenAI‑compatible API server, enabling rapid application prototyping on the Spark before deployment to Jetson Thor, DRIVE AGX Thor or IGX Thor devices.
Prerequisites include a DGX Spark with CUDA 13, TensorRT 10.16.1.11 or newer, and Python 3.10‑3.12; the tutorial was validated with TensorRT Edge‑LLM 0.11.1.
What it means for you
It also offers a path for users of free or locally hosted models to test and benchmark their applications before moving to production hardware.
Sources
- Run AI Models for Edge Application Development with Just a Few Commands, NVIDIA Developer (official), 30 Oct 2026
Written automatically from the sources above, then checked by our code: every number and name must appear in them, and no sentence may copy a source. Spotted a mistake? Report it.
Which AI can you use for free today?
Every free AI model, ranked by blind human votes, with the memory and speed it needs on your own GPU.