FreeLLMNVIDIA DGX Spark 64GB runs Nemotron‑3.5 Lightning on edge with TensorRT Edge‑LLM
Developers can prototype, fine‑tune and test the Nemotron‑3.5 Lightning 30B model on the DGX Spark using a few CLI commands and serve it locally via an OpenAI‑compatible API.
NVIDIA announced that the DGX Spark 64GB system now supports edge AI development with TensorRT Edge‑LLM, a lightweight C++ inference runtime.
Using the trtedge CLI, developers can download the Nemotron‑3.5 Lightning 30B model (about 21 GB) and build optimized TensorRT engines on the Spark platform.
The runtime also lets users launch a local OpenAI‑compatible API server, enabling rapid application prototyping on the Spark before deployment to Jetson Thor, DRIVE AGX Thor or IGX Thor devices.
Prerequisites include a DGX Spark with CUDA 13, TensorRT 10.16.1.11 or newer, and Python 3.10‑3.12; the tutorial was validated with TensorRT Edge‑LLM 0.11.1.
Cosa significa per te
It also offers a path for users of free or locally hosted models to test and benchmark their applications before moving to production hardware.
Fonti
- Run AI Models for Edge Application Development with Just a Few Commands, NVIDIA Developer (official), 30 Oct 2026
Scritto automaticamente dalle fonti sopra, poi controllato dal nostro codice: ogni numero e nome deve comparire, e nessuna frase può copiare una fonte. Hai trovato un errore? Segnalalo.
Quale AI puoi usare gratis oggi?
Tutti i modelli AI gratuiti, classificati da voti umani anonimi, con la memoria e la velocità necessarie sulla tua GPU.