FreeLLMNVIDIA DGX Spark 64GB runs Nemotron‑3.5 Lightning on edge with TensorRT Edge‑LLM
Developers can prototype, fine‑tune and test the Nemotron‑3.5 Lightning 30B model on the DGX Spark using a few CLI commands and serve it locally via an OpenAI‑compatible API.
NVIDIA announced that the DGX Spark 64GB system now supports edge AI development with TensorRT Edge‑LLM, a lightweight C++ inference runtime.
Using the trtedge CLI, developers can download the Nemotron‑3.5 Lightning 30B model (about 21 GB) and build optimized TensorRT engines on the Spark platform.
The runtime also lets users launch a local OpenAI‑compatible API server, enabling rapid application prototyping on the Spark before deployment to Jetson Thor, DRIVE AGX Thor or IGX Thor devices.
Prerequisites include a DGX Spark with CUDA 13, TensorRT 10.16.1.11 or newer, and Python 3.10‑3.12; the tutorial was validated with TensorRT Edge‑LLM 0.11.1.
Ce que cela signifie pour vous
It also offers a path for users of free or locally hosted models to test and benchmark their applications before moving to production hardware.
Sources
- Run AI Models for Edge Application Development with Just a Few Commands, NVIDIA Developer (official), 30 Oct 2026
Rédigé automatiquement à partir des sources ci‑dessus, puis vérifié par notre code : chaque nombre et chaque nom doivent y figurer, et aucune phrase ne doit copier une source. Une erreur ? Signalez‑la.
Quelle IA pouvez‑vous utiliser gratuitement aujourd’hui ?
Tous les modèles IA gratuits, classés par votes humains anonymes, avec la mémoire et la vitesse dont ils ont besoin sur votre propre GPU.