Run AI Models for Edge Application Development with Just a Few Commands

NVIDIA TensorRT Edge-LLM provides a lightweight C++ inference runtime for embedded workloads, enabling developers to run AI models on the NVIDIA DGX Spark 64GB platform with a few commands using the trtedge CLI. The framework supports deployment on platforms such as NVIDIA Jetson Thor, NVIDIA DRIVE AGX Thor, and NVIDIA IGX Thor, and includes features like DSpark speculative decoding to accelerate generation. A tutorial demonstrates running the NVIDIA Nemotron-3.5 Lightning text-only model on Spark, testing prompts in the terminal, and serving the model through an API for application prototyping.