The main reason to buy a Jetson: running LLMs without the cloud. Here are my measured numbers, no marketing figures.
Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b # small model to get startedWhat actually runs
Quantized models (Q4/Q8 GGUF) work best. The most important rule from practice: GPU clock > TOPS number — a model running at a high clock delivers fluent answers where a lower-clocked chip stutters.
Benchmarks are coming
This blog measures afterwards: tokens/s, RAM usage, power draw in watts — observed with jtop, on the real device. Follow-up articles with full measurement series are in the works.