Ollama on Jetson in 5 Minutes

From a freshly flashed JetPack to running LLMs — without the cloud, without a subscription. Here is the minimal path.

Step 1: Install Ollama

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama

Takes about 2 minutes. Ollama detects the Orin GPU automatically (CUDA ships with JetPack).

Step 2: Pull a model

ollama pull llama3.2:3b    # 2 GB, good starting point
ollama run llama3.2:3b "Explain CUDA grids in three sentences."

Step 3: Test the API

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Name three advantages of local LLMs",
  "stream": false
}'

My hands-on hardware recommendations

• Power supply: at least 45 W USB-C. Weak PSUs are the most common failure source — the Orin throttles mid-inference.
• Cooling: the included fan is fine for desktop loads. For sustained inference: active heatsink, otherwise it throttles within minutes.
• GPU clock > TOPS number. With sudo jtop (pip3 install jetson-stats) you can see live whether it’s throttling.

Next steps on this blog

A measurement series with identical prompts across 5 models (tokens/s, RAM, watts) is in progress. If you want to read it, write me — webmaster@atlas.shrink2core.com.