LLM models
Terminal runs a language model on your own machine. Not a hosted one you rent by the token, and not one that sees your positions.
Settings → Models carries the catalogue, the download, and the two assignments.
Reading the catalogue before you commit
The catalogue is curated by MPRO Lab and updates with the application. Every entry tells you what you need to know before a multi-gigabyte download starts:
| Column | What it settles |
|---|---|
| Size on disk | What the file actually costs you |
| RAM / VRAM needed | Whether your machine can load it at all |
| Context window | How much it can hold at once |
| Quantisation | The precision trade-off — Q4_K_M by default, Q8_0 for the small ones |
| Licence | Permissive only, always |
| Good at | One sentence: coding, reasoning, speed, or embedding |
| Evaluation score | How it did on our own harness, not on a public leaderboard |
| SHA-256 | The exact file, so a corrupted download is caught rather than run |
Only permissively licensed models are offered — Qwen3, DeepSeek-R1 distillations, Mistral Small and Devstral, Phi-4, and the embedding models. Families that require accepting a gate are not in the catalogue.
What your machine can run
Terminal detects your memory, your GPU and its VRAM, and your free disk, and makes one recommendation with a one-line reason. You may pick anything else the machine can load; the catalogue marks what it cannot.
| Your machine | Usually recommended |
|---|---|
| Apple Silicon, 16 GB | Qwen3-8B Q4, or Phi-4-mini |
| Apple Silicon, 32 GB | Qwen3-30B-A3B Q4 |
| Apple Silicon, 64 GB or more | Qwen3-32B Q4, or Mistral Small 24B Q8 |
| NVIDIA, 8 GB VRAM | Qwen3-8B Q4, partly offloaded |
| NVIDIA, 12–16 GB | Qwen3-14B Q4 |
| NVIDIA, 24 GB | Qwen3-32B Q4, or Mistral Small 24B Q4 |
| CPU only, 16 GB | Phi-4-mini or Qwen3-4B — and Terminal says it will be slow |
Apple Silicon uses Metal, NVIDIA uses CUDA, and there is a Vulkan and a CPU path. Threads, GPU layers and context length are settings with detected defaults; you do not have to touch them.
Downloading
Downloads come from Hugging Face over HTTPS and are resumable — close Terminal mid-download and it picks up where it stopped. Progress shows on the Models screen and in the status bar, and you can pause it.
Before it starts, Terminal checks you have the disk. When it finishes, it checks the file's SHA-256 against the catalogue and only then puts the file in place; a mismatch deletes the download and tells you. On a metered connection it asks first.
Two assignments
The terminal and the assistant each get their own model, and both default to the recommendation.
If both assignments do not fit in memory at once, switching between the two surfaces swaps them — the status bar says so, with how many seconds it will take. Settings shows you that arithmetic before you assign, so a combination that will thrash is visible as a choice rather than as a slow afternoon.
Hosted models, if you want them
Off by default. You can switch on a hosted provider in Settings with your own API key, kept in the same secret store as everything else.
Two things then change, both visible: the status bar names the backend that is answering, and every turn that left your machine is marked as such in the local record. The harness, the guardrails and the confirmations are identical either way — the only difference is where the computation happened, and you can always see which it was.