Documentation

onpremo inspect <model.gguf>

Inspect GGUF metadata without loading weights.

--json emit machine-readable JSON instead of a table

onpremo inspect smollm2-135m-instruct-q4_k_m.gguf
onpremo inspect smollm2-135m-instruct-q4_k_m.gguf --json

onpremo check <model.gguf>

Test model compatibility with a device profile without loading the model.

--profile <name|path> built-in name or TOML path (default: pi-1gb-safe)
--json emit machine-readable admission outcome

Exit codes: 0 admitted or downgraded, 2 rejected, 1 for bad paths or invalid profiles.

onpremo check model.gguf --profile micro-500mb
onpremo check model.gguf --profile pi-1gb-safe --json

onpremo run <model.gguf>

One-shot inference with streaming tokens on stdout and immediate cleanup.

--profile <name|path> built-in name or TOML path; see onpremo profile suggest for a host-fit profile (default: pi-1gb-safe)
--prompt <text> required prompt string
--max-tokens <n> default: 64
--temperature <f> default: 0.7
--seed <u64> optional RNG seed for reproducible sampling
--report print machine-readable run report JSON to stderr
--chat treat --prompt as a single user chat message (model chat template, or a role-prefixed fallback)
--system <text> system message prepended when --chat is set (requires --chat)
--json-schema <path> JSON Schema file compiled to a GBNF grammar for constrained generation; output is validated after generation
--lora <path> optional LoRA adapter file (file size counted in admission)

Exit codes: 0 admitted or downgraded, 2 rejected, 3 schema violation (with --json-schema), 1 other errors (bad paths, invalid profiles, load failures).

onpremo run model.gguf \
  --profile pi-1gb-safe \
  --prompt "Return the intent as JSON" \
  --max-tokens 32

onpremo run model.gguf \
  --profile pi-1gb-safe \
  --prompt "hello world" \
  --max-tokens 4 \
  --report 2>report.json

onpremo run model.gguf \
  --profile pi-1gb-safe \
  --chat --system "Reply in one sentence" \
  --prompt "What is admission?" \
  --json-schema schema.json

onpremo chat <model.gguf>

Interactive multi-turn terminal chat. Loads the model once; streams each turn. In-REPL: /exit quits, /clear resets history (keeps the system message). EOF / Ctrl-D also exits cleanly.

--profile <name|path> default: pi-1gb-safe (Pi certification identity; on a laptop prefer the profile recommend printed)
--system <text> optional system message kept across /clear; with --agent, appended after the manifest system
--agent <path> agent manifest TOML (master prompt + tool registry). Tool calls print for the operator to type a result at tool result>, except the implicit escalate tool which is auto-executed
--max-tokens <n> per turn (overrides agent default; default 128)
--temperature <f> overrides agent default; default 0.7
--seed <u64> optional
--lora <path> optional LoRA adapter (overrides manifest lora = "…")

onpremo chat model.gguf --profile pi-1gb-safe
onpremo chat model.gguf --profile pi-1gb-safe --system "Be brief"
onpremo chat model.gguf --agent examples/agents/workshop-gateway.toml

onpremo benchmark <model.gguf>

Repeatable workload; writes a machine-readable JSON report (schema version 1).

--profile <name|path> default: pi-1gb-safe
--context <n> optional context override
--batch <n> optional batch override
--max-tokens <n> default: 64
--iterations <n> default: 3
--warmup <n> default: 1
--prompt <text> default: Summarize: the pump pressure is low.
--output <path> write JSON here (default: stdout)

onpremo benchmark model.gguf \
  --profile pi-1gb-safe \
  --context 512 \
  --batch 16 \
  --max-tokens 64 \
  --iterations 3 \
  --warmup 1 \
  --output result.json

onpremo serve <model.gguf>

Bounded local HTTP API. Memory budget is owned by profiles + admission (there is no free-form --memory-limit flag). Binds loopback by default. On start, serve prints both GUI URLs on stderr:

onpremo serve: listening on 127.0.0.1:11500 (profile=…, backend=…)
  chat:   http://127.0.0.1:11500/
  status: http://127.0.0.1:11500/status

Web chat at / is the primary GUI (embedded browser UI). /status is the secondary operator page (model, admitted config, live memory). Both are always printed so discovery does not require reading docs.

--profile <name|path> default: pi-1gb-safe
--bind <addr> default: 127.0.0.1:11500
--idle-unload <secs> unload after idle seconds; 0 disables (default: 30)
--max-request-bytes <n> hard body ceiling (default: 65536)
--agent <path> agent manifest applied to every conversation
--lora <path> refused: the server load path does not apply LoRA in this build (use run/chat)
--no-tray skip the menu bar / system tray icon (macOS, Windows, and Linux builds made with --features tray-linux; board builds have no icon to skip)

onpremo serve model.gguf \
  --profile pi-1gb-safe \
  --bind 127.0.0.1:11500

Menu bar / system tray (built-in on macOS and Windows; Linux via opt-in tray-linux): while serve runs, a mascot icon appears in the menu bar (macOS), notification area (Windows), or StatusNotifier tray (Linux when compiled in). Menu items, in order: model label (disabled), Open chat (opens /), Open status page (opens /status), Quit OnPremo. The icon is rasterized at startup from the same 19×19 mascot grid as site/mascot.svg — no image assets. The mark is black-and-white and matches the bar it sits on: macOS registers it as a template image so the menu bar recolors it per appearance, and Windows reads the taskbar theme at startup and picks the black or white variant accordingly.

Desktop Linux gets the same menu when the binary is built with the opt-in tray-linux cargo feature: a StatusNotifierItem over the session D-Bus (pure Rust, no GTK), shown out of the box on Ubuntu, KDE Plasma, and XFCE; stock Debian GNOME needs the AppIndicator shell extension. Default Linux builds — including every published board binary — omit the tray so the certified musl/armv7 images stay lean; use the printed URLs (or onpremo status) instead. Headless SSH sessions: pass --no-tray. If the icon cannot be created (no window server, no D-Bus), serve falls back to a plain foreground process with a warning.

Non-loopback binds are refused unless ONPREMO_ALLOW_NONLOCAL=1 is set. When allowed, serve prints a one-time startup warning: the API has no authentication.

onpremo pull [NAME]

Download a model via the system curl (falling back to wget): a catalogue name, or any GGUF by URL with --url. The OnPremo binary links zero network crates; only this convenience command uses the network, and only by shelling out — same approach as the installer. Inference itself never needs the network.

NAME catalogue model name (case-insensitive exact or unique substring); with --url, optionally overrides the saved filename (must end in .gguf)
--list list catalogue entries (name, quantization, size, license, status, verified)
--dir <path> destination (default: $ONPREMO_MODELS_DIR or ~/.onpremo/models)
--catalogue <dir> TOML manifests (default: embedded catalogue)
--allow-unverified allow download when the catalogue entry has no SHA-256 pin
--url <URL> direct URL to a .gguf (bring your own model); filename derived from the URL
--sha256 <hex> with --url: verify a fresh download, or verify an already-downloaded file in place without re-fetching

Bring your own model. With --url the verification story inverts: no pin exists until you create one, so the first pull is unverified and the computed SHA-256 is printed prominently — pin later pulls (and CI installs) with --sha256. Split multi-part GGUFs (-00001-of-000NN) are refused: download all parts into one directory, then point onpremo run at the -00001-of- file (llama.cpp loads the rest). And pull is a convenience, not a gate — every model-consuming command accepts a plain path to any local .gguf, and recommend ranks uncatalogued files in the models directory alongside catalogue entries.

pull --list prints a VERIFIED column: sha256 for pinned entries, or UNVERIFIED for unpinned ones. A footer reports how many entries are SHA-256-pinned. onpremo pull <name> refuses unverified entries unless --allow-unverified is passed; after an unverified fetch it prints the observed hash and size so you can pin them in the manifest. Size column uses exact size_bytes when known, else admission approx_weight_bytes.

onpremo pull --list
onpremo pull SmolLM2-135M
onpremo pull some-unpinned-model --allow-unverified
onpremo pull --url https://huggingface.co/<repo>/resolve/main/<model>.gguf
onpremo pull --url <same-url> --sha256 <printed-hash>   # pin: verifies in place

onpremo recommend

Probe live memory and rank catalogue / local models by admission against a profile. When --profile is omitted, the profile is derived with the same pure suggestion math as profile suggest.

--profile <name|path> built-in name or TOML path; omit to auto-suggest from the host
--catalogue <dir> catalogue TOML directory (default: embedded)
--models-dir <dir> local *.gguf directory (default: $ONPREMO_MODELS_DIR or ~/.onpremo/models)
--json stable JSON array of recommendation rows

Ranking (first-experience default). Candidates order by admission verdict first: admitted → downgraded → rejected → no-specs. Within admitted (and within downgraded):

1. Already downloaded locally (a real .gguf beats a pull)
2. Catalogue-verified (non-empty SHA-256) before unverified
3. Smaller weight first
4. Name ascending as a stable tie-break

Unverified multi-GB catalogue entries therefore sink toward the bottom of the admitted band even when they fit. Human output prints up to two headlines when they differ:

• headline: best first pick = … — top of the first-experience order (safe default next step)
• headline: largest admitted = … — heaviest admitted model (capability signal; omitted when it is the same row)

Every run action line carries an explicit --profile <name-or-path> matching the profile used for admission. When recommend auto-suggested the profile (no --profile flag), suggested names are not loadable as built-ins, so actions are a two-step form: onpremo profile suggest --out <path> then run … --profile <path>. Pull actions stay onpremo pull <name> (pull does not take a profile).

Exit codes: 0 if any candidate is admitted or downgraded, 3 if nothing fits. run/chat/serve still default to pi-1gb-safe when you type them by hand — that is the Pi certification identity; prefer the profile recommend printed.

onpremo recommend
onpremo recommend --profile pi-1gb-balanced
onpremo recommend --json

onpremo profile suggest

Probe host memory and emit an honest hardware profile whose process budget, context/batch tiers, and safety margin scale with actual RAM (and a tighter cgroup limit when present). The three fixed built-ins cover the certified tiny tier; this command is the first-class path for every other host.

--name <n> default: suggested-<total-GiB-rounded>gb
--out <path> write the profile here instead of stdout; refuses to overwrite unless --force
--force allow replacing an existing --out file
--json emit the same Profile as JSON
--streaming include a [streaming] table when this build's Profile supports it

Default stdout is TOML with documented budget-math comments. Unprobed platforms (total_bytes == 0) print an error and exit 1. Suggested TOML loads with --profile <path> on check / run / chat / serve / agent like any other profile.

onpremo profile suggest
onpremo profile suggest --out ~/.onpremo/profiles/suggested-16gb.toml
onpremo profile suggest --json

onpremo status

Scripting twin of the embedded /status page. Fetches GET /v1/status and, when present, GET /v1/agent via system curl/wget (no network crates). Requires a running onpremo serve.

--url <base> server base URL, no trailing path (default: http://127.0.0.1:11500)
--json one merged JSON object (status + optional agent) on a single stdout line

Human output is a compact table. Exit 0 when healthy, 1 when unreachable or malformed.

onpremo status
onpremo status --json
onpremo status --url http://127.0.0.1:11500