OnPremo

Install

One-liner installers download a prebuilt binary, verify SHA-256, and put onpremo on your PATH.

curl -fsSL https://onpremo.org/install-linux-x86_64.sh | sh

curl -fsSL https://onpremo.org/install-linux-arm64.sh | sh

curl -fsSL https://onpremo.org/install-linux-armv7.sh | sh

curl -fsSL https://onpremo.org/install-macos-apple-silicon.sh | sh

curl -fsSL https://onpremo.org/install-macos-intel.sh | sh

powershell -c "irm https://onpremo.org/install-windows.ps1 | iex"

curl -fsSL https://onpremo.org/install-from-source.sh | sh

The memory contract

Most local LLM tools let you pick a model and hope it fits. OnPremo treats memory as a control plane: estimate first, load only if the estimate passes, and never treat the OS OOM killer as normal control flow.

Required memory is estimated as:

The process budget (weights + KV + compute + overhead + margin) is what admission enforces. The OS reserve is reported for device-level capacity but sits outside that process ceiling.

Illustrative onpremo check output

Shape copied from the examples walk-through — numbers depend on host, profile, and model.

decision          admitted
context           512
batch             16
kv cache          F16
max output tokens 128
threads           4
weights           107374182 (102.4 MiB)
kv cache          ...
compute           ...
runtime overhead  ...
os reserve        ...
safety margin     ...
process total     ...  (budget 629145600 (600.0 MiB))
device total      ...

Downgrade order

When the profile default does not fit, admission applies downgrades in this fixed order:

1. Start with the requested profile
2. Reduce prompt batch size
3. Reduce context length
4. Select a lower-memory KV-cache type
5. Reduce maximum output tokens
6. Reject the model as incompatible if it still cannot fit

Rejected with a reason code beats an OOM kill. When nothing in the profile’s configuration space fits, OnPremo refuses the run with a machine-readable reason code instead of swapping the system to death or relying on the kernel OOM killer.

Models

One contract, every rung of the ladder. The table below is the same admission math applied from a 500 MB board to a 122B-parameter open-weight model: fit, downgrade in a defined order, or reject with a reason code. Certified where it is hardest (the Pi tier), useful all the way up — onpremo recommend computes exactly this for your machine.

Verdicts computed by onpremo recommend against real profiles: micro-500mb, pi-1gb-safe, and a 16 GB laptop's profile suggest output. Admission math, pending on-device certification for the tiny tier; weights are fetched only by an explicit onpremo pull — sources and licenses are recorded in the catalogue.

Every entry's source, license, and (where pinned) SHA-256 lives in models/catalogue/. Bring-your-own GGUFs get the same verdicts — see Quickstart below.

Quickstart

Three ways to a first model — all end at the same admission contract, and OnPremo never downloads weights unless you ask.

onpremo recommend

onpremo pull --list
onpremo pull SmolLM2-135M

onpremo pull --url https://…/model.gguf
onpremo run /path/to/model.gguf \
  --profile pi-1gb-safe --prompt "hi"

Then the same moves work for every model, wherever it came from:

# 1. See what this machine can actually run
onpremo recommend

# 2. Inspect metadata without loading weights
onpremo inspect smollm2-135m-instruct-q4_k_m.gguf

# 3. Test compatibility with a device profile
onpremo check smollm2-135m-instruct-q4_k_m.gguf --profile pi-1gb-safe

# 4. Run one-shot streaming generation
onpremo run smollm2-135m-instruct-q4_k_m.gguf \
  --profile pi-1gb-safe \
  --prompt "Return the intent as JSON" \
  --max-tokens 32

Exit codes for check / run: 0 admitted or downgraded, 2 rejected, 1 for bad paths or invalid profiles. On machines beyond the built-in tiny profiles, onpremo profile suggest derives a profile scaled to the host, and onpremo discover inventories local hardware. Full flag reference lives on the docs page.

Status