Documentation

Reference for the OnPremo CLI, device profiles, memory admission model, optional local HTTP API, benchmark report shape, uninstall, and building from source. Claims below match the pre-0.1 surface in this repository.

▶ On this page

CLI reference

Binary name: onpremo. Five subcommands: inspect, check, run, benchmark, and (feature-gated) serve.

onpremo inspect <model.gguf>

Inspect GGUF metadata without loading weights.

--json emit machine-readable JSON instead of a table 

onpremo inspect smollm2-135m-instruct-q4_k_m.gguf
onpremo inspect smollm2-135m-instruct-q4_k_m.gguf --json

onpremo check <model.gguf>

Test model compatibility with a device profile without loading the model.

--profile <name|path> built-in name or TOML path (default: pi-1gb-safe)
--json emit machine-readable admission outcome

Exit codes: 0 admitted or downgraded, 2 rejected, 1 for bad paths or invalid profiles. 

onpremo check model.gguf --profile micro-500mb
onpremo check model.gguf --profile pi-1gb-safe --json

onpremo run <model.gguf>

One-shot inference with streaming tokens on stdout and immediate cleanup.

--profile <name|path> default: pi-1gb-safe
--prompt <text> required prompt string
--max-tokens <n> default: 64
--temperature <f> default: 0.7
--seed <u64> optional RNG seed
--report print machine-readable run report JSON to stderr 

onpremo run model.gguf \
  --profile pi-1gb-safe \
  --prompt "Return the intent as JSON" \
  --max-tokens 32

onpremo run model.gguf \
  --profile pi-1gb-safe \
  --prompt "hello world" \
  --max-tokens 4 \
  --report 2>report.json

onpremo benchmark <model.gguf>

Repeatable workload; writes a machine-readable JSON report (schema version 1).

--profile <name|path> default: pi-1gb-safe
--context <n> optional context override
--batch <n> optional batch override
--max-tokens <n> default: 64
--iterations <n> default: 3
--warmup <n> default: 1
--prompt <text> default: Summarize: the pump pressure is low.
--output <path> write JSON here (default: stdout) 

onpremo benchmark model.gguf \
  --profile pi-1gb-safe \
  --context 512 \
  --batch 16 \
  --max-tokens 64 \
  --iterations 3 \
  --warmup 1 \
  --output result.json

onpremo serve <model.gguf>

Optional bounded local HTTP API. Requires the CLI server feature (enabled in the default build). Binds loopback by default.

--profile <name|path> default: pi-1gb-safe
--bind <addr> default: 127.0.0.1:11500
--idle-unload <secs> unload after idle seconds; 0 disables (default: 30)
--max-request-bytes <n> hard body ceiling (default: 65536) 

onpremo serve model.gguf \
  --profile pi-1gb-safe \
  --bind 127.0.0.1:11500

Non-loopback binds are refused unless ONPREMO_ALLOW_NONLOCAL=1 is set.

Profiles

Built-in device profiles ship as TOML under profiles/. Pass a name (pi-1gb-safe) or a path to a custom TOML file.

Name Process budget OS reserve Default context / batch KV types Max out Notes
micro-500mb 300 MiB
(314572800)		 200 MiB 384 / 16
(256–512 / 8–16)		 f16, q8_0, q4_0 64 ~500 MB total RAM or 300 MB process budget; 135M Q4 and smaller; require_no_swap
pi-1gb-safe 600 MiB
(629145600)		 400 MiB 512 / 16
(512–1024 / 16–32)		 f16, q8_0 128 1 GB total RAM, conservative; 135M–360M Q4; CLI default
pi-1gb-balanced 750 MiB
(786432000)		 250 MiB 512 / 16
(256–1024 / 16–32)		 f16, q8_0, q4_0 192 Minimal OS reserve; measured headroom for selected 0.5B models

Safety margin ratios: micro-500mb 0.15, pi-1gb-safe 0.10, pi-1gb-balanced 0.08.

Memory model summary

Memory management is the central product feature. The runtime distinguishes model file size from peak process memory. Required memory is estimated as: 

Required memory =
    model residency
  + KV cache
  + compute buffers
  + tokenizer/runtime overhead
  + operating-system reserve
  + safety margin

The process budget (weights + KV + compute + overhead + margin) is distinct from the OS reserve. Admission enforces the process ceiling; the OS reserve is reported for device-level capacity.

Admission flow

  1. Read the device profile and current available memory
  2. Inspect model metadata and quantization without fully loading
  3. Estimate a conservative initial configuration
  4. If needed, reduce batch, context, and KV precision in a fixed order
  5. Reserve OS and emergency margin
  6. Load only after the estimate passes
  7. Monitor RSS and terminate cleanly if the hard process budget is approached
  8. Record measured peak memory vs. the estimate

Downgrade order

  1. Start with the requested profile
  2. Reduce prompt batch size
  3. Reduce context length
  4. Select a lower-memory KV-cache type
  5. Reduce maximum output tokens
  6. Reject with a machine-readable reason code if still incompatible
Rejected with a reason code beats an OOM kill. The OS OOM killer is never treated as normal control flow. Authoritative certification metric: cgroup v2 memory.peak / memory.current when a cgroup limit applies; process RSS is a secondary field.

HTTP API

Bounded local interface served by onpremo serve. One request at a time, request-size ceiling, optional idle unload, loopback by default.

Method Path Description
POST /v1/generate JSON body GenerateRequest → GenerateResponse (text + stats). Fields: prompt (required), optional max_tokens, temperature, top_p, seed, stop. max_tokens is clamped to the admitted config.
GET /v1/health Liveness: {"status":"ok","model_loaded":bool}
GET /v1/model Model GGUF summary and active RunConfig
GET /v1/status Live status: model, config, model_loaded, requests_served, memory (current / peak RSS vs process budget)
GET /v1/agent Loaded agent registry (name, system, tools). 404 JSON when no manifest. Never includes executor wiring.
GET /status Read-only status page (mirrors /v1/status + agent)
GET / Embedded web chat 
curl -sS http://127.0.0.1:11500/v1/generate \
  -H 'content-type: application/json' \
  -d '{"prompt":"Summarize: the pump pressure is low.","max_tokens":64}'

# Scripting twin of the status page
onpremo status --json
curl -sS http://127.0.0.1:11500/v1/status

Surfaces: CLI, API, UI

Three surfaces, one capability contract — CLI, API, and UI are thin views over the same engine. Operating-loop capabilities have full parity; provisioning (pull, discover, benchmark) is CLI/API-first by design: a LAN-facing gateway page must not mutate the machine.

Capability CLI API UI
Chat onpremo chat POST /v1/chat/completions GET /
Tool calls + confirmation chat --agent, agent step/loop OpenAI tools on chat (no exec) Web chat tool-call bubble
Model / config inspect, check, status GET /v1/model, /v1/status /status model + config
Live memory onpremo status GET /v1/status → memory /status memory meter
Agent registry agent validate, --agent GET /v1/agent /status agent section
Structured one-shot run --json-schema response_format on chat —(by design)
Pull / discover / benchmark pull, discover, benchmark —(by design) —(by design)

Benchmark report fields

Schema version 1. Every compatibility claim should be reproducible from a documented command. Top-level JSON object: 

{
  "schema_version": 1,
  "device": { "...": "DeviceInfo" },
  "model": { "...": "ModelSummary" },
  "backend": "mock",
  "profile_name": "micro-500mb",
  "admission": { "decision": "admitted", "...": "..." },
  "config": { "...": "RunConfig or null" },
  "prompt_chars": 42,
  "iterations": [ { "...": "IterationResult" } ],
  "aggregate": { "...": "Aggregate or null" }
}

Field groups

Group Contents
Device hostname, os, kernel, arch, cpu_model, total_memory_bytes, runtime_version
Model GGUF summary: name, architecture, quantization, file/tensor sizes, layer/head/context/vocab counts
Admission admitted | downgraded (with steps) | rejected (reason + detail). Rejected runs leave config/aggregate null and iterations empty.
Config context, batch, kv_cache_type, max_output_tokens, threads
Per iteration GenerationStats (prompt/generated tokens, load/TTFT/total ms, tokens/s), RssReport (start/peak/end RSS, optional cgroup peak), swap before/after, completed
Aggregate Mean/min/max generation tokens/s, mean prompt tokens/s, mean load and first-token ms, max peak RSS, max swap delta

Certification expects repeatable completion without swap use or OOM termination, peak memory under the declared process budget with a documented safety margin, and disclosure of thermal throttling.

Uninstall

Default install prefix is $HOME/.onpremo (override with ONPREMO_PREFIX). The uninstall script removes the binary and PATH block; it does not stop services (none are installed). 

curl -fsSL https://onpremo.org/uninstall.sh | sh

Non-interactive: set ONPREMO_UNINSTALL=1 and/or ONPREMO_NONINTERACTIVE=1 as documented by the installer. Manual removal: delete $HOME/.onpremo/bin/onpremo and the # >>> onpremo PATH >>> … # <<< onpremo PATH <<< markers from your shell profile.

Build from source

Requirements: stable Rust toolchain, cmake, and a C++ toolchain when enabling the real inference backend. Linux (ARM64 / x86-64) or macOS for development. 

# Mock backend (default — development and CI)
cargo build --workspace
cargo build -p onpremo-cli
# Binary: target/debug/onpremo

# Real llama.cpp backend
cargo build -p onpremo-cli --features backend-llama
# or
cargo install --path crates/onpremo-cli --features backend-llama

One-liner installer that clones and builds with the llama backend: 

curl -fsSL https://onpremo.org/install-from-source.sh | sh
Do not enable backend-llama in CI; mock-backend workspace tests are the default gate. Prebuilt installer assets always target the llama-backed binary when published — see bin/FLAVORS and SHA256SUMS.