CLI Reference

llmux                      # launch the interactive TUI (default)
llmux tui                  # launch the TUI explicitly
llmux top <profile>        # btop-style monitor in a plain terminal (no TUI)
llmux <group> <command>    # headless control
llmux <alias>              # top-level shortcut (see below)
llmux --version            # print the installed version and exit

Command map

Common concepts

top

The btop-style system monitor, rendered in a plain terminal without the Textual TUI. Good for low-bandwidth SSH or dumb terminals. Every GPU is shown always — util, memory, temperature, power, PCIe rx/tx — whether or not anything is running, plus a panel per running model (throughput and KV-cache braille graphs, cache-hit, requests, latency percentiles: TTFT/E2E p50/p95/p99 and phases). With no argument it shows every running model; pass a profile name to narrow it to that one. Press q or Ctrl+C to exit. The same monitor is reachable inside the TUI with the t key.

llmux top <profile>

Backend detection

Most commands take a profile name and resolve its backend (vllm or llamacpp) automatically by scanning profiles.yaml:

• If the name exists in exactly one backend, that backend is used.
• If it exists in both, you must disambiguate with --backend / -b.
• If it exists in neither, the command fails with a clear usage error.

--backend always wins when supplied.

JSON output

Commands that produce structured data accept --json to emit a JSON payload (UTF-8, 2-space indent) instead of the default padded table or text. Commands with --json support:

Exit codes

Lifecycle commands propagate the underlying docker / docker compose exit code. Ctrl-C while streaming (up, logs, build-dev) exits with 130. system env-check exits 1 when .env.common is missing or has issues.

container — container lifecycle

llmux container up|down|logs|ps|render-env …

Start, stop, inspect, and re-render profile containers. up, down, logs, ps, and render-env are also exposed as top-level aliases.

container up

Start a profile's container. Streams docker compose output to stdout.

llmux container up PROFILE [OPTIONS]

# Start a vLLM profile (official image, highest local versioned tag)
uv run llmux container up qwen3-0-6b

# Start it from a locally-built dev image
uv run llmux container up qwen3-0-6b --dev --tag main-20260515

# One-off: ignore a stale pinned image_tag and use the compose default
uv run llmux up qwen3-0-6b --default-image

# Start a llama.cpp profile, disambiguating an ambiguous name
uv run llmux up gemma-3-4b --backend llamacpp

# llama.cpp dev image from a personal fork
uv run llmux up gemma-3-4b --dev \
    --repo-url https://github.com/me/llama.cpp.git --branch master

container down

Stop a profile's container.

llmux container down PROFILE [OPTIONS]

uv run llmux container down qwen3-0-6b

container logs

Stream container logs (Ctrl-C to stop following).

llmux container logs PROFILE [OPTIONS]

# Follow live logs
uv run llmux logs qwen3-0-6b

# Print the last 50 lines and exit
uv run llmux container logs qwen3-0-6b -n 50 --no-follow

container ps

List profiles and their container status across backends.

llmux container ps [OPTIONS]

Table columns: backend, profile, status, port, gpu, container, model. JSON rows additionally include a boolean running field.

uv run llmux ps
uv run llmux ps --running --json
uv run llmux container ps -b vllm

container render-env

Re-render .runtime/<backend>/<profile>.env from profiles.yaml. Useful after editing profiles.yaml by hand. Prints the rendered path(s).

llmux container render-env [PROFILE] [OPTIONS]

# Re-render a single profile
uv run llmux render-env qwen3-0-6b

# Re-render every llama.cpp profile
uv run llmux container render-env --backend llamacpp

container benchmark · bench

Benchmark a running container's generation speed via /v1/chat/completions. A warmup call is discarded, then --runs measured calls report the median tok/s (a single cold call conflates model warmup with steady-state decode).

llmux bench PROFILE [OPTIONS]

uv run llmux bench qwen3-0-6b --runs 5
uv run llmux bench qwen3-0-6b --json

container stats · stats

Live token throughput (tok/s) for every running container, polled from its Prometheus /metrics endpoint. Rates are deltas between successive samples, so the first line appears one interval in. A server that's down or was started without metrics exposed shows n/a and keeps being polled.

llmux stats [OPTIONS]

uv run llmux stats                 # live, Ctrl-C to stop
uv run llmux stats --once --json   # one machine-readable sample

profile — profile CRUD + quick-setup

llmux profile list|show|new|edit|delete|quick-setup …

Profiles live in profiles.yaml at the repo root — the single source of truth for both backends. Fields not applicable to a backend stay at their defaults.

profile list

List all profiles across (or within) backends.

llmux profile list [OPTIONS]

Columns: backend, name, port, gpu_id, config, model.

uv run llmux profile list
uv run llmux profile list -b vllm --json

profile show

Show a profile's full record.

llmux profile show NAME [OPTIONS]

uv run llmux profile show qwen3-0-6b --json

profile new

Create a new profile entry in profiles.yaml. The name must match ^[a-z0-9][a-z0-9_-]*$ — lowercase only, because docker compose project names are lowercase-only and a cross-backend profile must validate on both sides. Creation fails if the name already exists in the chosen backend.

llmux profile new NAME [OPTIONS]

# vLLM profile
uv run llmux profile new qwen3-8b --backend vllm --model Qwen/Qwen3-8B \
    --gpu-id 0,1 --port 8000 --set VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8=1

# llama.cpp profile
uv run llmux profile new gemma-3-4b --backend llamacpp --port 8080 \
    --model-file gemma-3-4b-it-q4_k_m.gguf \
    --hf-repo unsloth/gemma-3-4b-it-GGUF --hf-file gemma-3-4b-it-q4_k_m.gguf

profile edit

Edit fields of an existing profile. Only the options you pass change; everything else is left untouched.

llmux profile edit NAME [OPTIONS]

# Pin a llama.cpp profile to a locally-built dev image
uv run llmux profile edit gemma-3-4b --image-tag llamacpp-dev:mtp_main

# Add an env var, remove another
uv run llmux profile edit qwen3-8b --set VLLM_ATTENTION_BACKEND=FLASHINFER --unset OLD_VAR

profile rename

Rename a profile. The profile's container must be stopped — it is named after the profile, so renaming under a running container would orphan it.

An unset config link is pinned to the old name, so the profile keeps resolving to the config file it already used. An unset container name follows the new profile name.

llmux profile rename OLD NEW [OPTIONS]

uv run llmux profile rename qwen3-8b qwen3-8b-fp8

profile clone

Copy a profile to a new name, keeping the same linked config. The clone reuses the source's port — change it before running both.

llmux profile clone SRC DST [OPTIONS]

uv run llmux profile clone qwen3-8b qwen3-8b-tuned

profile delete

Delete a profile (and optionally its linked config YAML).

llmux profile delete NAME [OPTIONS]

uv run llmux profile delete old-profile --with-config --yes

profile quick-setup

Create a profile and a config from a model id in one step — mirrors the TUI's "Quick Setup". If --name is omitted, a name is derived from the model id (lowercased, non-alphanumerics replaced with -); collisions get a numeric suffix.

llmux profile quick-setup MODEL [OPTIONS]

uv run llmux profile quick-setup Qwen/Qwen3-8B --gpu-id 0,1 --gpu-mem 0.92

config — config (YAML) CRUD

llmux config list|show|new|edit|clone|delete …

Per-model engine configs live in config/<backend>/<name>.yaml:

• vLLM — config/vllm/<name>.yaml: model: + gpu-memory-utilization: plus arbitrary vllm serve flags.
• llama.cpp — config/llamacpp/<name>.yaml: a flat dict of llama-server flags.

Config names are validated against ^[a-zA-Z0-9][a-zA-Z0-9._-]*$ to prevent path traversal. For show / edit / delete, if the same name exists in both backends you must disambiguate with --backend.

config list

List config files.

llmux config list [OPTIONS]

Columns: backend, name, model, params (number of keys in the YAML).

config show

Print a config YAML.

llmux config show NAME [OPTIONS]

config new

Create a new config YAML. For vLLM, the file is seeded with model: and gpu-memory-utilization:; for llama.cpp it starts empty. --set values are YAML-parsed, so --set foo=2 stores an int and a bare --set bar stores true.

llmux config new NAME [OPTIONS]

# vLLM config
uv run llmux config new qwen3-8b --model Qwen/Qwen3-8B --gpu-mem 0.9 \
    --set max-model-len=8192 --set enable-prefix-caching

# llama.cpp config
uv run llmux config new gemma-3-4b --backend llamacpp \
    --set ctx-size=8192 --set n-gpu-layers=999 --set flash-attn

config edit

Patch fields in an existing config.

llmux config edit NAME [OPTIONS]

uv run llmux config edit qwen3-8b --set max-model-len=16384 --unset enable-prefix-caching

# Toggle a flag off without losing its value, then back on
uv run llmux config edit qwen3-8b --disable enforce-eager
uv run llmux config edit qwen3-8b --enable enforce-eager

A disabled parameter is stored as a # llmux:disabled <key>: <value> comment marker — inert to the server, visible to llmux. Hand-written # comments in the file are preserved across edits. See Toggling parameters on / off.

config clone

Clone SRC config to DST within the same backend. Useful when profile quick-setup --copy-from isn't enough — for example, when you want to clone a tuned config to a new name without creating a profile in the same step. Routes through the backend's canonical serializer so the resulting YAML is byte-identical to a TUI-edited config.

llmux config clone SRC DST [OPTIONS]

uv run llmux config clone qwen3-0-6b qwen3-0-6b-experiment

config from-recipe

Create a vLLM config from the model's official vllm-project/recipes recipe. The TUI shows an interactive review window (see Import a vLLM recipe); this headless form picks a variant with --variant, or auto-picks the highest-quality one that fits the detected GPU.

llmux config from-recipe MODEL_ID [OPTIONS]

uv run llmux config from-recipe Qwen/Qwen3-32B --list
uv run llmux config from-recipe Qwen/Qwen3-32B --variant fp8 --feature reasoning
uv run llmux config from-recipe Qwen/Qwen3-32B --variant awq --json

config rename

Rename a config YAML and repoint every profile that referenced it. Refused while any referencing profile's container is running.

llmux config rename OLD NEW [OPTIONS]

uv run llmux config rename qwen3-0-6b qwen3-0-6b-fp8

config delete

Delete a config YAML.

llmux config delete NAME [OPTIONS]

image — Docker image inventory + dev build

llmux image list|pull|remove|build-dev …

image list

List local (and optionally remote) Docker images.

llmux image list [OPTIONS]

Columns: source (local / local-dev / remote), repository, tag, size, created.

uv run llmux image list --dev
uv run llmux image list --remote --json

image pull

docker pull <repo>:<tag> — surfaces raw docker output and exit code.

llmux image pull TAG [OPTIONS]

uv run llmux image pull v0.20.1
uv run llmux image pull nightly

image remove

Wraps docker rmi <ref> so you can drop a local image without leaving the CLI. Use --force to drop images that are still referenced by a stopped container.

llmux image remove REF [OPTIONS]

uv run llmux image remove llamacpp-dev:fork-master
uv run llmux image remove vllm/vllm-openai:v0.20.0 --force

image build-dev

Build a <backend>-dev:<tag> image from source, streaming docker build output. The build pipeline (git clone/update → docker build → image labels) is shared; the tags applied are <prefix>:<custom_tag or branch-YYYYMMDD> and <prefix>:<branch> (a stable alias to the latest build).

• vLLM → vllm-dev:<tag> (target vllm-openai, docker/Dockerfile)
• llama.cpp → llamacpp-dev:<tag> (target server, .devops/cuda.Dockerfile)

llmux image build-dev [OPTIONS]

# vLLM dev image from an unmerged PR branch
uv run llmux image build-dev --backend vllm --branch my-feature \
    --repo-url https://github.com/me/vllm.git

# llama.cpp dev image, pinned to a single GPU arch for a fast build
uv run llmux image build-dev --backend llamacpp --branch master --cuda-arch 89

See Dev Builds for the full workflow, and Architecture for how the pipeline works.

system — system info & env validation

llmux system gpu|mem-estimate|disk|env-check …

system gpu

Print an nvidia-smi summary, one row per GPU. Also available as the top-level alias llmux gpu.

llmux system gpu [OPTIONS]

Columns: index, name, memory_used, memory_total, utilization, temperature. Prints (no GPUs detected — is nvidia-smi installed?) if none are found.

uv run llmux gpu
uv run llmux system gpu --json

system mem-estimate

Estimate a Hugging Face model's VRAM footprint via hf-mem — the same engine the TUI's memory estimator uses. Reads HF_TOKEN from .env.common (or the environment) for gated models.

llmux system mem-estimate MODEL_ID

uv run llmux system mem-estimate Qwen/Qwen3-8B
# → ~17.2GB (model: 15.3GB + KV: 1.9GB)

system disk

Show the llama.cpp model directory, its GGUF inventory + sizes, and df -h filesystem usage for the partition that hosts it. Mirrors the TUI's System → Disk / Model Dir tab. The vLLM equivalent (HF cache usage) lives in the TUI's System → Disk / HF Cache tab.

llmux system disk

uv run llmux system disk
# Model dir : /home/me/Project/llmux/models
# GGUF files: 2 (total 17.1 GB)
# df ...: used 583G, avail 285G (68%)

system env-check

Validate .env.common and report key paths. Checks that the file exists, that HF_TOKEN / HF_CACHE_PATH / VLLM_VERSION are set, and that HF_CACHE_PATH is absolute. Also available as the top-level alias llmux env-check.

llmux system env-check [OPTIONS]

Exits 0 when everything checks out, 1 when .env.common is missing or any issue is found. HF_TOKEN is masked in the text report (shown as <set, N chars>).

uv run llmux env-check
uv run llmux system env-check --json

See .env.common Reference for every variable this command validates.

Top-level aliases

For convenience — and so agents can write llmux up <profile> directly — the most-used commands are mirrored at the top level:

These aliases share the exact arguments, options, and defaults of their canonical commands.