Using the TUI

Launching

llmux          # no arguments → launches the TUI
llmux tui      # explicit

Running llmux with no subcommand drops you straight into the unified dashboard.

The dashboard

The dashboard is one DataTable showing every profile from both backends. Running containers float to the top so the active workload is always visible without scrolling; within the running / stopped groups, rows stay sorted by (backend, name).

┌ llmux ─ vLLM + llama.cpp ───────────────────────────────────────────────────────────────────┐
│  vLLM 1/2  ·  llama.cpp 0/1  ·  Enter = actions                   │
├─────────────────────────────────────────────────────────────────────┤
│ Status      Backend    Profile      Port   tok/s  Model        Detail  │
│ ● running   vLLM       qwen3-8b     8000   62.4   Qwen3-8B     TP=1    │
│ ○ stopped   llama.cpp  gemma-3-4b   8080   —      gemma-3-4b   —       │
│ ○ stopped   vLLM       qwen3-0-6b   8001   —      Qwen3-0.6B   —       │
└─────────────────────────────────────────────────────────────────────┘
 GPU0 ████████░░░░ 64%  12.1/24GB
 🔍  Estimate HF model memory (press m then type, Enter to run)

A status bar shows running / total counts per backend. The tok/s column shows live generation throughput for each running model, polled from its /metrics endpoint every 3 seconds (see llmux stats). A GPU bar below the table auto-refreshes every few seconds, and the table itself re-scans every 5 seconds. The memory-estimator search row is hidden by default to keep the dashboard compact — press m to reveal it (and Esc to hide it again).

Keyboard shortcuts

Move the cursor with ↑ / ↓, then act on the selected row.

Dashboard

The u / d / l / e / c / x keys are deliberately hidden from the footer — they do exactly the same thing as picking the matching item from the Enter action menu.

App-level

Log / startup viewers

The action menu

Pressing Enter on a profile opens a context menu. The options depend on whether the container is running:

Benchmark fires a single /v1/chat/completions request at the running server and reports tokens/second. Delete is blocked while a container is running — stop it first. To rename a profile, edit its Name field in the profile form (also only while the container is stopped, since it's named after the profile).

Live monitor

Press v on any row (or pick 📊 Monitor from the action menu) to open a full-screen, btop-style system view. It polls nvidia-smi and each running server's /metrics once a second. Opening it does not require a running container — with nothing up you still get the GPU panel, which is usually the reason to open a monitor at all.

Always shown:

• GPU — every GPU on the box (card name in the panel title): utilization, memory, temperature, power, and PCIe rx/tx as heat bars.

Then one panel per running model — the running list is re-scanned every tick, so a model started elsewhere appears on its own. With exactly one model up you get the full detail view; with several, each gets a compact summary row (tok/s, KV, running/waiting, TTFT, TPOT).

• Throughput — generation and prompt tok/s as a braille graph, with current and peak;
• KV cache — utilization trend as a braille graph, current and peak;
• Requests — running, waiting, completions/s, and preemptions;
• Cache hit — KV, prefix, and external prefix hit rate as heat bars (vLLM; llama.cpp shows KV only);
• Latency — TTFT and E2E percentiles (p50/p95/p99) from the histograms, plus TPOT, queue, and prefill/decode/inference phases (vLLM; llama.cpp shows — where it exposes no histogram).

Both backends work — the differing Prometheus metric names are matched by family, and any metric a backend doesn't expose reads — rather than a made-up value. Keys: p pause, r reset peaks, +/- poll interval, l language, Esc/q back.

Terminal monitor (plain)

Press t on any row to suspend the Textual UI and open the same monitor — the same GPU panel, braille graphs, cache-hit, and latency percentiles — as a plain-terminal page in your normal terminal, auto-refreshing every second. No Textual rendering, so it holds up over low-bandwidth SSH or on terminals that render the TUI poorly. Same keys; q (or Ctrl+C) returns.

The same monitor is available without the TUI at all: llmux top <profile>.

Quick Setup

n opens a backend picker, then a Quick Setup form that creates a matching profile + config in one step. For vLLM the form takes a Hugging Face model id, GPU id, port, GPU-memory-utilization, an Enable-LoRA toggle, and an optional "copy params from" existing config. The profile name is derived from the model id automatically.

Memory estimator

Press m (or click the search box) and type a Hugging Face model id, then Enter. llmux estimates the model's memory footprint and renders it against your detected GPUs:

• A per-GPU bar shows the estimated fit ratio — green / yellow / red, or a red OVER marker when the estimate exceeds available VRAM.
• With multiple GPUs, it also shows the per-GPU figure assuming tensor parallelism (TP=N).

This runs entirely on the dashboard — no profile or container needed.

Container start screen

Picking Start Container opens a full-screen launcher with a Version radio set. The options differ per backend:

Selecting Dev Build reveals Repo URL and Branch inputs, pre-filled from .env.common. Pressing Start streams the startup log live, then switches to following the container's logs. See Dev Builds for the build mechanics.

Profile and config forms

Edit Profile and Edit Config open modal forms:

• The profile form edits the profile name, container name, port, GPU id, the linked config, and image_tag. Changing the name renames the profile on save rather than creating a second one — the container must be stopped and the new name must be free in both backends.
• The config form is a dynamic list of flag rows — add a row, type a flag name (autocompleted from the real flag set of the vllm serve / llama-server image you're running — see Flag autocomplete), and a value. Leaving a boolean flag's value blank stores true. A help line shows a one-line description of the focused flag. Each row also has an on/off switch: flip it off to park a flag without deleting its value (the row dims), flip it back on to restore it. See Toggling parameters on / off. Editing the config's name renames the file and repoints every profile that referenced it.

Both forms map straight onto profiles.yaml and the config YAML — there is no separate TUI state.

See also

• Profiles / Model Configs — what the forms edit
• Container Lifecycle — what Start / Stop / Logs do
• Dev Builds — the Dev Build start mode
• CLI Reference — the same operations, headless