Container Lifecycle

Startup flow

Starting a profile is more than docker compose up. llmux runs an ordered pipeline and aborts early with a clear message if any step fails.

flowchart TD
    A[Resolve profile from profiles.yaml] --> B[Port conflict check]
    B --> C[Render profile env file]
    C --> D{Backend}
    D -->|vLLM| E[Validate .env.common + linked config]
    D -->|llama.cpp| F[Render override.yaml from config]
    E --> G[GPU conflict warnings]
    F --> G
    G --> H[docker compose up -d]
    H --> I[Post-start health validation]
    I --> J[Container ready]

1. Profile env render

The profile is read from profiles.yaml and rendered to .runtime/<backend>/<name>.env. This file is what docker compose --env-file consumes — it carries CONTAINER_NAME, the port, GPU_ID, CONFIG_NAME, and backend-specific values. It is regenerated on every start, so editing it directly is pointless; edit the profile instead.

2. Config / override render

3. Conflict gates

Before compose up, llmux checks for resource clashes:

• Port conflicts — the target port is probed against running Docker containers and local processes (a raw socket bind). A static overlap between two stopped profiles is ignored.
• GPU conflicts — the profile's gpu_id set is compared against every running container from both backends. Overlap produces a warning.

In the CLI, port conflicts are hard errors. In the TUI, the cross-backend conflict gate surfaces port + GPU clashes in a confirmation dialog so you can Start anyway or cancel.

4. compose up + health validation

llmux runs docker compose ... up -d with the rendered env files. After compose returns, a post-start validation loop polls docker inspect:

• If the container is restarting / exited / dead or becomes unhealthy, startup is reported as failed and the last log lines are printed (this catches e.g. a GPU OOM during engine init).
• Otherwise llmux waits for the model endpoint to respond — /v1/models for vLLM, /health for llama.cpp. If the model is still loading when the timeout hits, you get a warning, not a failure.

The lifecycle commands

All commands auto-detect the backend from the profile name; pass --backend / -b to disambiguate when a name exists in both. These are top-level CLI aliases — llmux up is shorthand for llmux container up.

up — start a profile

llmux up qwen3-0-6b
llmux up qwen3-0-6b --backend vllm

# image overrides
llmux up qwen3-0-6b --tag v0.20.1          # specific vllm/vllm-openai tag
llmux up qwen3-0-6b --tag v0.20.1 --pull   # force --pull always (vLLM only)
llmux up qwen3-0-6b --dev                  # use the locally-built dev image
llmux up gemma-3-4b --tag llamacpp-dev:master

Compose output streams to stdout. The command exits non-zero if startup fails.

down — stop a profile

llmux down qwen3-0-6b

llmux prefers docker compose down for a clean network teardown. If the container is orphaned (no override file, or compose down fails), it falls back to docker stop + docker rm (docker rm -f for llama.cpp). Stopping an already-stopped profile is a no-op that exits 0.

logs — stream container logs

llmux logs qwen3-0-6b              # follow (default), last 200 lines first
llmux logs qwen3-0-6b --no-follow  # print recent lines and exit
llmux logs qwen3-0-6b --tail 500   # change the initial backlog

While following, Ctrl+C stops the stream (exit code 130).

ps — list profiles and status

llmux ps                    # all profiles, both backends
llmux ps --backend vllm     # one backend only
llmux ps --running          # only running containers
llmux ps --json             # machine-readable output

Output columns: backend, profile, status, port, gpu, container, model. Status is derived from docker ps health state — healthy, unhealthy, starting, running, exited, or stopped.

render-env — regenerate runtime env files

llmux render-env qwen3-0-6b   # one profile
llmux render-env              # re-render every profile
llmux render-env --backend llamacpp

Useful after hand-editing profiles.yaml, or to inspect exactly what compose will see. up does this automatically — you rarely need it directly.

Healthchecks

Both compose definitions declare a Docker healthcheck that curls the in-container /health endpoint every 30s. The start_period is generous to allow for model loading — 600s for vLLM, 300s for llama.cpp — so a slow first load won't be flagged as unhealthy. ps and the TUI dashboard read this health state to show healthy / starting / unhealthy.

See also

• Profiles — the launch recipe up resolves
• Model Configs — what gets rendered into the container at startup
• Using the TUI — the same lifecycle, driven interactively
• Dev Builds — the --dev image path