.env.common Reference

cp .env.common.example .env.common
# then edit the values

All variables at a glance

¹ Required only for gated/private Hugging Face repos — public models download without it.
² Required only when a vLLM profile has enable_lora: true.
³ vllm container up resolves a tag automatically from local images when --tag is empty; env-check still flags VLLM_VERSION as unset because it is the variable the official docker-compose.yaml substitutes.
⁴ Reserved for the host GGUF directory; current llama.cpp profiles download GGUF files directly into the HF cache via llama-server -hf (see below).

Hugging Face

HF_TOKEN

Hugging Face access token. Used to download gated or private models on both backends — vLLM passes it as HUGGING_FACE_HUB_TOKEN, llama.cpp as HF_TOKEN. If you only ever serve public repositories you can leave it empty.

HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

The token is also read by llmux system mem-estimate to size gated models, and is masked in env-check output.

HF_CACHE_PATH

Required. Absolute host path mounted into every container as the Hugging Face cache (/root/.cache/huggingface). Both backends download model weights here, so the cache is shared and persistent across container restarts and across the two backends.

HF_CACHE_PATH=/home/$USER/.cache/huggingface

vLLM

LORA_BASE_PATH

Host directory containing LoRA adapter folders. When set, it is mounted read-only at /app/lora inside the vLLM container (via compose/vllm/docker-compose.lora.yaml), so a profile's lora_modules entries can reference /app/lora/<adapter>. Only consumed when the profile has enable_lora: true.

LORA_BASE_PATH=/home/$USER/lora-adapters

VLLM_VERSION

Tag of the official vllm/vllm-openai image used by compose/vllm/docker-compose.yaml (image: vllm/vllm-openai:${VLLM_VERSION}). In practice llmux container up selects the highest local versioned tag (or the tag you pass with --tag) and injects it for you — but setting VLLM_VERSION gives env-check and a bare docker compose invocation a known-good default.

VLLM_VERSION=v0.20.1

VLLM_USE_V2_MODEL_RUNNER

vLLM ships two model runners and defaults to V2. The V2 runner does not support the per-request thinking_token_budget parameter — a request that uses it fails with:

400: thinking_token_budget is not yet supported by the V2 model runner.
Run vLLM with VLLM_USE_V2_MODEL_RUNNER=0

Set it to 0 in .env.common to cap how long a reasoning model thinks:

VLLM_USE_V2_MODEL_RUNNER=0

The budget itself is a per-request field, passed top-level — nesting it inside chat_template_kwargs silently does nothing:

client.chat.completions.create(
    model=..., messages=...,
    extra_body={
        "chat_template_kwargs": {"enable_thinking": True},
        "thinking_token_budget": 512,   # top-level
    },
)

VLLM_REPO_URL / VLLM_BRANCH

Defaults for the vLLM dev-build pipeline (llmux image build-dev --backend vllm). When --repo-url / --branch are omitted on the CLI, these values are used; if they are also unset, the pipeline falls back to https://github.com/vllm-project/vllm.git and main.

VLLM_REPO_URL=https://github.com/vllm-project/vllm.git
VLLM_BRANCH=main

llama.cpp

MODEL_DIR

Host directory intended for GGUF files (mounted at /models in the container).

MODEL_DIR=/home/$USER/Project/docker/llmux/models

LLAMACPP_IMAGE

Tag of the official llama.cpp server image used by compose/llamacpp/docker-compose.yaml (image: ${LLAMACPP_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda}). Override it to pin a specific upstream build.

LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda

To run a profile on a locally-built image instead, build with llmux image build-dev --backend llamacpp and set image_tag: llamacpp-dev:<branch> on that profile in profiles.yaml — that is a per-profile override and does not touch LLAMACPP_IMAGE.

LLAMACPP_REPO_URL / LLAMACPP_BRANCH

Defaults for the llama.cpp dev-build pipeline (llmux image build-dev --backend llamacpp). When --repo-url / --branch are omitted, these are used; if also unset, the pipeline falls back to https://github.com/ggml-org/llama.cpp.git and master.

LLAMACPP_REPO_URL=https://github.com/ggml-org/llama.cpp.git
LLAMACPP_BRANCH=master

Common

LLMUX_LANG

UI language for the TUI — ko or en. Read from the process environment (not .env.common, which is a container-side file), and resolved once at startup, so set it before launching:

LLMUX_LANG=en llmux

Unset falls back to the system locale. This affects the TUI only; CLI output is English.

TZ

Timezone applied to both backend containers (TZ=${TZ:-UTC} in both Compose files). Affects log timestamps inside the container. Defaults to UTC if unset.

TZ=Asia/Seoul

Template

The shipped .env.common.example you copy from:

# llmux 공통 설정 템플릿
# cp .env.common.example .env.common 후 값 수정

# ─── HuggingFace ───────────────────────────────────────────────
# 게이트 모델 다운로드에 사용 (공개 repo 만 쓰면 비워도 됨)
HF_TOKEN=
HF_CACHE_PATH=/home/$USER/.cache/huggingface

# ─── vLLM 전용 ─────────────────────────────────────────────────
# LoRA adapter 경로 (container 안 /app/lora 로 mount)
LORA_BASE_PATH=

# ─── llama.cpp 전용 ────────────────────────────────────────────
# GGUF 저장 경로 (container 안 /models 로 mount)
MODEL_DIR=/home/$USER/Project/docker/llmux/models

# llama.cpp 이미지 태그
LLAMACPP_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda

# llama.cpp dev-build (per-fork / PR branch). 비워두면 ggml-org/master 사용.
LLAMACPP_REPO_URL=https://github.com/ggml-org/llama.cpp.git
LLAMACPP_BRANCH=master

# 타임존
TZ=Asia/Seoul

For where these values flow at launch time, see Architecture. For the per-command flags that override these defaults, see the CLI Reference.