# Serve local GGUFs to opencode with llama-server router mode

> Run llama-server in router mode as a systemd --user unit that starts with opencode, sleeps when idle and stops when the last opencode exits.

Published 2026-11-11. HTML version: https://guides.tsvinc.dev/serve-local-models-to-opencode-llama-router/


A wrapper named `oc` starts a llama.cpp `llama-server` in router mode as a transient `systemd --user` unit, runs opencode against it as a `local` provider that lists only fully downloaded GGUFs, and stops the router when the last opencode exits.

> **Scope** — Verified on: Ryzen 7 PRO 6850U laptop (32 GiB LPDDR5, Radeon 680M with 4 GiB carve-out + 13.58 GiB GTT), Fedora 44, kernel 7.2.10, Mesa 26.2.4 RADV, llama.cpp b11292 `ubuntu-vulkan-x64` prebuilt, opencode 1.19.3, Python 3.14.8 (2026-10-21). That session's logs and output files were not kept: the outputs, times and speeds pasted below come from it and cannot be re-checked. The llama.cpp, opencode and npm facts were re-checked against the sources at the end on 2026-11-05.
> **Not verified:** opencode 2.x (`@opencode/cli`), other GPUs, `--sleep-idle-seconds 900` end to end (the sleep test used 60).
> **Needs:** the b11292 Vulkan build unpacked in `<ROOT>/bin/vulkan/`, GGUF files in `<ROOT>/models/`, opencode installed from npm (`opencode-ai`), a login session (the unit is a user unit).
> **Time:** 30 min, plus model downloads. **Risk:** low. `gen-opencode.py --apply` rewrites `~/.config/opencode/opencode.json` after copying it to a timestamped backup. The router is a transient unit: nothing is enabled, and it does not survive a reboot, or a logout unless lingering is on for your user (`loginctl enable-linger`).

## Before you start

- Placeholders used below:
  - `<ROOT>` — the working directory. Mine is `~/work/llm-testing`; the scripts set it in one line each.
  - `<MODEL_ID>` — a section name from `models.ini`, for example `ornith-35b`. It becomes the model id in the API and in opencode.
- The Vulkan build answers from its own directory:
  ```console
  $ ~/work/llm-testing/bin/vulkan/llama-server --version
  version: 0.5.0-dev (build 11292, commit 90c908d06)
  built with GNU 11.4.0 for Linux x86_64
  ```
- opencode is the npm build:
  ```console
  $ opencode --version
  1.19.3
  ```
- RADV sees the iGPU:
  ```console
  $ vulkaninfo --summary 2>/dev/null | grep -E 'deviceName|driverInfo'
  	deviceName         = AMD Radeon Graphics (RADV REMBRANDT)
  	driverInfo         = Mesa 26.2.4
  	deviceName         = llvmpipe (LLVM 22.1.8, 256 bits)
  	driverInfo         = Mesa 26.2.4 (LLVM 22.1.8)
  ```
- Port 8080 is free (no output = nothing listens). Fedora also packages an older `llama-server` that must not be the one answering there:
  ```console
  $ ss -ltnH 'sport = :8080'
  $ rpm -qf /usr/bin/llama-server
  llama-cpp-b6153-3.fc44.x86_64
  ```
- opencode has a global config file. `gen-opencode.py` reads it unconditionally and fails without it. If it is missing, create the minimal one:
  ```bash
  mkdir -p ~/.config/opencode && test -f ~/.config/opencode/opencode.json || printf '{\n  "$schema": "https://opencode.ai/config.json"\n}\n' > ~/.config/opencode/opencode.json
  ```
- Backup: `gen-opencode.py --apply` copies `~/.config/opencode/opencode.json` to `opencode.json.bak-<YYYYmmdd-HHMMSS>` before writing. To be safe before the first run, take your own copy too: `cp ~/.config/opencode/opencode.json ~/opencode.json.before-router`.

## Steps

1. **List every model in a download manifest with its exact size in bytes.** `gen-opencode.py` reads `<ROOT>/download_models.sh` and treats a GGUF as complete only when its size matches. Each entry is one quoted line, `"<hf-repo> <file>.gguf <bytes>"`. Two of my entries:
   ```bash
   MODELS=(
     "unsloth/LFM2.5-8B-A1B-GGUF LFM2.5-8B-A1B-UD-IQ4_NL.gguf 4358844000"
     "ornith-ai/Ornith-1.5-35B-A3B-GGUF Ornith-1.5-35B-Q4_K_M.gguf 21713463040"
   )
   ```
   Expected: one line per GGUF you plan to serve. A file with no manifest entry passes on the magic bytes alone.

   > [!GOTCHA] A partial download already starts with `GGUF`
   > On 2026-10-21 `Qwen-AgentWorld-35B-A3B-UD-IQ3_S.gguf` was 236,871,911 bytes of 14,984,322,304 mid-download and its first four bytes were already `GGUF`. A magic-byte check alone would have offered it to opencode. The size match is what rejects it.

2. **Write `<ROOT>/serve/models.ini`.** One `[*]` section with what every model shares, then one section per model; the section name is the model id. Keys are `llama-server` options without the leading dashes (short forms allowed). Use absolute model paths. An excerpt of mine:
   ```ini
   ; llama-server router presets (b11292 Vulkan) for opencode. Section name = model id in the API.
   ; Only models whose file exists are exposed to opencode (gen-opencode.py checks).
   version = 1

   [*]
   ; whole context for one opencode session; fit picks -ngl / n-cpu-moe to fit the 680M pool
   c = 65536
   np = 1
   fit = on
   fitt = 1024
   ctk = q8_0
   ctv = q8_0
   fa = on
   jinja = true
   t = 8

   [lfm2.5-8b]
   model = /home/user/work/llm-testing/models/LFM2.5-8B-A1B-UD-IQ4_NL.gguf

   [ornith-35b]
   model = /home/user/work/llm-testing/models/Ornith-1.5-35B-Q4_K_M.gguf
   ; without it the CPU-side host buffer is too big for one RADV submission:
   ; "radv/amdgpu: Not enough memory for command submission" -> ErrorDeviceLost, even at -ngl 0
   no-host = true
   ; MTP head (blk.40.nextn): n-max 2 = 22.4 vs 21.0 t/s (+7%, ~67% accepted); n-max 3 slower. 2026-10-21
   spec-type = draft-mtp
   spec-draft-n-max = 2
   ; coding sampling, as lukesdevlab's [ornith-1.5-35b-coding]
   temp = 1.0
   top-p = 0.95
   min-p = 0.0
   presence-penalty = 0.0
   ```
   Expected: `c` in `[*]` is also the context limit opencode is told about (step 3). Write it as `c`: the script reads only that key, so `ctx-size = …` would give opencode a context of 0. `np = 1` gives one slot, so the whole 64k context goes to the one opencode session.

3. **Save `<ROOT>/serve/gen-opencode.py`.** The listing below is my script as of 21:16 on 2026-10-21, the revision steps 4 and 5 ran; later revisions also write per-model thinking variants, which this manual does not cover. It writes `active.ini` (the presets whose GGUF is complete) and builds opencode's `local` provider from the same list. It locates everything relative to its own path; no edits needed.
   ```python
   #!/usr/bin/env python3
   """Build serve/active.ini (presets whose GGUF exists) and the opencode `local` provider from it.

     gen-opencode.py            write active.ini, print diff of ~/.config/opencode/opencode.json
     gen-opencode.py --apply    also write opencode.json (backup next to it first)
     gen-opencode.py --ini-only write active.ini only (used by `oc`)
   """
   import difflib
   import json
   import re
   import shutil
   import sys
   import time
   from pathlib import Path

   SERVE = Path(__file__).resolve().parent
   SRC, ACTIVE = SERVE / "models.ini", SERVE / "active.ini"
   OPENCODE = Path.home() / ".config/opencode/opencode.json"
   BASE_URL = "http://127.0.0.1:8080/v1"
   OUTPUT_TOKENS = 8192


   def parse(path):
       """INI as ordered [(section or None, [lines])]; keeps comments, values as-is."""
       blocks, cur = [], (None, [])
       for line in path.read_text().splitlines():
           s = line.strip()
           if s.startswith("[") and s.endswith("]"):
               blocks.append(cur)
               cur = (s[1:-1], [])
           cur[1].append(line)
       blocks.append(cur)
       return blocks


   def kv(lines):
       out = {}
       for line in lines:
           s = line.split(";", 1)[0].strip()
           if "=" in s:
               k, v = (x.strip() for x in s.split("=", 1))
               out[k] = v
       return out


   def expected_sizes():
       """file name -> bytes, from download_models.sh (a partial download already starts with GGUF)."""
       script = SERVE.parent / "download_models.sh"
       out = {}
       for m in re.finditer(r'^\s*"\S+ (\S+\.gguf) (\d+)"', script.read_text(), re.M):
           out[m.group(1)] = int(m.group(2))
       return out


   def is_gguf(p, sizes):
       try:
           with open(p, "rb") as f:
               if f.read(4) != b"GGUF":
                   return False
           want = sizes.get(Path(p).name)
           return want is None or Path(p).stat().st_size == want
       except OSError:
           return False


   def main():
       blocks = parse(SRC)
       ctx = int(next(kv(l) for s, l in blocks if s == "*").get("c", 0))
       keep, models = [], {}
       sizes = expected_sizes()
       for sec, lines in blocks:
           if sec in (None, "*"):
               keep.append(lines)
               continue
           path = kv(lines).get("model", "")
           if not is_gguf(path, sizes):
               print(f"skip {sec}: no complete GGUF at {path}", file=sys.stderr)
               continue
           keep.append(lines)
           models[sec] = {
               "name": f"{sec} (local)",
               "tool_call": True,
               "limit": {"context": ctx, "output": OUTPUT_TOKENS},
           }
       if not models:
           sys.exit("no usable models in models.ini")
       ACTIVE.write_text("\n".join(l for lines in keep for l in lines) + "\n")
       print(f"active.ini: {', '.join(models)}", file=sys.stderr)
       if "--ini-only" in sys.argv:
           return

       old = OPENCODE.read_text()
       cfg = json.loads(old)
       cfg.setdefault("provider", {})["local"] = {
           "npm": "@ai-sdk/openai-compatible",
           "name": "llama.cpp local (680M)",
           "options": {"baseURL": BASE_URL},
           "models": models,
       }
       new = json.dumps(cfg, indent=2) + "\n"
       if new == old:
           print("opencode.json: unchanged", file=sys.stderr)
           return
       sys.stdout.writelines(difflib.unified_diff(
           old.splitlines(True), new.splitlines(True), str(OPENCODE), str(OPENCODE) + " (new)"))
       if "--apply" in sys.argv:
           bak = OPENCODE.with_name(f"opencode.json.bak-{time.strftime('%Y%m%d-%H%M%S')}")
           shutil.copy2(OPENCODE, bak)
           OPENCODE.write_text(new)
           print(f"written; backup {bak}", file=sys.stderr)


   if __name__ == "__main__":
       main()
   ```

4. **Dry-run it and read the diff.** Without flags it writes only `active.ini` and prints what it would change in `opencode.json`. This run used a copy of my `serve/` directory and `HOME` pointed at a scratch directory holding that minimal three-line `opencode.json`, so the diff shows only the new provider:
   ```console
   $ python3 -I serve/gen-opencode.py
   skip agentworld-35b-iq3: no complete GGUF at /home/user/work/llm-testing/models/Qwen-AgentWorld-35B-A3B-UD-IQ3_S.gguf
   skip nail-35b: no complete GGUF at /home/user/work/llm-testing/models/Nail-Qwen3.6-35B-A3B-MTP-UD-IQ4_XS.gguf
   skip nail-35b-iq3: no complete GGUF at /home/user/work/llm-testing/models/Nail-Qwen3.6-35B-A3B-MTP-UD-IQ3_S.gguf
   skip fablevibes-14b: no complete GGUF at /home/user/work/llm-testing/models/Qwen3.6-14B-A3B-FableVibes-Q4_K_M.gguf
   skip mellum-12b: no complete GGUF at /home/user/work/llm-testing/models/Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf
   skip swift-9b: no complete GGUF at /home/user/work/llm-testing/models/Swift-1.5-Ornith-9B-Q4_K_M.gguf
   active.ini: lfm2.5-8b, gemma4-26b, lfm2.5-24b, ornith-35b, agentworld-35b, swift-27b
   --- <SCRATCH_HOME>/.config/opencode/opencode.json
   +++ <SCRATCH_HOME>/.config/opencode/opencode.json (new)
   @@ -1,3 +1,62 @@
    {
   -  "$schema": "https://opencode.ai/config.json"
   +  "$schema": "https://opencode.ai/config.json",
   +  "provider": {
   +    "local": {
   +      "npm": "@ai-sdk/openai-compatible",
   +      "name": "llama.cpp local (680M)",
   +      "options": {
   +        "baseURL": "http://127.0.0.1:8080/v1"
   +      },
   +      "models": {
   +        "lfm2.5-8b": {
   +          "name": "lfm2.5-8b (local)",
   +          "tool_call": true,
   +          "limit": {
   +            "context": 65536,
   +            "output": 8192
   +          }
   +        },
   …
   +        "swift-27b": {
   +          "name": "swift-27b (local)",
   +          "tool_call": true,
   +          "limit": {
   +            "context": 65536,
   +            "output": 8192
   +          }
   +        }
   +      }
   +    }
   +  }
    }
   ```
   Expected: one `skip` line per preset whose file is missing or incomplete, an `active.ini:` line naming the rest, and a diff that adds or replaces only `provider.local`. Other keys in your `opencode.json` stay as they are.

5. **Apply it.**
   ```bash
   python3 -I <ROOT>/serve/gen-opencode.py --apply
   ```
   Expected: the same diff, then `written; backup <path>` on stderr. In the scratch copy from step 4:
   ```text
   active.ini: lfm2.5-8b, gemma4-26b, lfm2.5-24b, ornith-35b, agentworld-35b, swift-27b
   written; backup <SCRATCH_HOME>/.config/opencode/opencode.json.bak-20261021-213941
   ```
   At about 21:40 on 2026-10-21, `~/.config/opencode/` on my real config held two such backups. The first is from my first `--apply` at 13:49:05; the second was written at 20:41:34 and at that time differed from the live file only by `"lsp": true`:
   ```console
   $ ls -l ~/.config/opencode/opencode.json.bak-*
   -rw-------. 1 user user  515 Oct  5 16:08 /home/user/.config/opencode/opencode.json.bak-20261021-134905
   -rw-------. 1 user user 1684 Oct 21 13:49 /home/user/.config/opencode/opencode.json.bak-20261021-204134
   ```
   The name carries the time of the write; the listed date is the old file's, because the copy keeps its mtime. Re-run `--apply` whenever a download finishes: `oc` refreshes only `active.ini`, not opencode's model list.

6. **Write the per-session config `<ROOT>/serve/opencode-local.json`.** `oc` passes it as `OPENCODE_CONFIG`, so plain `opencode` keeps your global default model.
   ```json
   {
     "$schema": "https://opencode.ai/config.json",
     "model": "local/ornith-35b",
     "small_model": "local/ornith-35b",
     "lsp": true
   }
   ```
   Expected: `small_model` is the same model on purpose. opencode uses `small_model` for side tasks such as session titles; with `--models-max 1` a different model there would unload the main one on every such request. `"lsp": true` turns on opencode's built-in language servers, which are off when the key is absent (checked in Verify).

7. **Save the wrapper `<ROOT>/serve/oc`, make it executable and put it on `PATH`.** The `ROOT=` line is the one to change; on my box it is an absolute path. This is my `serve/oc` at commit 2a85e80 (file time 21:45 on 2026-10-21), verbatim except for the `ROOT=` line. The `oc` sessions in the field notes (20:26–20:56) ran earlier revisions of it; the health-check refusal and the SDKMAN block came later. The SDKMAN block has nothing to do with the router: it puts Java tools on `PATH` for opencode's non-interactive shell, and you can drop it. The "+32.5 s per session" in the compat comment is one n=1 A/B on one task; see the 20:30–20:36 field note before you rely on it.
   ```bash
   #!/usr/bin/env bash
   # opencode with the local llama.cpp router: start it if down, stop it when the last opencode exits.
   # Usage: oc [opencode args...]    Env: OC_KEEP=1 leaves the router running on exit,
   #        OC_CLAUDE=1 keeps Claude Code compat (CLAUDE.md, ~/.claude/skills) — see occ.
   set -uo pipefail
   ROOT="$HOME/work/llm-testing"
   UNIT=llm-router
   PORT=8080
   LOG="$ROOT/serve/router.log"

   if ! systemctl --user is-active -q "$UNIT"; then
     # a running bench would be skewed by the router, and the router by the bench
     if pgrep -x llama-bench >/dev/null && [ "${FORCE:-}" != 1 ]; then
       echo "oc: llama-bench is running (bench-all?); FORCE=1 to start the router anyway" >&2
       exit 1
     fi
     # opencode would otherwise talk to whatever else answers there (e.g. Fedora's b6153 llama-server)
     if ss -ltnH "sport = :$PORT" | command grep -q .; then
       echo "oc: port $PORT is taken by something other than $UNIT:" >&2
       ss -ltnpH "sport = :$PORT" >&2
       exit 1
     fi
     python3 -I "$ROOT/serve/gen-opencode.py" --ini-only || exit 1
     systemctl --user reset-failed "$UNIT" 2>/dev/null
     systemd-run --user --unit="$UNIT" -q \
       -p StandardOutput=append:"$LOG" -p StandardError=append:"$LOG" \
       "$ROOT/bin/vulkan/llama-server" --host 127.0.0.1 --port "$PORT" \
       --models-preset "$ROOT/serve/active.ini" --models-max 1 --sleep-idle-seconds 900 || exit 1
     for _ in $(seq 60); do
       curl -sf "http://127.0.0.1:$PORT/health" >/dev/null && break
       sleep 0.5
     done
     curl -sf "http://127.0.0.1:$PORT/health" >/dev/null || { echo "oc: router not healthy, see $LOG" >&2; exit 1; }
   elif ! curl -sf "http://127.0.0.1:$PORT/health" >/dev/null; then
     # the unit name is also used by benchmark/experiment jobs (so queued tests wait); do not touch them
     echo "oc: unit $UNIT is active but nothing answers on :$PORT (an experiment is using the name?)." >&2
     echo "    Wait for it to finish: systemctl --user status $UNIT" >&2
     exit 1
   fi

   # opencode's shell is non-interactive: the SDKMAN lazy-load functions from ~/.bashrc (mvn, mn, ...)
   # do not exist there, so put the current SDKMAN tools on PATH explicitly
   SDK="${SDKMAN_DIR:-$HOME/.sdkman}/candidates"
   for c in java maven mvnd gradle micronaut springboot jbang; do
     [ -d "$SDK/$c/current/bin" ] && PATH="$SDK/$c/current/bin:$PATH"
   done
   [ -d "$SDK/java/current" ] && export JAVA_HOME="$SDK/java/current"
   export PATH

   # default model only for oc sessions (Ornith); plain `opencode` keeps the global default.
   # small_model = same model: with --models-max 1 a different one would swap on every title request.
   # no Claude Code compat: measured +3.5k prompt tokens (17.2k vs 13.7k) = +32.5 s per session on Ornith;
   # `occ` (OC_CLAUDE=1) keeps it on for comparison; tui.json adds the local-LLM status (sidebar + home footer)
   claude_off=1
   [ "${OC_CLAUDE:-}" = 1 ] && claude_off=0
   OPENCODE_DISABLE_CLAUDE_CODE=$claude_off \
   OPENCODE_TUI_CONFIG="$ROOT/serve/opencode-tui/tui.json" \
   OPENCODE_CONFIG="$ROOT/serve/opencode-local.json" opencode "$@"
   rc=$?

   # process name: opencode.exe up to 1.18, opencode since 1.19
   if [ "${OC_KEEP:-}" != 1 ] && ! pgrep -x opencode >/dev/null && ! pgrep -x opencode.exe >/dev/null; then
     systemctl --user stop "$UNIT"
   fi
   exit "$rc"
   ```
   The `OPENCODE_TUI_CONFIG` line belongs to [ocmon](/ocmon-local-llm-status-for-opencode/), which documents it. Without ocmon, delete that line `[UNVERIFIED: oc without the OPENCODE_TUI_CONFIG line not run]`. The optional `occ` keeps opencode's Claude Code compatibility on, for comparison:
   ```bash
   #!/usr/bin/env bash
   # oc with opencode's Claude Code compat ON (reads CLAUDE.md, ~/.claude/skills): for comparing against plain oc.
   OC_CLAUDE=1 exec "$(dirname "$(readlink -f "$0")")/oc" "$@"
   ```
   Install both:
   ```bash
   chmod +x <ROOT>/serve/oc <ROOT>/serve/occ
   ln -s <ROOT>/serve/oc ~/.local/bin/oc
   ln -s <ROOT>/serve/occ ~/.local/bin/occ
   ```
   Expected: `occ` finds `oc` next to its own target through `readlink -f`, so the symlinks work.

   > [!NOTE] The router listens on 127.0.0.1 only
   > Every start logs `security: no API key is set and CORS allows all origins` and `router mode enabled - do not expose to untrusted environments`. `oc` binds `--host 127.0.0.1`; keep it that way.

8. **Start a session with `oc` and send the first prompt.** The router starts with no model loaded; the first request loads `ornith-35b`. From `<ROOT>/serve/router.log` of one such session:
   ```text
   0.00.118.715 I srv    operator(): Available models (5):
   0.00.118.718 I srv    operator():   [    preset] gemma4-26b
   0.00.118.719 I srv    operator():   [    preset] lfm2.5-24b
   0.00.118.719 I srv    operator():   [    preset] lfm2.5-8b
   0.00.118.720 I srv    operator():   [    preset] ornith-35b
   0.00.118.720 I srv    operator():   [    preset] swift-27b
   0.00.118.789 W srv  llama_server: security: no API key is set and CORS allows all origins (see https://github.com/ggml-org/llama.cpp/pull/25655)
   0.00.118.793 W srv  llama_server: security: router mode enabled - do not expose to untrusted environments
   0.00.118.794 I srv  llama_server: starting server in router mode. models will be automatically loaded on-demand
   0.00.120.996 I srv  llama_server: listening on http://127.0.0.1:8080
   0.00.120.999 W srv  llama_server: notice: server default port will be changed to :9931 in a future release (ref: https://github.com/ggml-org/llama.cpp/pull/26508)
   0.00.486.331 I srv          join: request for name=ornith-35b queued at position 1
   0.00.486.340 I srv  ensure_model: waiting until model name=ornith-35b is fully loaded...
   0.00.486.345 I srv  ensure_model: slot available, loading queued model name=ornith-35b
   0.00.486.475 I srv          load: spawning server instance with name=ornith-35b on port 34017
   …
   [34017] 0.21.081.884 I srv  llama_server: model loaded
   [34017] 0.21.081.890 I srv  llama_server: listening on http://127.0.0.1:34017
   ```
   Expected: `listening on http://127.0.0.1:8080` within the 30 s `oc` waits, then one `spawning server instance` per model switch. The child's own timestamps (`[34017] 0.21…`) count from the child's start: Ornith took 21 s to load here.

   The child is the router's own binary, not whatever `llama-server` is first on `PATH`. At commit `90c908d06`, `server-models.cpp` resolves `/proc/self/exe` (line 461) and spawns that; the log shows it as the first argument:
   ```text
   0.07.495.373 I srv          load: spawning server instance with args:
   0.07.495.382 I srv          load:   /home/user/work/llm-testing/bin/vulkan/llama-server
   ```
   So Fedora's `llama-cpp-b6153` package never runs as a child.

9. **Quit opencode.** When no process named `opencode` (or `opencode.exe`) is left, `oc` runs `systemctl --user stop llm-router`. The router log of the same session ends with:
   ```text
   3.36.594.919 I srv    operator(): operator(): cleaning up before exit...
   3.36.594.990 I srv    unload_all: stopping model instance name=ornith-35b
   [34017] 3.36.101.047 I srv    operator(): exit command received, exiting...
   [34017] 3.36.103.853 W ~llama_context:    Vulkan0 compute buffer size of 282.0000 MiB, does not match expectation of 275.5000 MiB
   3.37.349.512 I srv           run: instance name=ornith-35b exited with status 0
   ```
   Expected: `exited with status 0`. Start opencode with `OC_KEEP=1 oc` to leave the router running after exit.

   > [!WARNING] The unit name is global to your user session
   > Other jobs can run under the transient unit name `llm-router` too. On 2026-10-21 at 21:15 another experiment on the same box started its benchmark script under that name: the unit was `active`, port 8080 had no listener, and `curl -sf http://127.0.0.1:8080/health` exited 7. An earlier `oc` checked only `systemctl --user is-active`, so in that state it would have skipped the start and, on exit, stopped the other job's unit. The `oc` above refuses instead. In a live run at about 21:45 that day, with an experiment holding the unit and nothing on :8080, it printed this and exited 1 before starting opencode, and the unit stayed `active`:
   > ```text
   > oc: unit llm-router is active but nothing answers on :8080 (an experiment is using the name?).
   >     Wait for it to finish: systemctl --user status llm-router
   > ```
   > Use `llm-router` for the router only, or change `UNIT=`.

## Verify

- **Tool calls go through the router.** With the router up (`OC_KEEP=1 oc`, then quit), send a request that offers one tool. Request body `toolreq.json` (set `"model"` to a `<MODEL_ID>`):
  ```json
  {"model":"ornith-35b","max_tokens":300,
   "messages":[{"role":"user","content":"What is the weather in Kyiv right now? Use the tool."}],
   "tools":[{"type":"function","function":{"name":"get_weather","description":"Get current weather for a city",
     "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]}
  ```
  ```bash
  curl -s http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d @toolreq.json > tool-<MODEL_ID>.json
  ```
  Expected: `finish_reason` = `tool_calls` and a `get_weather` call with `{"city":"Kyiv"}`. My five saved responses (2026-10-21, 17:36–17:41), summarised by a short Python script (model, `finish_reason`, first tool call, `timings.predicted_per_second`):
  ```text
  lfm2.5-8b   tool_calls get_weather({"city":"Kyiv"}) 62.8 t/s
  lfm2.5-24b  tool_calls get_weather({"city":"Kyiv"}) 40.0 t/s
  gemma4-26b  tool_calls get_weather({"city":"Kyiv"}) 29.9 t/s
  ornith-35b  tool_calls get_weather({"city":"Kyiv"}) 21.7 t/s
  swift-27b   tool_calls get_weather({"city":"Kyiv"}) 5.2 t/s
  ```
  The t/s column is one short answer per model, not a benchmark.

- **LSP is on only through `opencode-local.json`.** A Go file with one type error (`var x int = "not an int"`) in `<SCRATCH>/lsp-go/`, checked with `HOME` pointed at a scratch directory whose `opencode.json` holds only `$schema`, once without and once with `OPENCODE_CONFIG`:
  ```console
  $ HOME=<SCRATCH_HOME> opencode debug lsp diagnostics main.go
  {}
  $ HOME=<SCRATCH_HOME> OPENCODE_CONFIG=~/work/llm-testing/serve/opencode-local.json opencode debug lsp diagnostics main.go
  {
    "<SCRATCH>/lsp-go/main.go": [
      {
        "range": {
          "start": {
            "line": 3,
            "character": 13
          },
          "end": {
            "line": 3,
            "character": 25
          }
        },
        "severity": 1,
        "code": "IncompatibleAssign",
        "codeDescription": {
          "href": "https://pkg.go.dev/golang.org/x/tools/internal/typesinternal#IncompatibleAssign"
        },
        "source": "compiler",
        "message": "cannot use \"not an int\" (untyped string constant) as int value in variable declaration"
      }
    ]
  }
  ```
  Expected: `{}` and the log line `all LSPs are disabled` without the key; a gopls diagnostic with it (`gopls` must be on `PATH`).

- **An idle model sleeps and wakes.** `--sleep-idle-seconds` frees the model and its KV cache after that many idle seconds; `GET /health`, `/props`, `/models` and `/metrics` neither wake it nor reset the timer (llama.cpp server README). My test ran the router with `--sleep-idle-seconds 60` instead of 900, loaded `lfm2.5-8b`, and logged GPU memory use (`gtt`, `vram`, MiB) and the model status from `GET /models` every 30 s:
  ```text
  20:21:10 loaded gtt=1812 vram=3719 [('lfm2.5-8b', 'loaded')]
  20:21:40 t+30s gtt=1815 vram=3713 [('lfm2.5-8b', 'loaded')]
  20:22:10 t+60s gtt=42 vram=463 [('lfm2.5-8b', 'sleeping')]
  20:22:40 t+90s gtt=59 vram=486 [('lfm2.5-8b', 'sleeping')]
  20:23:10 t+120s gtt=59 vram=525 [('lfm2.5-8b', 'sleeping')]
  wake request http=200 1.812305s
  20:23:12 after-wake gtt=1405 vram=4080 [('lfm2.5-8b', 'loaded')]
  ```
  Expected: status `sleeping` once the idle time passes, about 4.9 GiB freed for this 8B model, and the next real request answered after a reload (1.8 s here, Ornith takes longer). With `oc`'s 900 s the same happens after 15 minutes `[UNVERIFIED: 900 s timeout not observed end to end]`.

## Undo / rollback

- Stop the router now: `systemctl --user stop llm-router` (what `oc` runs on exit; exercised in every session above). The unit is transient: nothing to disable, gone at reboot (and at logout without lingering).
- Restore opencode's config from the backup `--apply` made, or from your own copy: `cp ~/opencode.json.before-router ~/.config/opencode/opencode.json`. Exercised on the scratch copy: after `--apply`, the backup was byte-identical to the file before it (`cmp`), and copying it back restored that file. Removing only `provider.local` from the JSON has the same effect. The `opencode.json.bak-*` files stay in `~/.config/opencode/` until you delete them.
- Remove the links: `rm ~/.local/bin/oc ~/.local/bin/occ`. `<ROOT>/serve/` and the models can stay; nothing outside them was changed.

## Failure modes

| Symptom (exact message) | Cause | Fix |
|---|---|---|
| Child log: `radv/amdgpu: Not enough memory for command submission.` then `ggml_vulkan: device lost on Vulkan0` and `vk::Queue::submit: ErrorDeviceLost`; opencode gets `{"error":{"code":500,"message":"model name=ornith-35b failed to load","type":"server_error"}}` | Ornith-1.5-35B Q4_K_M (21,713,463,040 bytes, 20.2 GiB) is larger than the 680M's 17.58 GiB pool, so part stays in a CPU-side host buffer that RADV cannot submit in one go. Fewer GPU layers did not help: nine launch variants failed the same way (see field notes). | `no-host = true` in the model's section (`--no-host`, "bypass host buffer"). It then loaded in 18 s standalone, 21 s through the router. |
| `error: invalid argument: --no-mmap` | b11292's `llama-server --help` has no `--no-mmap`; model loading is set with `-lm, --load-mode MODE` (`auto`, `none`, `mmap`, `mlock`, `mmap+mlock`, `dio`). | Use `load-mode = …`, or check `llama-server --help` of your build before copying flags from older guides. |
| `oc: unit llm-router is active but nothing answers on :8080 (an experiment is using the name?).` and `oc` exits 1 | Another job owns the transient unit name `llm-router` (see the warning in step 9). `oc` leaves that unit running. | Wait for the job (`systemctl --user status llm-router`), or set a different `UNIT=` in `oc`. |
| A model whose download has finished is in `active.ini` (`gen-opencode.py` prints it on the `active.ini:` line) but missing from opencode's model list | `oc` runs `gen-opencode.py --ini-only`; opencode's `provider.local` is written only by `--apply`. | `python3 -I <ROOT>/serve/gen-opencode.py --apply`. |

## Field notes

- **2026-10-21, Ryzen 7 PRO 6850U laptop (logs not kept), Fedora 44, kernel 7.2.10, Mesa 26.2.4, llama.cpp b11292 Vulkan, opencode 1.19.3** (times EEST; router starts and stops from `journalctl --user -u llm-router`):
  - 21:18–21:40 Re-checks for this page: `gen-opencode.py` dry run and `--apply` plus restore on a copy of `serve/` with a scratch `HOME` (steps 4–5, Undo), LSP with an empty global config (Verify), and the process name. On 1.19.3 the npm file is still `bin/opencode.exe`, but a running opencode shows as `opencode` (`pgrep -x opencode` matches, `pgrep -x opencode.exe` does not). `oc` checks both names; that 1.18 showed `opencode.exe` comes from the script's comment `[UNVERIFIED: not re-checked on 1.18]`.
  - 21:15 The `llm-router` unit name was taken by another experiment's benchmark script; nothing listened on 8080 (see the warning in step 9). I did not run `oc` in that state.
  - 20:51 and 20:53 Two router starts listed six presets, including `agentworld-35b`, whose GGUF was still downloading: the file was complete only at 21:15:24 (18,112,191,744 bytes). `gen-opencode.py` was changed at 20:53:47, and the next start, at 20:56, listed five. The script shown in step 3 rejects a file until its size matches the manifest; at 21:18 it skipped the IQ3_S file at 236,871,911 bytes of 14,984,322,304, which already began with `GGUF`.
  - 20:39 opencode logged `all LSPs are disabled` with my global config of that time; `"lsp": true` brought up gopls and its diagnostic for a deliberately wrong Go file.
  - 20:30–20:36 `occ` against `oc`, one run each, a one-line task in an empty directory on Ornith: 186.5 s with opencode's Claude Code compatibility on, 154.0 s off. The router runs 20:30:13–20:33:20 and 20:33:20–20:35:56 end at the same seconds as the two output files, so they are those sessions. Their first large prompt was 17,190 tokens with compatibility on and 13,660 off, read at about 136 t/s, which took 127 s and 100 s. Compatibility added about 3.5k tokens here. My `~/.claude/CLAUDE.md` alone is 9,789 tokens in Ornith's tokenizer (36,444 bytes), but opencode reads it only when `~/.config/opencode/AGENTS.md` does not exist (opencode rules docs). That file has existed on this box since 2026-10-05, so it was in both runs, and the 13.7k-token prompt with compatibility off includes it. One run per arm, so treat the 32.5 s as an example, not a measurement. `oc` sets `OPENCODE_DISABLE_CLAUDE_CODE=1`.
  - 20:26–20:30 An `oc` session on Ornith read a first large prompt of 17,227 tokens in 135.8 s, about the size of the compatibility-on run above. Quitting opencode stopped the router (step 9 log).
  - 20:23–20:26 MTP on Ornith, using the `blk.40.nextn` draft head in the GGUF (the plain run logs those tensors as unused): two runs per arm, 300 tokens each. Base 20.98 / 20.90 t/s. `spec-draft-n-max 2` 22.49 / 22.26 t/s: +6.9 % on the means, with 340 of 510 drafted tokens accepted (66.7 %). `n-max 3` 22.00 / 21.06 t/s. I kept n-max 2.
  - 20:21–20:23 Sleep test with a 60 s idle timeout (not `oc`'s 900): `lfm2.5-8b` went from 1812 MiB GTT + 3719 MiB VRAM to 42 + 463 MiB, and a request woke it in 1.81 s.
  - 17:36–17:43 Tool-call check through the router: all five models returned a `get_weather` tool call. Ornith would not load on Vulkan. Nine launch variants ended in `radv/amdgpu: Not enough memory for command submission` → `ErrorDeviceLost`. Their log files carry the run labels `ngl0`, `c4k-ngl10`, `c4k-ncmoe99`, `ncmoe20`, `ncmoe99`, `fit1024`, `lm-none`, `nooffload` and `nowarmup`, and the exact command lines were not logged. `--no-host` loaded Ornith in 18.2 s at 64k context, with 21.63 t/s generation on a 60-token answer. The CPU-only build's `llama-bench` gives 14.98 t/s (tg128) for the same file.
  - 13:49 First `gen-opencode.py --apply`: five presets became the `local` provider.

## Notes

- `--models-max 1` keeps one model in memory; asking for another unloads the current one (`evicting idle LRU name=… for a queued request` in the log). On a 17.58 GiB pool two of the 13–20 GiB models would not fit together anyway.
- The `[*]` section's `fit = on` lets llama-server choose GPU layers and CPU experts per model; I set no `ngl` per model.
- Swift-1.5-Qwen3.8-27B (dense) answered at 5.2 t/s in the tool check; the MoE models with 1–4B active parameters ran at 21–63 t/s.
- opencode 2 ships as a separate npm package, `@opencode/cli` (2.1.2 on 2026-10-21). Not tested here.
- [ocmon](/ocmon-local-llm-status-for-opencode/) reads this router's log and `GET /models` to show load, prompt progress and sleep state inside opencode; source at [gitlab.com/myrants-oss/ocmon](https://gitlab.com/myrants-oss/ocmon).
- The bench numbers for these models on this box: [A Core Ultra CPU lost to a 2022 laptop iGPU](https://tsvinc.dev/posts/core-ultra-vs-680m-llama-bench/).

## Sources

- [llama.cpp server README at commit 90c908d06 (b11292)](https://raw.githubusercontent.com/ggml-org/llama.cpp/90c908d06/tools/server/README.md) — router mode, `--models-preset` INI format, `--models-max`, sleep mode and its exempt endpoints.
- [server-models.cpp at 90c908d06](https://raw.githubusercontent.com/ggml-org/llama.cpp/90c908d06/tools/server/server-models.cpp) — `get_server_exec_path()` reads `/proc/self/exe` (line 461); line 576 stores that path as the binary every child is spawned with.
- [llama.cpp release b11292](https://github.com/ggml-org/llama.cpp/releases/tag/b11292) — `llama-b11292-bin-ubuntu-vulkan-x64.tar.gz`, 2026-09-30.
- [opencode config docs](https://opencode.ai/docs/config/) — `OPENCODE_CONFIG` precedence, `small_model`, `lsp`, `OPENCODE_TUI_CONFIG`.
- [opencode providers docs](https://opencode.ai/docs/providers/) — custom provider with `@ai-sdk/openai-compatible` and `options.baseURL`.
- [opencode LSP docs](https://opencode.ai/docs/lsp/) — "LSP is disabled by default".
- [opencode rules docs](https://opencode.ai/docs/rules/) — Claude Code compatibility fallbacks and `OPENCODE_DISABLE_CLAUDE_CODE`.
- [npm: opencode-ai](https://registry.npmjs.org/opencode-ai) and [npm: @opencode/cli](https://registry.npmjs.org/@opencode%2Fcli) — versions and publish dates.
- [lukesdevlab/youtube `model-configs.ini`](https://github.com/lukesdevlab/youtube/blob/85c9420a7f5f671ed37055329bafa63984f898a4/model-configs.ini), section `[ornith-1.5-35b-coding]` (revision of 2026-10-19) — the coding sampling values (`temp`, `top-p`, `min-p`, `presence-penalty`) in step 2.
- [ornith-ai/Ornith-1.5-35B-A3B-GGUF](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B-GGUF) — `Ornith-1.5-35B-Q4_K_M.gguf`, 21,713,463,040 bytes.
- My own notes on the 2026-10-21 runs: the `oc` revision at commit 2a85e80 of a private repository, and the record of the live `oc` refusal run at about 21:45 (unit active, :8080 silent, exit 1, unit left running). Not published, and not available for re-checking.