Qwen released Qwen3.8-27B on August 14. It is a dense 27B model, with no mixture-of-experts routing: every parameter works on every token. It has thinking mode on by default, a 262,144 token context window, and an Apache 2.0 license.

Release-week posts on X called these the most important 20 gigabytes in human history. The same was said about DeepSeek R1. With less than 18 GB of weights now it performs frontier-tier coding benchmarks, on a license that lets you run them anywhere, forever.

On Arena’s WebDev leaderboard it ranks 18th of 128 models, a 27B open-weights model scoring within a point of GLM-5.2-max and ahead of Gemini 3.7 Flash, DeepSeek V4 Pro, and Claude Opus 4.8.

At 4-bit that is a home-server model. It serves me around 8 tokens per second on my AMD Ryzen 9 8945HS with the Radeon 780M iGPU.

This is my setup: the official llama.cpp Docker image with Vulkan, serving an OpenAI-compatible API.

GGUF

Unsloth’s Dynamic quants are the usual reference. The file that balances quality and size is for me Qwen3.8-27B-UD-Q4_K_XL.gguf at 17.6 GB. Download it to the models/ folder:

wget https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-UD-Q4_K_XL.gguf \
  -P ./models

The model is a vision-language model, but the GGUF splits the vision encoder into a separate projector file. If you want image input, download mmproj-F16.gguf too and pass it to the server with --mmproj. For text, the main file is enough.

compose.yml

services:
  llama:
    image: ghcr.io/ggml-org/llama.cpp:server-vulkan
    restart: unless-stopped
    command: >-
      -m /root/models/Qwen3.8-27B-UD-Q4_K_XL.gguf
      --alias local
      --host 0.0.0.0 --port 11434
      --ctx-size 131072
      --flash-attn on
      --cache-type-k q8_0
      --cache-type-v q8_0
      --jinja
      --spec-type draft-mtp
      --spec-draft-n-max 2
      --parallel 1
    devices:
      - /dev/dri
    ports:
      - "11434:11434"
    volumes:
      - ./models:/root/models

What each choice does:

• server-vulkan + /dev/dri: the Vulkan build runs on AMD and Intel GPUs, including iGPUs, with just the DRI device nodes passed through: no CUDA toolkit, no ROCm, no nvidia-container-toolkit. llama.cpp offloads layers to the GPU by default (--n-gpu-layers auto), so no offload flag is needed.
• --ctx-size 131072: half of the native 262,144. The context window costs RAM, and with 17.6 GB of weights the KV cache decides how much is left.
• --cache-type-k q8_0 / --cache-type-v q8_0: the KV cache at 8-bit instead of 16-bit, roughly halving its size with negligible quality loss. This is what keeps a large window affordable next to the weights.
• --jinja: serve with the model’s own chat template instead of a built-in fallback. For Qwen3.8 that is what makes thinking mode and tool calling behave as designed.
• --alias local + port 11434: the model answers to the name local, and the port is Ollama’s default, so any client already pointed at http://localhost:11434/v1 keeps working.
• --spec-type draft-mtp: speculative decoding with the model’s built-in multi-token prediction head, the free speed-up explained below. --spec-draft-n-max 2 drafts up to two tokens per pass, and --parallel 1 keeps a single stream, which is the regime the gain assumes: with several concurrent streams the advantage disappears.

Bring it up:

docker compose up -d

And test it at http://localhost:11434/.

The speed-up: MTP

Qwen trained multi-token prediction (MTP) heads into Qwen3.8, and unsloth kept them inside the GGUFs. llama.cpp can draft tokens with that built-in head and verify them with the main model: speculative decoding with no extra model file and no custom build. The --spec- lines in the compose turn it on; without them the MTP head loads and sits unused.

The gains hold across vendors, and they are hardware-dependent: on my Ryzen 9 8945HS the flag doubles the decode speed, from 4 to 8 tokens per second.