Qwen3-Coder Usage

Qwen3-coder is the SOTA LLM developed by the Qwen team, Alibaba Cloud.

SGLang has supported Qwen3-coder already. The tool-parser shall be used with SGLang >= 0.4.9.post3.

pip install --upgrade pip
pip install uv
uv pip install "sglang[all]>=0.4.9.post3"
​

Ongoing optimizations are tracked in the Roadmap.

Launch Qwen3-Coder with SGLang

To serve Qwen3 model on H200 GPUs:

For BF16 model

python3 -m sglang.launch_server --model-path Qwen/Qwen3-Coder-480B-A35B-Instruct --tp 8 --tool-call-parser qwen3 --enable-ep-moe --mem-fraction-static 0.85
​

For FP8 model

python3 -m sglang.launch_server --model-path Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 --tp 8 --tool-call-parser qwen3 --enable-ep-moe --mem-fraction-static 0.85
​

Configuration Tips

• FP8 models : With --tp Loading failure is expected; switch to expert-parallel mode using --enable-ep-moe.
• Tool call: Add --tool-call-parser qwen3 for tool call parser.

Roadmap

• Initial Qwen3-Coder Support
• Initial Qwen3-Coder User Guide
• Streaming Tool Call Support