Core Features and Quick Inference

Qwen3-Coder-Next runs in non-thinking mode without generating <think></think> blocks, simplifying outputs for coding tasks. Load it via transformers (latest version) with torch_dtype="auto" and device_map="auto" for automatic hardware placement. Use chat template for prompts like "Write a quick sort algorithm," generating up to 65,536 new tokens. To avoid OOM errors, cap context at 32,768 tokens. Local apps like Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers support it out-of-the-box, enabling fast prototyping without cloud dependency.

Benchmarks (via images) show top performance on coding evals like SWE-Bench Verified, positioning it for agentic coding over general models.

Efficient Deployment for Production

Serve with OpenAI-compatible APIs using SGLang (>=v0.5.8, pip install 'sglang[app]>=v0.5.8') or vLLM (>=0.15.0, pip install 'vllm>=0.15.0'). For SGLang: python -m sglang.launch_server --model Qwen/Qwen3-Coder-Next --port 30000 --tp-size 2 --tool-call-parser qwen3_coder starts at http://localhost:30000/v1 with 256K context on 2 GPUs (tensor parallel). vLLM: vllm serve Qwen/Qwen3-Coder-Next --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder at http://localhost:8000/v1. Reduce to 32,768 context if startup fails due to memory limits, trading length for reliability on smaller hardware.

Agentic Workflows and Optimization

Define JSON tools (e.g., square_the_number function taking input_num: number) and call via OpenAI client against local endpoint: client.chat.completions.create(..., tools=tools). Model handles function calling natively without thinking tokens. For best results, sample at temperature=1.0, top_p=0.95, top_k=40 to balance creativity and focus in code generation. Full details in linked blog, GitHub, and docs; cite the Qwen3-Coder-Next tech report for production use.