Harmony: Render gpt-oss Response Format in Rust/Python
OpenAI's harmony library encodes/decodes the harmony response format required for gpt-oss open-weight models in custom inference setups, mimicking the OpenAI API with multi-channel support for reasoning and tools.
Harmony Format Enables Structured gpt-oss Outputs
gpt-oss models demand the harmony response format for correct operation, as they were trained specifically on it. This format structures conversations, reasoning traces, and function calls using special tokens like <|start|>role<|message|> and <|end|>. It supports multiple channels (e.g., analysis, commentary, final) for separating chain-of-thought, tool preambles, and responses, plus tool namespaces and structured outputs with instruction hierarchies. Without harmony, gpt-oss fails; providers like HuggingFace, Ollama, or vLLM handle it automatically, but custom inference requires manual prompting.
Example system prompt specifies channels and tools:
<|start|>system<|message|>You are ChatGPT... Reasoning: high # Valid channels: analysis, commentary, final... Calls to 'functions' must go to commentary.<|end|>
<|start|>developer<|message|># Instructions Always respond in riddles # Tools ## functions namespace functions { type get_location = () => any; type get_current_weather = (_: {location: string...}) => any; }<|end|>
<|start|>user<|message|>What is the weather like in SF?<|end|>
<|start|>assistant
This mimics OpenAI's Responses API, easing transition for familiar users. See full guide at cookbook.openai.com/articles/openai-harmony.
Python and Rust Libraries for Encoding/Parsing
Install Python via pip install openai-harmony for high-level dataclasses mirroring chat structures (Role, Message). Rust core handles rendering/parsing via cargo add harmony, with full docs at docs/python.md and docs/rust.md.
Architecture: Rust crate (src/) with chat.rs for data structures, encoding.rs for logic, tiktoken tokenizer, and registry.rs for encodings. Python wrapper (python/openai_harmony/) uses pyo3 FFI bindings, producing openai_harmony.*.so. Repo includes tests/, test-data/, demo/harmony-demo, and AGENTS.md.
Local Development and Testing
Use maturin develop to build Rust extension into virtualenv, then pip install -e . for Python wrapper. Run Rust tests with cargo test, Python with pytest tests, or both via ./run_checks.sh. Optional: cargo fmt, ruff check .. Ensures Rust/Python parity for performance-critical rendering.