0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/examples/gpt_oss
Guillaume Wenzek e9b42f0ca8
examples: gpt oss (#329)
This implement support for OpenAI "gpt-oss" models. 

It's mostly a take over of #292 branch, but MoE code is kept inside
examples folder cause it's still a bit warm.
In particular the routing logic can break apart when there are undefined
memory in the inputs, where it wasn't an issue for normal MLP.

I only tested with the 20B version.
History 2025-09-25 14:03:37 +02:00
..
BUILD.bazel examples: gpt oss (#329) 2025-09-25 14:03:37 +02:00
GptOss.zig examples: gpt oss (#329) 2025-09-25 14:03:37 +02:00
main.zig examples: gpt oss (#329) 2025-09-25 14:03:37 +02:00
README.md examples: gpt oss (#329) 2025-09-25 14:03:37 +02:00

GptOss

Download the model (you can change 20b to 120b for the bigger version if you have the required hardware).

bazel run //tools/hf -- download openai/gpt-oss-20b --local-dir /tmp/models/openai/gpt-oss-20b --exclude metal/ original/

Run the model, use --@zml//runtimes:xxx flags to target the relevant platform for your hardware.

bazel run --config=release //examples/gpt_oss \
	--@zml//runtimes:cuda=true \
	--@zml//runtimes:cpu=false \
	--@zml//runtimes:tpu=false \
	--@zml//runtimes:rocm=false \
	-- --hf-model-path=/tmp/models/openai/gpt-oss-20b/ \
	--prompt='What are some spooky funfacts about animal for a Halloween party ?'