mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
This implement support for OpenAI "gpt-oss" models. It's mostly a take over of #292 branch, but MoE code is kept inside examples folder cause it's still a bit warm. In particular the routing logic can break apart when there are undefined memory in the inputs, where it wasn't an issue for normal MLP. I only tested with the 20B version.
258 lines
7.8 KiB
258 lines
7.8 KiB