This PR uses the new release of rules_zig where we upstreamed all the
work from the zml rules_zig fork.
Not much changes from the POV of the public API except for `copts`
becoming `zigopts`.
Also, the global `c` module is only available to Zig modules that depend
on a non empty `cc_library`.
This is behavior that can probably be modified in `rules_zig` later on.
Finally, ZLS support had to be completely rethought to comply with the
change in shape of ZigModuleInfo providers and subsequent rules.
This PR is blocked until `rules_zig` exposes `ZigModuleInfo` for
`zig_binaries`.
This implement support for OpenAI "gpt-oss" models.
It's mostly a take over of #292 branch, but MoE code is kept inside
examples folder cause it's still a bit warm.
In particular the routing logic can break apart when there are undefined
memory in the inputs, where it wasn't an issue for normal MLP.
I only tested with the 20B version.
Much of the work was done by @gwenzek
Sadly we drop the torch loader in the process, to be added back later !
---------
Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
This PR intends to remove XLA as a bzlmod and transfer it as a non
bzlmod dep.
This is because this module will never be upstreamed as is, so keep it
private.
Also, we fetch llvm-raw and stablehlo from it, which is fine.
While there, dummify the various local_config XLA symbols to please the
imports, as we don't use those parts in ZML itself.
Closes#305
Modern bert implementation from Victor: @vctrmn
Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.
---------
Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
Following the new release of the VSCode Zig plugin, rework the code
completion logic to leverage a build runner. This is much lighter,
faster and more reliable.
**Changes**
- zml_utils: fix `AttributeError: type object 'ActivationCollector' has
no attribute 'CollectionOver'`
- zml_utils: remove dead code and comments
- docs(howto_torch2zml): add explanation on why `output` variable can be
`None`