0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-18 18:57:29 +00:00
Commit graph

1,078 commits

Author SHA1 Message Date
Hugo Mano
e5bc971c75
xla: bump to 9a77a88 (#357) 2025-11-12 11:55:48 +01:00
Corentin Kerisit
06842784d3
workspace: use apple_support cc toolchains vs rules_cc (#355)
This is the recommended toolchain and is where all further improvements
is done.
2025-10-28 17:08:23 +01:00
Corentin Kerisit
e2c27b790b
workspace: default to LLVM backend for all zig compile actions (#356)
Use the new zigopt global setting from rules_zig to set -fllvm to all
zig copmile actions.

The Zig self-hosted backend is more fragile that we thought before. It
fails in numerous cases, in debug, but also in release sometimes. USe
LLVM backend until it is more stable.
2025-10-28 17:06:48 +01:00
Guillaume Wenzek
e58a5359f9
zml/aio: fix pickle loader for Zig 0.15.2 (#354)
zig 0.15.2 changed the meaning of `reader.takeDelimiterExclusive` to not
consume the separator. I rewrote the code to use
`takeDelimiterInclusive` since we want to consume the delimiter.
2025-10-22 17:43:44 +02:00
Guillaume Wenzek
13aa52b1d7
tools/hf: add hf_transfer implicit dependency (#344)
Fix #339
2025-10-20 12:28:45 +02:00
Corentin Kerisit
7d825caf2d
zml/aio/torch/pickle: disable test until implement is fixed (#352)
0.15.2 made this test fail with:

112/117 aio.torch.pickle.test.parse protocol 0...error(zml/aio): Unknow
pickle operator @enumFromInt(10), note we are only supporting pickle
protocol up to version 5.

cc @gwenzek
2025-10-17 21:19:08 +02:00
Corentin Kerisit
773253dfaf
zml: use .host_pinned memory for ROCm (#353)
It was failing with:

```
error(zml): Platform rocm doesn't have memory host_unpinned
```
2025-10-17 21:18:11 +02:00
Corentin Kerisit
a5b925f46b
examples/callback: fix compilation (#351)
Left over from when CI was not working most probably.
2025-10-17 19:06:17 +02:00
Corentin Kerisit
8ceacce067
runtimes/neuron: fix $ORIGIN now that zigopts can use Make Variables (#350)
zigopts is not subject to Make Variables expansion and so $ should be
escaped.
2025-10-17 19:06:05 +02:00
Corentin Kerisit
711ed46f5b
workspace: update zig version to 0.15.2 (#349) 2025-10-17 19:05:47 +02:00
Corentin Kerisit
f1dce257fe
all: provide -fllvm to all targets that requires it (#348)
Until zig x86 backend is fixed.
2025-10-17 19:05:29 +02:00
Corentin Kerisit
892766c652
zls: add missing cc toolchain (#347) 2025-10-17 19:04:39 +02:00
Corentin Kerisit
2613c1137c
workspace: use rules_zig archive_override pre 0.12.2 (#345)
Fixes shared_library and PJRT loading.
Fixes os/log.h non inclusion.
2025-10-17 19:04:17 +02:00
Guillaume Wenzek
2ff320253f
ci: handle no /tmp/zig-cache (#342) 2025-10-17 14:55:49 +02:00
Corentin Kerisit
1a263cf4e2
chore: remove unused target (#346) 2025-10-17 14:51:46 +02:00
Corentin Kerisit
319cf48b2c
pjrt/cpu: fix compilation without cpu (#343)
Zig modules that want to access the "c" module needs to have a non empty
CcCompilationContext for now.
So basically depending on an empty cc_library is not enough, it needs to
contain something (defines or headers or includes or wathever)
2025-10-17 14:51:29 +02:00
Corentin Kerisit
bc05231f30
workspace: use libxev from the BCR (#341)
This marks the last PR for getting out of our custom registry.

Module is now at https://registry.bazel.build/modules/libxev and @steeve
is a maintainer as well.
2025-10-16 11:48:09 +02:00
Guillaume Wenzek
876ca22ff2
add patches to http_deb_archive (#338)
add patches to http_deb_archive
add documentation for command_buffer_compatible
pinToCore
Optional head dim for Llama 3.1
2025-10-15 15:06:58 +02:00
Corentin Kerisit
36d583dd58
workspace: use upstreamed rules_zig instead of our fork (#340)
This PR uses the new release of rules_zig where we upstreamed all the
work from the zml rules_zig fork.

Not much changes from the POV of the public API except for `copts`
becoming `zigopts`.

Also, the global `c` module is only available to Zig modules that depend
on a non empty `cc_library`.
This is behavior that can probably be modified in `rules_zig` later on.

Finally, ZLS support had to be completely rethought to comply with the
change in shape of ZigModuleInfo providers and subsequent rules.

This PR is blocked until `rules_zig` exposes `ZigModuleInfo` for
`zig_binaries`.
2025-10-13 14:34:33 +02:00
Jean-Baptiste Dalido
53ad583089
workspace: add CI for cuda and rocm platforms, split jobs (#327) 2025-09-26 17:47:15 +02:00
0xC0
b04d483ecd
runtimes/cpu: fix darwin arm64 PJRT tarball checksum (#335)
Update sha256 to match v13.0.0 pjrt-cpu_darwin-arm64.tar.gz; unblocks
Bazel fetch on macOS arm64.
2025-09-26 11:25:00 +02:00
Guillaume Wenzek
25bd4d08ee
zml: cleanup some todos (#331) 2025-09-25 14:05:26 +02:00
Guillaume Wenzek
e9b42f0ca8
examples: gpt oss (#329)
This implement support for OpenAI "gpt-oss" models. 

It's mostly a take over of #292 branch, but MoE code is kept inside
examples folder cause it's still a bit warm.
In particular the routing logic can break apart when there are undefined
memory in the inputs, where it wasn't an issue for normal MLP.

I only tested with the 20B version.
2025-09-25 14:03:37 +02:00
Guillaume Wenzek
0312d6f678
zml: gather (#330)
Revamp the `gather` api to match `scatter`

(prerequesite of #329 )

## Gather new api

Now the syntax is to give a named list of indices. Each name must
correspond to one axis in the gathered tensor.
This match `scatter`, and avoid the user to have to concatenate indices
in the same order than the input axes,
which is error prone cause any screw up will compile but index at the
wrong offsets.

For 1 axis it doesn't change much: 

```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc}, .{ idx }, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx }, .{});
```

But for two it helps a lot:

```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc, .d}, .{ .concatenate(&.{idx_voc, idx_d}, .last)}, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx_voc, .d = idx_d }, .{});
```

Also adds a `gather_` that offers the same apis but with axis position
instead of names.
This one is more suitable inside library functions when dealing with
unknown names, making it complicated to create the "indices" struct.
I think we could generalize this pattern of offering "non-tagged" api
with an underscore suffix, simplifying the job of LSP.

## topk new API

I realized that most of the code calling topk was renaming the output
axis.
The idea is that logprob.topk(.voc) doesn't return you a tensor with voc
entry but only top_voc entries.
For the values it doesn't make much a difference but for the indices
it's embarrassing and it break usages of gather/scatter.

Now the user of `topK` can either use the unnamed api and pass a number,
of if using the name api must give a new name for the indexed axis.

## Yarn

Support Yarn rope embeddings. It's equivalent to the "llama3" embedding
except they do the interpolation in time space instead of freq space.
2025-09-25 12:29:46 +02:00
Guillaume Wenzek
21cd3ed532
mlir,pjrt,zml: expose all missing dtype (#328)
Adding the following missing data types:

- u2
- i1
- i2
- f4e2m1fn
- f8e3m4
- f8e4m3
- f8e8m0fnu

(ground truth is PJRT)

I've also realized the code in from/to Float32 wasn't correct and was
truncating to zero to often.
Hence the rewrite and bug fix.
2025-09-22 09:11:05 +02:00
Guillaume Wenzek
1480f77a55
zml: add the platform tag inside the Buffer (#326)
CPU buffers behave differently from Cuda/Rocm/TPU buffers,
and in particular they are addressable from main memory.
This was annoying and prevented us to add some safety checks that
weren't valid for CPU.

I'm adding a tag to buffers so that we know which target they refer to.

Also there is a bug in PJRT unitialized which doesn't respected the
memory. I've opened https://github.com/openxla/xla/pull/31292
but in the meantime I've a workaround in ZML that uses the regular
.fromHostBuffer.
(I've also a patched version of PJRT plugins in
https://github.com/zml/pjrt-artifacts/pull/41 but not merged yet)
2025-09-18 15:35:25 +02:00
Guillaume Wenzek
dda7f6c3f0
zml: remove old writer interface APIs (#324)
We still had a few old reader/writer left over.
I think we are good now
2025-09-18 11:36:23 +02:00
Guillaume Wenzek
df77b33d8e
async: remove asynk, asyncc, awaitt, await_ (#325) 2025-09-18 10:05:54 +02:00
Emmanuel Ferdman
03865b2f3d
Update bazel.sh source (#323)
## PR Summary
This small PR fixes the `bazel.sh` source.

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-09-16 11:39:51 +02:00
Guillaume Wenzek
988f2711c5
zml: loadById (#322)
We already assign a unique tensor id to each tensor object created by a
BufferStore.
But we weren't using the id to load find the corresponding buffer in the
store, instead relying on the name.

This is allow to decouple the shape of the model in the struct vs the
shape of the model inside the safetensor file.
I also moved the id logic out of Tensor and into BufferStore.

I also updated llama/main.zig to use the new logic. 

Bonus:
* fix to `zml.call` tag hashing leading to different MLIR for every
recompile.
* expose  Cpu device count to control the number of CPU "devices"
2025-09-11 10:28:16 +02:00
Guillaume Wenzek
3f95bdd900
zml: callback api (#319)
Resuming work from @hugomano #244 
This is still a bit experimental.

The core API is here, but there are some caveats.

ZML should help with copying buffers to host memory (which is often
needed) but I'm not sure the current implementation using PJRT is good:
- does it allow graph capture
- what's the overhead ?
We should probably use directly cuMemcpy here instead of going through
PJRT.
Current implem should work with all plugins though.

This fix `tensor.print()`

I've also made a slightly more interesting examples/callback that
manually calls a cuda kernel. (there are other way to achieve this but
this shows you can do whatever you want with `zml.callback`

Note: 
the feature was originally named `custom_call` but it was conflicting
with existing bindings for xla builtin custom calls,
that are fully handled by the backend.
So I renamed it to `callback`

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-09-04 16:57:34 +02:00
Hugo Mano
10d3da2449
xla: bump to b3fbfee (#321)
- Temporarily disable libnvptxcompiler usage since it's not available in
the PjRT CUDA plugin v13.0
- Add nvshmem in the sandbox, required by the PjRT CUDA plugin
2025-09-03 14:24:04 +02:00
Hugo Mano
5d6c04ec69
ci: revamp commit workflow (#320)
- Deduplicate workflows (introduced by
57fd9cda27)
- Fix linking issue on Linux since upb was introduced
- Fix duplicated `_start` symbol for some tests targets
- Temporarily use `-c opt` instead of `--config=debug` to bypass static
archive bug
2025-09-03 14:23:22 +02:00
Guillaume Wenzek
8994a37792
bump runtimes/* code for 0.15 (#318)
Fix #316, close #304 

- Bump platform specific Zig code that doesn't get built in CI.
- Restore pytorch loader, using `std.fs.File` instead of `async.File`,
allowing to reuse `std.zip`
- Update `zig fmt` in CI to `0.15.1` and ran it on the codebase
(third_party excluded)
- Remove `stdx.io` which was pointing to remove
`std.io.BufferededReader`.

The Neuron target is still broken though:
`bazel build //runtimes/neuron:libneuronxla`

As well as the Cuda plugin in `--config=debug` mode:
 `bazel build --config=debug //runtimes/cuda:zmlxcuda`
2025-09-02 15:16:09 +02:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
Steeve Morin
56ac3e27dd
zml: remove gguf and torch loaders (#317)
Torch loader might be reintroduced later, if the need arises.
2025-08-27 09:43:39 +02:00
Steeve Morin
5f3a2e74c3
workspace: buildifier pass, remove rules_uv, rework tools/hf deps (#315)
Title says it all
2025-08-22 16:37:36 +02:00
Steeve Morin
fc384eb43e
runtimes/neuron: fix neuron runtime (#314)
This PR fixes the neuron runtime with the following:

Proxy the PJRT Api method to enforce the client struct sizes since the
neuron PJRT plugin doesn't use `>=` but `==` to assert them, breaking
PJRT compatibility guarantees.
Fixes https://github.com/aws-neuron/aws-neuron-sdk/issues/1095

Reimplement `libneuronxla` in Zig to control neuronx-cc sandboxing and
invocation.

Implement a python bootstrapper in Zig to create a full blown
`neuronx-cc` executable, avoiding the infamous chicken and egg problem
of python executables boostrapping when sandboxed (due to fixed path
shebangs).

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-08-22 16:28:23 +02:00
Steeve Morin
0c796302e2
runtimes/cuda: implement zmlxcuda in Zig (#313) 2025-08-21 18:34:09 +02:00
Steeve Morin
b0978b4b2d
runtimes/rocm: implement zmlxrocm in Zig (#312)
Also, sandbox `amdgpu.ids` and restore safetensors json parsing.
2025-08-21 18:15:30 +02:00
Steeve Morin
24575c2232
workspace: switch from zig_cc_binary to zig_binary (#311)
Following the rules_zig bump that adds the `linkmode` setting, we can
use the C linker directly.

We can now remove the `zig_cc_binary` hack.
2025-08-21 13:12:15 +02:00
Steeve Morin
cd8a65334f
zml: remove nemo, sentencepiece loaders, remove zig-yaml (#310)
Progress towards #298
2025-08-20 00:15:55 +02:00
Steeve Morin
de4a0e0beb
third_party/xla: de-modularize xla (#309)
This PR intends to remove XLA as a bzlmod and transfer it as a non
bzlmod dep.

This is because this module will never be upstreamed as is, so keep it
private.

Also, we fetch llvm-raw and stablehlo from it, which is fine.

While there, dummify the various local_config XLA symbols to please the
imports, as we don't use those parts in ZML itself.

Closes #305
2025-08-19 23:35:01 +02:00
Steeve Morin
aa87c024c1 workspace: re-add .vscode folder
removed by mistake in #308
2025-08-19 19:44:20 +02:00
Steeve Morin
3b81c5630e
zml: remove examples workspace (#308)
Closes #302
2025-08-19 19:42:50 +02:00
Steeve Morin
2044cd2de5
zml: remove zig-protobuf, switch to upb (#307)
Closes #301
2025-08-19 16:14:54 +02:00
Steeve Morin
8279e43038
pjrt,zml: remove profiler (#306)
Closes #303

Progress towards #301
2025-08-19 16:04:43 +02:00
Guillaume Wenzek
9f2ccf8a54
package HF cli so users don't have to create Python venv (#293) 2025-08-01 16:42:29 +02:00
Corentin Godeau
5263a170d6
examples/llama: switch to --hf-model-path flag (#270)
Instead of giving config, model weights and tokenizer paths, rely on
`huggingface-cli` download.
2025-08-01 09:46:07 +02:00
Corentin Kerisit
52d36e1c7e
runtimes/neuron: sandbox neuron dependencies (#290)
The difference with other sandboxing PRs is that we also define a
trampoline PJRT so file which will prove itself useful to circumvent
PJRT version mismatch because of
https://github.com/aws-neuron/aws-neuron-sdk/issues/1095

Additionally, we artificially create a empty repository so that we can
sandbox the distroless deps inside the neuron module extension.

---------

Co-authored-by: EC2 Default User <ec2-user@ip-172-31-64-93.ec2.internal>
2025-07-25 11:35:04 +02:00