0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-18 18:57:29 +00:00
Commit graph

1,078 commits

Author SHA1 Message Date
Steeve Morin
0f747f48b2
workspace: add Zed settings (#239)
This PR adds Zed configuration.
2025-05-29 16:46:44 +02:00
Steeve Morin
741c8be571
runtimes/cuda: add cuda.h to C namespace when using the CUDA runtimes (#236)
This allows for custom calls to potentially use CUDA functions.
2025-05-21 11:34:04 +02:00
Guillaume Wenzek
f2f37e6d8d
zml: add some buffer / hostbuffer helpers (#232)
* buffer/hostbuffer utils
* more precise f32 -> bf16 conversion
* visit should not read irrelevant fields
* HostBuffer.data -> HostBuffer.bytes()
* zml.aio loadBuffers: type inference
* store expected input shapes in the executable
* zml.meta.visit support std.BoundedArray
* stdx.json.TaggedUnion
* protection against undefined float hyperparams
* avoid meta.TagPayload which can be quite slow
* improve logging
2025-05-20 15:08:35 +02:00
Steeve Morin
35a5af6b88
third_party/sentencepiece: add missing protobuf_lite dependency (#235)
While we're here, bump it.
2025-05-20 11:53:52 +02:00
Guillaume Wenzek
09a3cb2d11
stdx.fmt: format slices (#231)
this is an improved version from what was implemented in "prettyPrinter"
but I had to dig a bit into the fmt internal to achieve it.
2025-05-19 22:39:56 +02:00
Steeve Morin
a566e396d2
workspace: add debug/release configurations and various stdx (#234)
This PR bumps rules_zig to add command line to the zig compilation in
order to enable debug and release configs from the bazel command line
(and bazelrc).

Most notably, this is useful for passing `-mcpu` flags to Zig
(especially for SIMD support like AVX512).

While we're there, add `stdx.time.Duration.hz()` and
`stdx.math.roundeven()`.
2025-05-19 21:26:28 +02:00
Guillaume Wenzek
bf79263919
Fix llama3 rope scaling (#230) 2025-05-09 14:32:15 +02:00
Steeve Morin
7ccfeaf524
workspace: various fixes and bumps (#229)
This PR add various fixes and bumps in the workspace, notably to prevent
dual builds for LLVM tools.
2025-04-22 16:12:26 +02:00
Corentin Godeau
cf038ccfa5
pjrt: bind PJRT_Device_AddressableMemories (#228) 2025-04-22 11:29:48 +02:00
Corentin Godeau
251e7d27db
zml/tensor: fix returned value in Tensor.toMemory (#227)
The `_output_memory_kind` in the returned result was not set correctly
2025-04-22 11:28:36 +02:00
Guillaume Wenzek
5aa1a22d9f
async should only use sync logging (#225)
currently using async.LogFn inside async runtime itself panics.
So I'm avoiding that by forcing async.zig logging to go through
std.log.defaultLog which is synchronous.

you can reproduce the crash by changing log level to .debug
2025-04-16 09:35:18 +02:00
Guillaume Wenzek
7db76fdaeb
pjrt: add ffi bindings for custom calls (#223)
---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
Corentin Godeau
6bcb2194d1
zml/ops: added wiring for operand output alias in zml.ops.triton (#222) 2025-04-08 10:52:00 +02:00
Corentin Godeau
c348da9446
async: fix due to xev bump (#221) 2025-04-08 10:51:51 +02:00
Corentin Godeau
347c46f86c
zml/aio: handle bool in struct in populateStruct (#220) 2025-04-08 10:51:42 +02:00
Corentin Godeau
1bd161d46c
async: TCP connect fixes (#219)
Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-04-08 10:51:33 +02:00
Guillaume Wenzek
e9706222bf
add mini-DSL to create mlir common attributes and types (#208)
Leverages Zig 0.14 to simplify creation of mlir.Type and mlir.Attribute.
2025-03-31 17:31:53 +02:00
Steeve Morin
1bdde64755
workspace: fix vscode-zig update (#217)
The extension checks that the ZLS path is absolute, else it will run
which to find it in the PATH.

Add a heading / so that the path looks absolute. This works because we
use ${workspaceFolder}.
2025-03-28 13:41:01 +01:00
Hugo Mano
8ec91a0edf
zml: support nvtx tracing (#215) 2025-03-27 15:26:36 +01:00
Hugo Mano
d0f5f0c7be
async: detect xev only if the API is dynamic (fix macOS) (#216)
ref: 94ed6af7b2/src/dynamic.zig (L44)
2025-03-27 15:26:28 +01:00
Victor Moin
3fd5c49c32
workspace: fix ZLS macOS x86_64 incorrect checksum (#214) 2025-03-27 15:13:28 +01:00
Hugo Mano
6be509e5b6
async: detect xev most preferable backend to use (#212)
Since change allow use to leverage io_uring instead of xev if the
backend is supported.
2025-03-25 16:41:58 +01:00
Steeve Morin
e0336dc9b3
workspace: fetch llvm, stablehlo and triton from xla, bump XLA (#206)
This enables making sure versions are the same across the board.

Also, import `triton` repo for later use.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-19 10:00:27 +01:00
Hugo Mano
0376e5bb26
stablehlo: fix custom call optional attributes forwarding (#207)
Operand and result layouts are optionals
ref: 7639178ebd/xla/mlir_hlo/mhlo/IR/hlo_ops.td (L2568-L2569)
2025-03-18 17:58:49 +01:00
Victor Moin
123082de35
runtimes: add macOS x86_64 (#175)
This PR showcases an initial implementation to run ZML on MacOS Intel
(x86_64). While Mac Intel is *deprecated*, this work allowed me to
contribute to [modernbert example](https://github.com/zml/zml/pull/149).
It relies on a custom
[pjrt-cpu_darwin-amd64.tar.gz](https://github.com/vctrmn/pjrt-artifacts/releases/download/v3.0.0-rc/pjrt-cpu_darwin-amd64.tar.gz),
which is not officially supported by the ZML team.

This PR __is not intended for merging__ but serves as a reference for
users who want to experiment with ZML on older Mac.

https://github.com/zml/zml/issues/105
2025-03-18 16:52:18 +01:00
Guillaume Wenzek
2458981020
zml: remove usingnamespace from floats.zig (#203)
I tried to remove usingnamespace from ZML and it's deps.

Even though `-fincremental` works now, it doesn't provide improved
compilation speed.
I guess because our compilation time are dominated by the linking of all
our C++ deps, and the linking isn't incremental ATM.

I split this in 3 pull requests,
I find the code a bit uglier everywhere, not sure if it's worth it.

* floats.zig this PR
* mlir.zig #204
* asyncio.zig #205
2025-03-18 16:19:02 +01:00
Hugo Mano
1c87992ca3
mlir: rework stablehlo custom call (#201)
Also add Triton example

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-03-17 16:03:07 +01:00
Steeve Morin
586a14da9f
mlir: rework DenseElementsAttribute to properly slice its inputs (#202)
In the process, make `.as()` not return an optional.
2025-03-17 14:04:09 +01:00
Corentin Godeau
97b99e7533
mlir: add MLIR index type (#198) 2025-03-14 23:27:36 +01:00
Steeve Morin
a93c7631e5
workspace,third_party/rules_zig: minor fixes for zls+zig 0.14.0 (#200)
ZLS now requires `--version` to be the first param, account for that.

Also, added missing keys to the BuildConfig object used for code
completion.
2025-03-14 13:42:10 +01:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Steeve Morin
5bb1d46640
workspace: rework runtimes sandboxing, bump PJRT plugins, cuda=12.8 and ROCm=6.3.4 (#199)
This PR significantly reworks the runtimes sandboxing to leverage
rules_distroless apt resolution, bump pjrt-artifacts=7.0.0 which inclues
Triton ROCm fixes.

Finally, bump CUDA=12.8 cuDNN=9.8, ROCm=6.3.4
2025-03-13 22:17:41 +01:00
jaco-bro
7e62b6551c
workspace: update rules_cc=0.1.1 and bazel=8.1.1 (#196)
This PR updates the `rules_cc` dependency from `0.1.0` to `0.1.1` and
the Bazel version from `8.0.1` to `8.1.1`.

### Changes:
1. Updated `rules_cc` version in `zml/MODULE.bazel` from 0.1.0 to 0.1.1:
    ```bazel
    bazel_dep(name = "rules_cc", version = "0.1.1")
    ```
2. Updated the Bazel version in `zml/examples/.bazelversion` from
`8.0.1` to `8.1.1`.

### Context:
1. Initially, running `bazel run -c opt //mnist` failed due to a Bazel
version mismatch (required `8.0.1` but the currently brew-installable
version is `8.1.1`). This was resolved by updating `.bazelversion`.
2. After fixing the Bazel version, the build failed again this time due
to a yanked version of `rules_cc` (`0.1.0`).

> ERROR: Error computing the main repository mapping: Yanked version
detected in your resolved dependency graph: rules_cc@0.1.0, for the
reason: rules_cc 0.1.0 is yanked due to incompatible change (prematurely
removing cc_proto_library from defs.bzl), please upgrade to 0.1.1.
> Yanked versions may contain serious vulnerabilities and should not be
used. To fix this, use a bazel_dep on a newer version of this module. To
continue using this version, allow it using the --allow_yanked_versions
flag or the BZLMOD_ALLOW_YANKED_VERSIONS env variable.

3. Updating `rules_cc` to `0.1.1` resolved this issue.

### Testing:
1. Verified the update by running:
    ```zsh
    git clone https://github.com/jaco-bro/zml
    cd zml/examples
    bazel run -c opt //mnist
    ```
2. The build and execution succeeded without errors.

    ```
    info(mnist): ✅ RECOGNIZED DIGIT:
               +-------------+
               |     ###     |
               |    #   #    |
               |   #     #   |
               |   #     #   |
               |   #     #   |
               |    #   #    |
               |     ###     |
               +-------------+
    ```
2025-03-03 16:46:50 +01:00
Steeve Morin
cb1010948b
workspace: bump to libxev=20250222.0-07bcffa (#197)
This introduces `Dynamic`, which will switch between IO Uring and Epoll
in linux depending on availablity.
2025-03-03 16:46:34 +01:00
Guillaume Wenzek
a672bb8247
examples: add modernBERT (#192)
Modern bert implementation from Victor: @vctrmn

Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.

---------

Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
2025-02-27 18:07:42 +01:00
Hugo Mano
d712ab7534
workspace: fix libxev version (#195) 2025-02-21 12:48:00 +01:00
Corentin Kerisit
44b74dea40
workspace: fix mistakes in modules bumps (#191)
Fix missing = in stbalehlo integrity.
Changes back - by . in xla module name and folders.
Correctly depend on `xla@20250204.1-6789523`
2025-02-18 15:54:56 +01:00
Corentin Kerisit
9227fc9ad4
stablehlo: bump to head and use new dialect capi (#190)
This drastically reduce the number of build steps (from 3589 to 2553 steps)
2025-02-17 18:43:07 +01:00
Corentin Godeau
744e0d2843
zml/ops.zig: Added zml.ops.case operation (#188)
This can be used to select which branch will be run at runtime.

It wraps the `stablehlo.case` operation.
2025-02-13 20:47:53 +01:00
Hugo Mano
2fd9516773
workspace: query, build and test (#187)
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.

As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
2025-02-12 10:40:20 +01:00
Guillaume Wenzek
1c2580b3c2
zml: FnCache should handle non-tensor values for less confusing behavior (#186)
Currently if you used the FnCache (through zml.call) on a function with
non tensor fields you would get a struct with undefined memory.
This wasn't nice. So now we copy the value of the function returned by
the first zml.call, and use this as base for future calls, reusing
non-tensor fields, and adding correct mlir.values to the previous
tensors.
This is easy because now zml.call only supports fixed size structs (no
slice/pointer)

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2025-02-11 16:49:05 +01:00
Guillaume Wenzek
53a43432e0
pjrt: profiler support std writer API (#185)
expose a more low-level function to customize where to write profile
reports
2025-02-11 16:47:28 +01:00
Hugo Mano
ceec8a8c9a
workspace: bump xla to version 20250204.0-6789523 (#184)
Also improves Bazel 8 compatibility

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-02-07 15:36:15 +01:00
Guillaume Wenzek
5de2adcec3
Fix llama generations (#181)
Current implementation of llama is discarding the first outputted token,
and feeding the last prompt token a second time in the model.

Due to how self attention works this wasn't too disturbing for attention
but it was technically wrong.
Also it was doing a bit extra compute for nothing.

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2025-02-06 14:10:02 +01:00
Steeve Morin
86dc444964
workspace: MODULE.bazel cleanup (#183)
Title says it all !
2025-02-05 17:34:06 +01:00
Steeve Morin
cbd9e50a79
workspace: bump libxev to 20252401.0-31eed4e (#182)
Includes:
- https://github.com/mitchellh/libxev/pull/128
- https://github.com/mitchellh/libxev/pull/141
2025-02-05 13:40:03 +01:00
Steeve Morin
17723f5f88
pjrt,zml: add memory bindings (#180)
This preliminary PR binds PJRT memory endpoints and adds them to
`zml.Buffer`.

A follow up PR will properly integrate it inside `zml.Buffer`
2025-02-04 11:11:02 +01:00
Rene Schallner
a9057737f3
llama: clean up llama3 prompt template encoding. (#172)
There was an unnecessary re-encoding of the newline string, and we added
a newline at the end.
2025-02-03 12:10:19 +01:00
Guillaume Wenzek
f1d0c64db3
Restore tinyllama models (#173)
This is a follow up on #162 

* restore homemade tokenizer, integrate it with the new zml/tokenizer
library. This will only be used for working with tinyllama, I could also
make it work to support GGUF if needed (couple lines of code).
* this allows us to use the original tinyllama files from karpathy repo
which generates actual sentences. This is important when working on
modeling because subtle bugs can degrade the output quality, which is
not detectable if the output is crap to begin with.
* fix casing of zml/tokenizer api: fromFile, tokenToId. Let's stick to
the Zig style guide. Function written in Rust follows rust convention,
but the Zig bindings should follow Zig convention.
* make tokenizer/main.zig a proper tokenizer test tool reusing
llama/test_tokenizer.zig

~~use native_binary instead of cc_binary~~
Reverted because apparently native_binary break the runfiles and
therefore the cuda sandbox
2025-02-03 12:06:33 +01:00
Corentin Kerisit
71fa3764d1
bazel: use prebuilt protoc binaries (#178)
This removes ~1300 build steps by depending on prebuilt binaries.

Note that this PR is blocked by
https://github.com/aspect-build/toolchains_protoc/pull/44 since
`rules_proto` depends on `protobuf@29.1` and
https://github.com/aspect-build/toolchains_protoc doesn't expose
`protoc@29.1` and we can't pin module versions unless adding an entry in
the local BCR which I think is overkill for this.
2025-01-31 23:12:04 +01:00