0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

38 commits

Author SHA1 Message Date
Guillaume Wenzek
21cd3ed532
mlir,pjrt,zml: expose all missing dtype (#328)
Adding the following missing data types:

- u2
- i1
- i2
- f4e2m1fn
- f8e3m4
- f8e4m3
- f8e8m0fnu

(ground truth is PJRT)

I've also realized the code in from/to Float32 wasn't correct and was
truncating to zero to often.
Hence the rewrite and bug fix.
2025-09-22 09:11:05 +02:00
Guillaume Wenzek
dda7f6c3f0
zml: remove old writer interface APIs (#324)
We still had a few old reader/writer left over.
I think we are good now
2025-09-18 11:36:23 +02:00
Guillaume Wenzek
8994a37792
bump runtimes/* code for 0.15 (#318)
Fix #316, close #304 

- Bump platform specific Zig code that doesn't get built in CI.
- Restore pytorch loader, using `std.fs.File` instead of `async.File`,
allowing to reuse `std.zip`
- Update `zig fmt` in CI to `0.15.1` and ran it on the codebase
(third_party excluded)
- Remove `stdx.io` which was pointing to remove
`std.io.BufferededReader`.

The Neuron target is still broken though:
`bazel build //runtimes/neuron:libneuronxla`

As well as the Cuda plugin in `--config=debug` mode:
 `bazel build --config=debug //runtimes/cuda:zmlxcuda`
2025-09-02 15:16:09 +02:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
Steeve Morin
24575c2232
workspace: switch from zig_cc_binary to zig_binary (#311)
Following the rules_zig bump that adds the `linkmode` setting, we can
use the C linker directly.

We can now remove the `zig_cc_binary` hack.
2025-08-21 13:12:15 +02:00
Steeve Morin
3b81c5630e
zml: remove examples workspace (#308)
Closes #302
2025-08-19 19:42:50 +02:00
Guillaume Wenzek
ee1163a113
Bump rules_zig, add zig_srcs targets, cleanup build (#267)
This is the non-controversial part of #262 

It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
2025-07-15 14:18:15 +02:00
Guillaume Wenzek
e15d3a9b90
remove usingnamespace from mlir (#204) 2025-06-10 15:51:30 +02:00
Corentin Godeau
8bddaffcb8
mlir/dialect/stablehlo: set defaults to operand_layouts and result_layouts (#252) 2025-06-06 18:20:25 +02:00
Steeve Morin
7ccfeaf524
workspace: various fixes and bumps (#229)
This PR add various fixes and bumps in the workspace, notably to prevent
dual builds for LLVM tools.
2025-04-22 16:12:26 +02:00
Guillaume Wenzek
7db76fdaeb
pjrt: add ffi bindings for custom calls (#223)
---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
Guillaume Wenzek
e9706222bf
add mini-DSL to create mlir common attributes and types (#208)
Leverages Zig 0.14 to simplify creation of mlir.Type and mlir.Attribute.
2025-03-31 17:31:53 +02:00
Hugo Mano
0376e5bb26
stablehlo: fix custom call optional attributes forwarding (#207)
Operand and result layouts are optionals
ref: 7639178ebd/xla/mlir_hlo/mhlo/IR/hlo_ops.td (L2568-L2569)
2025-03-18 17:58:49 +01:00
Hugo Mano
1c87992ca3
mlir: rework stablehlo custom call (#201)
Also add Triton example

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-03-17 16:03:07 +01:00
Steeve Morin
586a14da9f
mlir: rework DenseElementsAttribute to properly slice its inputs (#202)
In the process, make `.as()` not return an optional.
2025-03-17 14:04:09 +01:00
Corentin Godeau
97b99e7533
mlir: add MLIR index type (#198) 2025-03-14 23:27:36 +01:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Corentin Kerisit
9227fc9ad4
stablehlo: bump to head and use new dialect capi (#190)
This drastically reduce the number of build steps (from 3589 to 2553 steps)
2025-02-17 18:43:07 +01:00
Hugo Mano
2fd9516773
workspace: query, build and test (#187)
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.

As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
2025-02-12 10:40:20 +01:00
Steeve Morin
17723f5f88
pjrt,zml: add memory bindings (#180)
This preliminary PR binds PJRT memory endpoints and adds them to
`zml.Buffer`.

A follow up PR will properly integrate it inside `zml.Buffer`
2025-02-04 11:11:02 +01:00
Guillaume Wenzek
b360a92f34
zml: rewrite simple transpose (#142)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.

I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
2025-01-07 10:56:58 +01:00
Guillaume Wenzek
6578a3d11a
zml: emit func.call operations (#116)
Restore the machinery to emit individual functions and calls to them.

this should unblock composites, and generates cleaner MLIR in general

* functions are cached based on the input shapes including tags.
* donations/tags are propagated through function calls

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-12-05 17:45:07 +01:00
Guillaume Wenzek
d669bdc6bf
zml: cleanup module.zig (#115)
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
2024-12-05 17:33:19 +01:00
Hugo Mano
85685174c9
zml/pjrtx: use our own stablehlo version if plugin provided version is newer (#108)
Discovered use case:

- PJRT plugin attribute `stablehlo_current_version` is `1.7.8`
- `stablehloVersionFromCompatibilityRequirement(WEEK_12)` is `1.1.0`
-  `stablehloGetCurrentVersion` is `1.7.6`

The target version for artifact serialization is `1.7.8` but should be
`1.7.6`.

XLA ref:
2f99455cdf (diff-7f335a7892)
2024-11-27 18:29:04 +01:00
Hugo Mano
d366f00aed
workspace: bump llvm / xla / stablehlo / pjrt plugins and restore disabled tests xla@4663f04 (#98)
- workspace: new module versions added to Bazel registry:
  - deps/xla: 4663f04d26 (20241025.0-4663f04) 
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
  - deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)  
  - deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
 This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
  - deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension 
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
2024-11-25 15:53:25 +01:00
Guillaume Wenzek
15b0cdc9ad
Fix crash in for_ (#99)
Contains two bug fixes

1. values should be pushed to their block before we open a new block,
otherwise this lead to us assigning values to the wrong block.
* to detect future instance of this bug, and misusage of ZML apis I've
added a few more asserts.
* now every mlir block we create is either "open" or "hermetic", and we
check that hermetic block don't reference variables from outside the
block.

2. in `for_` make sure `first_step` is always used. This is a workaround
for #97. For some reason you need two nested loops to trigger the bug.
Note that this used to work, so I'm not sure what changed in how we
generate MLIR or on MLIR side.

to enable this I had to use the `.pad(.{ ._0 = ... })` syntax. I feel
this could solve a lot of usability usage we have, without having to
introduce too many `.operation1d` operators.
2024-11-22 17:46:47 +01:00
Guillaume Wenzek
282cf58b3c
zml: iota default to .i32 (#86)
It's midly annoying to have to specify iota dtype everywhere to .i32.

In practice if people want to work with floating point range of number
they generally use arange.
Also `iota(x.shape(), 0).convert(.f32)` also work and isn't much work
than saying `iota(x.shape(), .f32, 0)`

For good measure I've added a fallback to .i64 when the given axis has
too many elements. I don't think this would cause much trouble, just
because the tensor that big aren't very common in the first place.
2024-11-15 14:05:36 +01:00
Guillaume Wenzek
ecef330aa1
stablehlo: add DotAlgorithm support to stablehlo.dot_general (#85)
In the API I merged the "precision" and "algorithm" attributes into a
union because according to the spec,
precision must be default when algorithm is set, and precision=default
means "fast" when algorithm is not set.

https://github.com/openxla/stablehlo/blob/main/docs/spec.md#dot_general

For now it's not exposed to users because the API is a bit limited. 
In particular only a subset of algorithm are supported by openxla, and
only a subset of that is supported by chips.

I was able to successfully change the accumulation type for dot on Cuda,
but want to experiment more before deciding what to expose.

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-11-15 14:00:18 +01:00
Guillaume Wenzek
c32c0da46c
pjrtx: error instead of panic (#81)
We currently have an issue with our version of openxla not being able to
serialize the new "batching_dim" attribute of gather/scatter.
This is a version mismatch we should fix, but also it's bound to happen
again, so since the compilation can fail, I'd rather return an error in
that case.
2024-11-12 22:23:49 +01:00
Steeve Morin
fad7cc0876
zml: reintroduce pjrtx (#78)
In an async scenario, it is very easy to footgun yourself and block the
reactor, especially with Events.

Reintroduce pjrtx that deals with the specifics.
2024-11-05 19:25:50 +01:00
Guillaume Wenzek
cb0bcf2f7b
zml.tensor: add cumulativeSum, small refactor of maxPoolND (#64)
This PR adds the "cumulative sum" operator.

This is based on what jax does, which is leveraging reduce window
operator,
where for each n in [0..N] you compute the some of elements [0..n] 
I'm not sure how this get optimized in complex programs, to be
investigated.

**BREAKING CHANGES**:
* `reduceWindow` is still pretty hard to used but I merged the
`padding_shape` and `padding_value` into one operator and made the
signature more explicit.
* this also lead me to change a bit how `maxPool1D` and `maxPool2D` take
arguments, using proper tuples instead of slices that need to be of
specific len.
* `pad` now take options using 
     * "tagged syntax":`x.pad(0, .{ .a = .{ .low = 1, .high = -1 }});` 
* or the "AOS syntax": `x.pad(0, &.{.{}, { .low = 1, .high = -1 }});`
* the "SOA syntax" is **REMOVED**: `x.pad(0, .{ .low = &.{1}, .high =
&.{-1} });`
2024-11-04 09:45:32 +01:00
Hugo Mano
f26a022aa2
zml/sharding: user sharding constraints api and hlo integration (#46)
- Introduce sharding hints from the user code to specify how operations
should be split across user's devices mesh leveraging the HLO sharding
annotations
- Remove options arguments in `zml.module` including
`opts.add_donations_attributes`: it's for the function caching because inner function can't do donation, so we don't want to put the donation attributes there but for now we it's removed and we will do it differently next time around
2024-10-10 13:56:49 +02:00
Guillaume Wenzek
2251b1d815
zml: sharding (#26)
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.

Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
2024-10-01 21:12:25 +02:00
Corentin Godeau
4f1d6fec3e
Add StableHLO bindings for functions related to versioning (#34)
These functions can be used to serialize stablehlo in a portable way.
2024-09-30 17:18:53 +02:00
Guillaume Wenzek
6079d54927
Implement scatterSlices (#25) 2024-09-27 16:09:06 +02:00
Guillaume Wenzek
689751d7c4
zml: remove dead code (#27)
some dead code or commented out code we should have removed before
release.

copyslice is also broken as per the commented out tests.
it will need to be reimplemented if we need it again.
2024-09-26 11:03:41 +02:00
Guillaume Wenzek
714906265a
zml/nn: fix resize (#18)
Add more `refAllDecl` wich detected non compiling code.
I had to fix resizeBilinear / resizeBicubic.
(85 tests now)
2024-09-20 17:49:12 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00