Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.
As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.
I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
Restore the machinery to emit individual functions and calls to them.
this should unblock composites, and generates cleaner MLIR in general
* functions are cached based on the input shapes including tags.
* donations/tags are propagated through function calls
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
Discovered use case:
- PJRT plugin attribute `stablehlo_current_version` is `1.7.8`
- `stablehloVersionFromCompatibilityRequirement(WEEK_12)` is `1.1.0`
- `stablehloGetCurrentVersion` is `1.7.6`
The target version for artifact serialization is `1.7.8` but should be
`1.7.6`.
XLA ref:
2f99455cdf (diff-7f335a7892)
- workspace: new module versions added to Bazel registry:
- deps/xla: 4663f04d26 (20241025.0-4663f04)
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
- deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)
- deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
- deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
Contains two bug fixes
1. values should be pushed to their block before we open a new block,
otherwise this lead to us assigning values to the wrong block.
* to detect future instance of this bug, and misusage of ZML apis I've
added a few more asserts.
* now every mlir block we create is either "open" or "hermetic", and we
check that hermetic block don't reference variables from outside the
block.
2. in `for_` make sure `first_step` is always used. This is a workaround
for #97. For some reason you need two nested loops to trigger the bug.
Note that this used to work, so I'm not sure what changed in how we
generate MLIR or on MLIR side.
to enable this I had to use the `.pad(.{ ._0 = ... })` syntax. I feel
this could solve a lot of usability usage we have, without having to
introduce too many `.operation1d` operators.
It's midly annoying to have to specify iota dtype everywhere to .i32.
In practice if people want to work with floating point range of number
they generally use arange.
Also `iota(x.shape(), 0).convert(.f32)` also work and isn't much work
than saying `iota(x.shape(), .f32, 0)`
For good measure I've added a fallback to .i64 when the given axis has
too many elements. I don't think this would cause much trouble, just
because the tensor that big aren't very common in the first place.
In the API I merged the "precision" and "algorithm" attributes into a
union because according to the spec,
precision must be default when algorithm is set, and precision=default
means "fast" when algorithm is not set.
https://github.com/openxla/stablehlo/blob/main/docs/spec.md#dot_general
For now it's not exposed to users because the API is a bit limited.
In particular only a subset of algorithm are supported by openxla, and
only a subset of that is supported by chips.
I was able to successfully change the accumulation type for dot on Cuda,
but want to experiment more before deciding what to expose.
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
We currently have an issue with our version of openxla not being able to
serialize the new "batching_dim" attribute of gather/scatter.
This is a version mismatch we should fix, but also it's bound to happen
again, so since the compilation can fail, I'd rather return an error in
that case.
In an async scenario, it is very easy to footgun yourself and block the
reactor, especially with Events.
Reintroduce pjrtx that deals with the specifics.
This PR adds the "cumulative sum" operator.
This is based on what jax does, which is leveraging reduce window
operator,
where for each n in [0..N] you compute the some of elements [0..n]
I'm not sure how this get optimized in complex programs, to be
investigated.
**BREAKING CHANGES**:
* `reduceWindow` is still pretty hard to used but I merged the
`padding_shape` and `padding_value` into one operator and made the
signature more explicit.
* this also lead me to change a bit how `maxPool1D` and `maxPool2D` take
arguments, using proper tuples instead of slices that need to be of
specific len.
* `pad` now take options using
* "tagged syntax":`x.pad(0, .{ .a = .{ .low = 1, .high = -1 }});`
* or the "AOS syntax": `x.pad(0, &.{.{}, { .low = 1, .high = -1 }});`
* the "SOA syntax" is **REMOVED**: `x.pad(0, .{ .low = &.{1}, .high =
&.{-1} });`
- Introduce sharding hints from the user code to specify how operations
should be split across user's devices mesh leveraging the HLO sharding
annotations
- Remove options arguments in `zml.module` including
`opts.add_donations_attributes`: it's for the function caching because inner function can't do donation, so we don't want to put the donation attributes there but for now we it's removed and we will do it differently next time around
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.
Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
some dead code or commented out code we should have removed before
release.
copyslice is also broken as per the commented out tests.
it will need to be reimplemented if we need it again.