0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

25 commits

Author SHA1 Message Date
Steeve Morin
032f2d7d59 WIP 2024-11-29 16:12:14 +01:00
Guillaume Wenzek
c03448363c llama: update to new rope 2024-11-28 15:02:13 +01:00
Guillaume Wenzek
db3675242a bazelrc: --config=debug 2024-11-28 14:50:17 +01:00
Steeve Morin
6f6dfe9fe5
runtimes/cuda: rework sandboxing and dynamic symbol renaming (#109)
This change reworks the sandboxing and dynamic symbol renaming for the
cuda runtime.

Switch to the jax-cuda-pjrt pre-built PJRT plugin until we fix the pjrt
artifacts build. Most notably because our build of the plugin didn't
include the right compute capabilities.

Take the opportunity to bump the `cuda=12.6.2` `cudnn=9.5.1`.

Fixes #19
2024-11-27 19:39:17 +01:00
Hugo Mano
db55475b17
examples/loader: missing stdx dep (#106) 2024-11-27 17:09:25 +01:00
Steeve Morin
43c4348de9
runtimes/neuron: add support for AWS neuron / trainium / inferentia 2 (#102)
This PR adds support for AWS Neuron / Trainium 1 / Inferentia 2.

It can be enabled via the `--@zml//runtimes:neuron=true` flag.

Everything should work as expected.

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-11-26 14:20:47 +01:00
Guillaume Wenzek
09da897fe8
zml: fix memory usage of sdpaMemEfficient (#90)
use `zml.ops.while` instead of `zml.ops.for` the problem with the latter is that it concatenate all intermediary results, and thus using lot of memory unless the concat was optimized away which wasn't always the case
2024-11-26 13:57:27 +01:00
Hugo Mano
d366f00aed
workspace: bump llvm / xla / stablehlo / pjrt plugins and restore disabled tests xla@4663f04 (#98)
- workspace: new module versions added to Bazel registry:
  - deps/xla: 4663f04d26 (20241025.0-4663f04) 
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
  - deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)  
  - deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
 This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
  - deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension 
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
2024-11-25 15:53:25 +01:00
Steeve Morin
db88158518
async: rework async runtime to use coroutines and cross thread signaling (#100)
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.

API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)

Mostly, also, it's pretty fast.
2024-11-22 17:56:22 +01:00
Corentin Godeau
4d1d03cfaa
examples: fixes related to recent async changes (#96)
Recent changes to the `async` module reverted `asyncGeneric` to `asyncc`
but examples were not updated
2024-11-20 15:34:38 +01:00
Steeve Morin
c65f07a123
zml: hotfixes for #89 (#91)
Due to miscommunication on my part, the tests were _not_ supposed to be
failing!
2024-11-15 18:03:00 +01:00
Steeve Morin
783a3cdeb8
zml: add stdx, rework async signature inference (#89)
This one is a chonky.

This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.

Also, clean the logs in the executables.
2024-11-15 17:12:07 +01:00
Corentin Godeau
03feafa85a
examples: small cleanup (#84)
Changes were made to `asynk` and not applied everywhere
2024-11-15 10:23:11 +01:00
Steeve Morin
ccd925309b
async: rename async_ to asyncc and add Generic slugs (#71)
`asyncc` is arguably nicer to write than `async_`, so use that.

Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
2024-11-01 17:36:08 +01:00
Steeve Morin
be510cbb6b
bazel,runtimes/rocm: use dynamic symbol import renaming (#70)
Currently, in ROCm, we use
`__attribute__((ifunc("_zml_rocm_resolve_dlopen")))` to make sure
`dlsym(RTLD_NEXT, "dlopen")` will resolve to the real `dlopen` and not
the always failing stub.

This is a bit brittle.

Instead, leverage `patchelf`'s `rename_dynamic_symbols` feature to
import a custom `dlopen` implementation, and forward to the real one
normally.
2024-11-01 17:33:57 +01:00
Corentin Godeau
c61ef6bd1d
Fix LSP in Neovim (#65)
`zig_exe_path` needs to be set for the LSP to work properly
2024-10-23 14:07:48 +02:00
Corentin Godeau
7a35d53ea3
Add Llama 3.1 70B Instruct model (#56)
This can be handy to test beefier models on powerful accelerators
2024-10-17 17:05:24 +02:00
Corentin Godeau
cb31e4a749
Llama weights sharding (#58)
This PR introduces sharding attributes to Llama weights to take
advantage of Tensor Parallelism
2024-10-17 17:05:10 +02:00
Guillaume Wenzek
895492dee4
zml.aio.torch: rename zml.aio.Value -> zml.aio.Metadata (#55)
Part of ongoing improvements to the torch loader

* rename zml.aio.Value -> zml.aio.Metadata
* simplify the types zml.aio.Metadata can have
* pickle `parse` function moved to `pickle.zig`
* add dedicated test case
* fix small bug in parse where `short_binstring` was converted to
`bin_string`
* pickle `evaluate` function now returns `[]const zml.aio.torch.Value`

note: I didn't modified the core logic.
2024-10-17 16:16:05 +02:00
Guillaume Wenzek
164a2f2c70
zml.aio.json: fix loading empty array (#50)
safetensor loader was segfaulting when loading metadata with an empty
json array.

I also rewrote some of the torch loader code to improve the readability.
Notably every slice in `ops.zig` is now const. I've also realized that
our strategy to load Pytorch tensor is not future proof see the warning
at: https://pytorch.org/docs/stable/storage.html
2024-10-11 13:26:04 +02:00
Hugo Mano
f26a022aa2
zml/sharding: user sharding constraints api and hlo integration (#46)
- Introduce sharding hints from the user code to specify how operations
should be split across user's devices mesh leveraging the HLO sharding
annotations
- Remove options arguments in `zml.module` including
`opts.add_donations_attributes`: it's for the function caching because inner function can't do donation, so we don't want to put the donation attributes there but for now we it's removed and we will do it differently next time around
2024-10-10 13:56:49 +02:00
Guillaume Wenzek
2251b1d815
zml: sharding (#26)
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.

Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
2024-10-01 21:12:25 +02:00
Guillaume Wenzek
1c7d7dacef
workspace: bump xla to enabled batching in gather (#21)
Since we were using an "old" version of XLA we where missing the
batching dims stablehlo attribute.
This was preventing the usage of batching in the gather operator.

Since I had already wrote the batching test before hand and was just
ignoring the compilation error,
this PR just bump xla and remove the error suppression code in tests.
2024-09-24 09:40:55 +02:00
Guillaume Wenzek
493f61d31b
zml/tensor: gatherValues (#12)
gather1d was quite limited by design.
gatherValues is a generalization of gather1d, that handles tuple of
indices and batching.

For each coordinate in `indices`,
`gatherValues` extracts a single value of the given tensor.

* axes_ is a single axis, or a tuple of axis: .b, or .{ .b, .c }
* indices is an integer tensor
* result is a tensor whose shape is similar to the input shape
where the gathered axes have been replaced by axes from 'indices'.

Some example input for the base case where we work on one axis:
- gatherValues(f:[a]->float, .a, ind:[n]->int)[n] == f[ind[n]]
- gatherValues(f:[a, b], .a, ind:[n])[n, b] == f[ind[n], b]
- gatherValues(f: [a,b,c], .{.b}, ind: [n,m])[a, n, m, c] == f[a, ind[n,
m], c]

If an axis in common between `self` and `indices`,
it is treated as a "batching" axis, meaning that semantically
the operator is doing a gatherValues one time per dimension of this
axis:
- gatherValues(f: [a,b,c], .{.b}, ind: [a,n])[a, n] == f[a, ind[a, n]]

It is an error to have an axis present in `self`, `axes_` and `indices`.

If several axes are passed, then the last axis of indices is treated as
coordinates:
- gatherValues(f: [a,b,c], .{.b, .c}, ind: [n,2])[a, n] == f[a,
ind[n][0], ind[n][1]]
- gatherValues(f: [a,b,c,d], .{.b, .c}, ind: [a, n,2])[a, n, d] == f[a,
ind[a, n][0], ind[a, n][1], d]

It is possible to use gatherValues without tags, but batching won't be
available.


> [!WARNING]
> The semantic of inserting the indices axis where we are dropping the
input axis is very useful in practice:
> `roll`, `nearest`, `resize` benefit greatly from it.
> But it doesn't always play nice with batching dimensions.
> In particular the following currently isn't accepted:
> `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .n = 8, .a = 10 })`
> because MLIR will produce `.{ .n = 8, .a = 10 }` following the indices
shape,
> while we produce `.{.a = 10, n=8}` following the semantic of
"replacing .b with axes from indices".
> If you do `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .a = 10, .n =
8 })` then both convention agrees and it's a non issue.
> I think this inconsistency is ok for now because it only happens when
you have both batch axes and extra-indices axes which act like second
set of batching dimensions.
> the issue is avoidable from the user side by eg broadcasting the
values along `.n`, or transposing one of the operand.

---------

Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2024-09-19 16:17:07 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00