0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

41 commits

Author SHA1 Message Date
Hugo Mano
e5bc971c75
xla: bump to 9a77a88 (#357) 2025-11-12 11:55:48 +01:00
Guillaume Wenzek
21cd3ed532
mlir,pjrt,zml: expose all missing dtype (#328)
Adding the following missing data types:

- u2
- i1
- i2
- f4e2m1fn
- f8e3m4
- f8e4m3
- f8e8m0fnu

(ground truth is PJRT)

I've also realized the code in from/to Float32 wasn't correct and was
truncating to zero to often.
Hence the rewrite and bug fix.
2025-09-22 09:11:05 +02:00
Guillaume Wenzek
1480f77a55
zml: add the platform tag inside the Buffer (#326)
CPU buffers behave differently from Cuda/Rocm/TPU buffers,
and in particular they are addressable from main memory.
This was annoying and prevented us to add some safety checks that
weren't valid for CPU.

I'm adding a tag to buffers so that we know which target they refer to.

Also there is a bug in PJRT unitialized which doesn't respected the
memory. I've opened https://github.com/openxla/xla/pull/31292
but in the meantime I've a workaround in ZML that uses the regular
.fromHostBuffer.
(I've also a patched version of PJRT plugins in
https://github.com/zml/pjrt-artifacts/pull/41 but not merged yet)
2025-09-18 15:35:25 +02:00
Guillaume Wenzek
dda7f6c3f0
zml: remove old writer interface APIs (#324)
We still had a few old reader/writer left over.
I think we are good now
2025-09-18 11:36:23 +02:00
Guillaume Wenzek
df77b33d8e
async: remove asynk, asyncc, awaitt, await_ (#325) 2025-09-18 10:05:54 +02:00
Guillaume Wenzek
3f95bdd900
zml: callback api (#319)
Resuming work from @hugomano #244 
This is still a bit experimental.

The core API is here, but there are some caveats.

ZML should help with copying buffers to host memory (which is often
needed) but I'm not sure the current implementation using PJRT is good:
- does it allow graph capture
- what's the overhead ?
We should probably use directly cuMemcpy here instead of going through
PJRT.
Current implem should work with all plugins though.

This fix `tensor.print()`

I've also made a slightly more interesting examples/callback that
manually calls a cuda kernel. (there are other way to achieve this but
this shows you can do whatever you want with `zml.callback`

Note: 
the feature was originally named `custom_call` but it was conflicting
with existing bindings for xla builtin custom calls,
that are fully handled by the backend.
So I renamed it to `callback`

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-09-04 16:57:34 +02:00
Hugo Mano
10d3da2449
xla: bump to b3fbfee (#321)
- Temporarily disable libnvptxcompiler usage since it's not available in
the PjRT CUDA plugin v13.0
- Add nvshmem in the sandbox, required by the PjRT CUDA plugin
2025-09-03 14:24:04 +02:00
Guillaume Wenzek
8994a37792
bump runtimes/* code for 0.15 (#318)
Fix #316, close #304 

- Bump platform specific Zig code that doesn't get built in CI.
- Restore pytorch loader, using `std.fs.File` instead of `async.File`,
allowing to reuse `std.zip`
- Update `zig fmt` in CI to `0.15.1` and ran it on the codebase
(third_party excluded)
- Remove `stdx.io` which was pointing to remove
`std.io.BufferededReader`.

The Neuron target is still broken though:
`bazel build //runtimes/neuron:libneuronxla`

As well as the Cuda plugin in `--config=debug` mode:
 `bazel build --config=debug //runtimes/cuda:zmlxcuda`
2025-09-02 15:16:09 +02:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
Steeve Morin
24575c2232
workspace: switch from zig_cc_binary to zig_binary (#311)
Following the rules_zig bump that adds the `linkmode` setting, we can
use the C linker directly.

We can now remove the `zig_cc_binary` hack.
2025-08-21 13:12:15 +02:00
Steeve Morin
8279e43038
pjrt,zml: remove profiler (#306)
Closes #303

Progress towards #301
2025-08-19 16:04:43 +02:00
Corentin Kerisit
8274f9c2c0
runtimes/cuda: sandbox CUDA dependencies (#275)
This PR introduces CUDA sandboxing.

This essentially means:
- Removing all CUDA related dependencies off the leaf binary.
- Sandboxing the whole CUDA dependency graph.
- Mark dlopen direct dependencies of the PJRT shared library as NEEDED.
- Set the RPATH to the sandbox for all libraries that have sandbox
dependencies.
- Load the PJRT plugin from the sandbox.

And in the case of CUDA specifically for now:
- Enable usage of weak CUDA symbols without linking against CUDA
directly.
2025-07-16 11:33:48 +02:00
Guillaume Wenzek
ee1163a113
Bump rules_zig, add zig_srcs targets, cleanup build (#267)
This is the non-controversial part of #262 

It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
2025-07-15 14:18:15 +02:00
Corentin Godeau
0330e34788
pjrt: Bind PJRT_Client_CreateUninitializedBuffer (#266) 2025-06-19 09:11:53 +02:00
Corentin Godeau
3c2d319d20
zml/pjrt: added binding for PJRT_Device_MemoryStats (#264) 2025-06-18 18:32:15 +02:00
Guillaume Wenzek
657d4010a3
rename pjrt BufferType to match Zig and ZML conventions (#259) 2025-06-10 14:59:08 +02:00
Corentin Godeau
b0cfb64a0f
pjrt: added Executable.getCompiledMemoryStats (#250) 2025-06-06 19:03:09 +02:00
Corentin Godeau
88c627a309
Preliminary work for Custom Call (#247) 2025-06-06 11:24:21 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f (#238)
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily

XLA diff:
71c67e2a4f...cb67f2f7ce

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Guillaume Wenzek
f2f37e6d8d
zml: add some buffer / hostbuffer helpers (#232)
* buffer/hostbuffer utils
* more precise f32 -> bf16 conversion
* visit should not read irrelevant fields
* HostBuffer.data -> HostBuffer.bytes()
* zml.aio loadBuffers: type inference
* store expected input shapes in the executable
* zml.meta.visit support std.BoundedArray
* stdx.json.TaggedUnion
* protection against undefined float hyperparams
* avoid meta.TagPayload which can be quite slow
* improve logging
2025-05-20 15:08:35 +02:00
Corentin Godeau
cf038ccfa5
pjrt: bind PJRT_Device_AddressableMemories (#228) 2025-04-22 11:29:48 +02:00
Guillaume Wenzek
7db76fdaeb
pjrt: add ffi bindings for custom calls (#223)
---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Hugo Mano
2fd9516773
workspace: query, build and test (#187)
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.

As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
2025-02-12 10:40:20 +01:00
Guillaume Wenzek
53a43432e0
pjrt: profiler support std writer API (#185)
expose a more low-level function to customize where to write profile
reports
2025-02-11 16:47:28 +01:00
Steeve Morin
17723f5f88
pjrt,zml: add memory bindings (#180)
This preliminary PR binds PJRT memory endpoints and adds them to
`zml.Buffer`.

A follow up PR will properly integrate it inside `zml.Buffer`
2025-02-04 11:11:02 +01:00
Guillaume Wenzek
56ac7b1b07
pjrt: Profiler fixes (#146)
* allow i64 resource id
* reserve memory when creating array lists
2025-01-08 09:50:14 +01:00
Guillaume Wenzek
b360a92f34
zml: rewrite simple transpose (#142)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.

I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
2025-01-07 10:56:58 +01:00
Guillaume Wenzek
cf135bd428
zml: Reorganize the compilation output folders (#135)
* create one folder per target, then one sub-folder per mlir module.
* pass this folder to `xla_dump_to` so that all the xla files arrive
there.
* explicitly save the mlir and the serialized executable there

I'm also exposing James profile conversion tool into ZML as
`profiler.dumpDataAsJson(file)`
2024-12-17 16:55:11 +01:00
Guillaume Wenzek
e8fa410537
pjrt/zml: expose client creation flags (#129)
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125 

More settings are not exposed ATM, will follow up with #132 

3d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)

AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.

**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
2024-12-11 13:38:04 +01:00
Hugo Mano
2f70fa4a00
workspace: log message before return NotFound (#107)
This allow us to find which NotFound is raised
2024-11-27 17:21:34 +01:00
Hugo Mano
d366f00aed
workspace: bump llvm / xla / stablehlo / pjrt plugins and restore disabled tests xla@4663f04 (#98)
- workspace: new module versions added to Bazel registry:
  - deps/xla: 4663f04d26 (20241025.0-4663f04) 
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
  - deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)  
  - deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
 This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
  - deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension 
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
2024-11-25 15:53:25 +01:00
James Deal
0ea2e65419
PJRT: Profiling Output to JSON (#79)
Implements conversion of PJRT protobuf profiling output to JSON format.
2024-11-18 11:32:42 -06:00
Steeve Morin
783a3cdeb8
zml: add stdx, rework async signature inference (#89)
This one is a chonky.

This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.

Also, clean the logs in the executables.
2024-11-15 17:12:07 +01:00
Guillaume Wenzek
d33208a10e
Add custom call support, and tensor.print() (#82)
* restore the integration with PJRT custom call extension
* add a generic `zmlHostBufferCallback` custom call that copies the
received tensor to the host, and call a user supplied callback on it.
* add a default "print" method on `Tensor` that print the runtime value
of a given tensor
* print itself has very little code, it's easy for a user to define
their own print if they don't like ours.

limitations:
* only supported on Cuda atm
* pre allocates a buffer on the host to copy the content of the device
buffer,
this buffer won't be freed. There is exactly one buffer per "print" call
in the IR.
* it uses cuda api, maybe I should be using `createViewOfDeviceBuffer`
to make it less cuda specific
2024-11-15 11:22:01 +01:00
Steeve Morin
09f8506763
runtimes: add intermediate abstraction layer (#74)
Add a tiny abstraction layer between ZML and PJRT.

Most notably, to implement custom logic to decide wether or not we
should/can load a PJRT plugin.

For instance, `libtpu` will block the program trying to reach the GCP
metadata server, which doesn't exist outside GCP.

This also allows for comptime detection of wether a runtime is linked
with the program, and its dependencies.
2024-11-01 18:07:33 +01:00
Steeve Morin
a3a5a6336c
zml,pjrt: account for runtimes that not implementing Events (#72)
Some runtimes do not implement `PJRT_Event`. For those, we must dispatch
on our own thread pool for buffer copies and wait on events only if the
runtime returns them.

If there are no events, assume all operations are synchronous.
2024-11-01 17:38:14 +01:00
Guillaume Wenzek
2251b1d815
zml: sharding (#26)
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.

Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
2024-10-01 21:12:25 +02:00
Corentin Godeau
47674fbe31
PJRT: handle rank-0 special case in getDimensions (#33)
On some platforms, the returned pointer is null so we can't use it as a
slice, even if `num_dims` is 0.
2024-09-30 17:18:42 +02:00
Guillaume Wenzek
689751d7c4
zml: remove dead code (#27)
some dead code or commented out code we should have removed before
release.

copyslice is also broken as per the commented out tests.
it will need to be reimplemented if we need it again.
2024-09-26 11:03:41 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00