0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

23 commits

Author SHA1 Message Date
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f (#238)
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily

XLA diff:
71c67e2a4f...cb67f2f7ce

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Guillaume Wenzek
f2f37e6d8d
zml: add some buffer / hostbuffer helpers (#232)
* buffer/hostbuffer utils
* more precise f32 -> bf16 conversion
* visit should not read irrelevant fields
* HostBuffer.data -> HostBuffer.bytes()
* zml.aio loadBuffers: type inference
* store expected input shapes in the executable
* zml.meta.visit support std.BoundedArray
* stdx.json.TaggedUnion
* protection against undefined float hyperparams
* avoid meta.TagPayload which can be quite slow
* improve logging
2025-05-20 15:08:35 +02:00
Corentin Godeau
cf038ccfa5
pjrt: bind PJRT_Device_AddressableMemories (#228) 2025-04-22 11:29:48 +02:00
Guillaume Wenzek
7db76fdaeb
pjrt: add ffi bindings for custom calls (#223)
---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Hugo Mano
2fd9516773
workspace: query, build and test (#187)
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.

As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
2025-02-12 10:40:20 +01:00
Guillaume Wenzek
53a43432e0
pjrt: profiler support std writer API (#185)
expose a more low-level function to customize where to write profile
reports
2025-02-11 16:47:28 +01:00
Steeve Morin
17723f5f88
pjrt,zml: add memory bindings (#180)
This preliminary PR binds PJRT memory endpoints and adds them to
`zml.Buffer`.

A follow up PR will properly integrate it inside `zml.Buffer`
2025-02-04 11:11:02 +01:00
Guillaume Wenzek
56ac7b1b07
pjrt: Profiler fixes (#146)
* allow i64 resource id
* reserve memory when creating array lists
2025-01-08 09:50:14 +01:00
Guillaume Wenzek
b360a92f34
zml: rewrite simple transpose (#142)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.

I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
2025-01-07 10:56:58 +01:00
Guillaume Wenzek
cf135bd428
zml: Reorganize the compilation output folders (#135)
* create one folder per target, then one sub-folder per mlir module.
* pass this folder to `xla_dump_to` so that all the xla files arrive
there.
* explicitly save the mlir and the serialized executable there

I'm also exposing James profile conversion tool into ZML as
`profiler.dumpDataAsJson(file)`
2024-12-17 16:55:11 +01:00
Guillaume Wenzek
e8fa410537
pjrt/zml: expose client creation flags (#129)
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125 

More settings are not exposed ATM, will follow up with #132 

3d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)

AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.

**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
2024-12-11 13:38:04 +01:00
Hugo Mano
2f70fa4a00
workspace: log message before return NotFound (#107)
This allow us to find which NotFound is raised
2024-11-27 17:21:34 +01:00
Hugo Mano
d366f00aed
workspace: bump llvm / xla / stablehlo / pjrt plugins and restore disabled tests xla@4663f04 (#98)
- workspace: new module versions added to Bazel registry:
  - deps/xla: 4663f04d26 (20241025.0-4663f04) 
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
  - deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)  
  - deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
 This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
  - deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension 
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
2024-11-25 15:53:25 +01:00
James Deal
0ea2e65419
PJRT: Profiling Output to JSON (#79)
Implements conversion of PJRT protobuf profiling output to JSON format.
2024-11-18 11:32:42 -06:00
Steeve Morin
783a3cdeb8
zml: add stdx, rework async signature inference (#89)
This one is a chonky.

This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.

Also, clean the logs in the executables.
2024-11-15 17:12:07 +01:00
Guillaume Wenzek
d33208a10e
Add custom call support, and tensor.print() (#82)
* restore the integration with PJRT custom call extension
* add a generic `zmlHostBufferCallback` custom call that copies the
received tensor to the host, and call a user supplied callback on it.
* add a default "print" method on `Tensor` that print the runtime value
of a given tensor
* print itself has very little code, it's easy for a user to define
their own print if they don't like ours.

limitations:
* only supported on Cuda atm
* pre allocates a buffer on the host to copy the content of the device
buffer,
this buffer won't be freed. There is exactly one buffer per "print" call
in the IR.
* it uses cuda api, maybe I should be using `createViewOfDeviceBuffer`
to make it less cuda specific
2024-11-15 11:22:01 +01:00
Steeve Morin
09f8506763
runtimes: add intermediate abstraction layer (#74)
Add a tiny abstraction layer between ZML and PJRT.

Most notably, to implement custom logic to decide wether or not we
should/can load a PJRT plugin.

For instance, `libtpu` will block the program trying to reach the GCP
metadata server, which doesn't exist outside GCP.

This also allows for comptime detection of wether a runtime is linked
with the program, and its dependencies.
2024-11-01 18:07:33 +01:00
Steeve Morin
a3a5a6336c
zml,pjrt: account for runtimes that not implementing Events (#72)
Some runtimes do not implement `PJRT_Event`. For those, we must dispatch
on our own thread pool for buffer copies and wait on events only if the
runtime returns them.

If there are no events, assume all operations are synchronous.
2024-11-01 17:38:14 +01:00
Guillaume Wenzek
2251b1d815
zml: sharding (#26)
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.

Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
2024-10-01 21:12:25 +02:00
Corentin Godeau
47674fbe31
PJRT: handle rank-0 special case in getDimensions (#33)
On some platforms, the returned pointer is null so we can't use it as a
slice, even if `num_dims` is 0.
2024-09-30 17:18:42 +02:00
Guillaume Wenzek
689751d7c4
zml: remove dead code (#27)
some dead code or commented out code we should have removed before
release.

copyslice is also broken as per the commented out tests.
it will need to be reimplemented if we need it again.
2024-09-26 11:03:41 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00