0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

129 commits

Author SHA1 Message Date
Rene Schallner
46b5383bc3
WIP cleanups 2025-01-20 15:06:02 +01:00
Rene Schallner
788b72fafa
WIP rene: it generates 2025-01-20 13:57:27 +01:00
Rene Schallner
fc0ec52d5e
WIP rene : rework llama to work more like llmd thx @corendos 2025-01-17 12:43:38 +01:00
Rene Schallner
f327be0091
WIP rene 2025-01-17 12:24:58 +01:00
Steeve Morin
cd27681b09
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
17b1532175
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
7f283eb739
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
e3ae7c78d3
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
a64ee28613
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
b2080da0bf
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
7138664f1d
async: broadcasting on a Condition forces a reactor tick
This is way way more intuitive.
2025-01-17 12:23:03 +01:00
Steeve Morin
8996bb7d83
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
d34fc356fc
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
30325919bd
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
e91d6374c1
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
c4a569cc6f
WIP 2025-01-17 12:23:03 +01:00
Steeve Morin
5e4da63f75
zml/tokenizer: add huggingface tokenizers binding 2025-01-17 12:22:57 +01:00
Steeve Morin
52661f7825
WIP 2025-01-17 12:18:05 +01:00
Guillaume Wenzek
ec48976ce6
zml: revamp scatterSlices (#145)
Main issue with current `scatter` implementation is that it uses
broadcasting dims of `stablehlo.scatter`.
While nice in theory, the optimizer doesn't handle them well and they
often are unrolled into while loop.
Here I convert the batching dim to extra iotas indices.
2025-01-14 16:49:39 +01:00
Guillaume Wenzek
52f40144ce
fix #97 (#150)
Before we where using `module.op().writeBytecode(writer)` to compute the
hash of a model
but it crashes on some inputs, notably for unused variables.

So I used the text representation of the mlir.
2025-01-09 17:19:32 +01:00
Guillaume Wenzek
424bbc5dcb
zml: improve source locations (#148)
Put the name of the operator in the source location.
This seemed redundant at first because it's not needed when looking at
the stablehlo,
but it's nice to have it to understand the optimization passes.

I'm also including a few QoL:

* sdpa don't need a bias argument, attn_mask already play this role.
* gatherSlices should accept a shape literal `.{ .a = 5, .b = 128 }`
* Shape.outer to combine two shapes into the superset of their axes
* Tensor.all (since we already have Tensor.any)
* argMax doesn't need to specify the dtype, it can be inferred from the
size of the input tensors. If the user care about the dtype they can
manually convert.

This PR is stacked on top of #147
2025-01-09 17:18:20 +01:00
Guillaume Wenzek
016e152338
zml: HostBuffer.prettyPrint() (#147)
Add pretty printing of HostBuffer.

This will be leverage by the debug helper `x.print()`
It can also be used like this: `std.log.info("my buffer: {}",
.{host_buffer.pretty()})`
2025-01-09 17:17:53 +01:00
Guillaume Wenzek
56ac7b1b07
pjrt: Profiler fixes (#146)
* allow i64 resource id
* reserve memory when creating array lists
2025-01-08 09:50:14 +01:00
Guillaume Wenzek
b360a92f34
zml: rewrite simple transpose (#142)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.

I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
2025-01-07 10:56:58 +01:00
Hugo Mano
80ec28f3c7
workspace: bump llvm / xla / stablehlo / pjrt plugins xla@20250103.0-5f1fe6a (#141) 2025-01-06 15:06:52 +01:00
Corentin Godeau
d0643ffd59
zml/module.zig: fix superfluous optional unwrapping (#140)
Closes #137
2025-01-02 14:12:31 +01:00
Guillaume Wenzek
cf135bd428
zml: Reorganize the compilation output folders (#135)
* create one folder per target, then one sub-folder per mlir module.
* pass this folder to `xla_dump_to` so that all the xla files arrive
there.
* explicitly save the mlir and the serialized executable there

I'm also exposing James profile conversion tool into ZML as
`profiler.dumpDataAsJson(file)`
2024-12-17 16:55:11 +01:00
Victor Moin
72c56a21bc
Tutorial updates for write_first_model.md (#136)
Quick doc fixes
2024-12-16 21:04:36 +01:00
Steeve Morin
e0041974d2
workspace: rework code completion using a custom build runner (#131)
Following the new release of the VSCode Zig plugin, rework the code
completion logic to leverage a build runner. This is much lighter,
faster and more reliable.
2024-12-16 13:20:27 +01:00
Guillaume Wenzek
e5c152c9f6
zml: add an arena to the CompilationContext (#130)
* add an arena to the CompilationContext

There was already an allocator in the CompilationContext, this PR makes
it into an arena and expose the arena allocator.
This allows ops that need to allocate to do so without abusing
`std.BoundedArray` with arbitrary bounds, and in a safe way.
For now little ops use it, which I believe should stay the case, because
most of the time you want to know how many tensor you're working with.
It'll also simplify the life of Zig new-comers

this include some **breaking changes** to `chunkAllowTrailing` and
`split` that are allocating

* upgrade a few `axis_: i64` into `axis_: anytype` to handle tags

* add a few TODO for the next version of the Tensor API

---------

Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2024-12-11 13:38:55 +01:00
Guillaume Wenzek
e8fa410537
pjrt/zml: expose client creation flags (#129)
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125 

More settings are not exposed ATM, will follow up with #132 

3d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)

AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.

**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
2024-12-11 13:38:04 +01:00
Guillaume Wenzek
05879b41fa
Fix test layer (#124)
* testLayer was silently broken by #115 : test + fix so we detect this
in the future.
* mapAlloc had a regression in #115 wrt struct with 0 size fields: test
+ fix

* add Python `venv` to gitignore
2024-12-11 11:04:38 +01:00
Guillaume Wenzek
c1543278e6
ci: build the examples directory (#121)
Run `bazel build //llama` in CI to make sure we don't accidentally break
it.
I'm also building all the examples target.

For reference it took 5 minutes to run this CI.
2024-12-11 10:56:20 +01:00
Allison Durham
88e0cc7da9
zml: Llama 3.2 (text only models) support (#117)
llama 3.2 1B and 3B do not have lm_head. 

I've added transposing over the embed_tokens as a replacement and set
lm_head as optional for backward compatibility with the other models.

I also added both repositories+executions to bazel and updated the
README.
2024-12-10 16:30:21 +01:00
Rene Schallner
754587f6bb
docs: move image in concepts.md (#128) 2024-12-09 19:48:10 +01:00
Jean-Baptiste Dalido
b2ef326620 readme: fix the banner url 2024-12-09 19:41:41 +01:00
Guillaume Wenzek
38ee9df735
update instructions (#123)
following #115, `prepare` doesn't alloc anymore, `ExeWithWeights` is
`ModuleExe`
2024-12-06 15:28:56 +01:00
Guillaume Wenzek
8cbfe48fe4
async: add intrusive queue (#122)
all code contributed by @steeve

* add intrusive queue
* change the constructor of Channel with default AsyncThread executor

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2024-12-06 15:15:51 +01:00
Guillaume Wenzek
6578a3d11a
zml: emit func.call operations (#116)
Restore the machinery to emit individual functions and calls to them.

this should unblock composites, and generates cleaner MLIR in general

* functions are cached based on the input shapes including tags.
* donations/tags are propagated through function calls

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-12-05 17:45:07 +01:00
Guillaume Wenzek
d669bdc6bf
zml: cleanup module.zig (#115)
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
2024-12-05 17:33:19 +01:00
Corentin Godeau
bed7a57ae4
examples/llama+neuron: enable buffer donation (#112)
Donate verything except the `token_index` buffer as it messes up the
output
2024-12-04 10:51:43 +01:00
Guillaume Wenzek
a6ea0bd905
zml: revisit rope implementation (#114)
The current implementation first compute the full cos/sin matrix for the
full sentence,
then slice into it to extract the positions for the current token.
This is generally optimized by the compiler so it's not a performance
issue, but it's a readability issue,
because it means calling `rope` requires first a call to `ropeCosSin`.

I rewrote this to directly produce the optimized IR where we only
compute rope for given offsets.
2024-12-02 14:51:36 +01:00
Guillaume Wenzek
8de3d19bb3
zml: fix float8 <-> float32 conversions, support for Tensor.constant(.{}, .{ .f8 = 1.0}) (#113)
Mostly:
* fix float8 <-> float32 conversions
* support for `Tensor.constant(.{}, .{ .f8 = 1.0})` 

Misc:
* fix small inconsistencies between different versions of sdpa
* better error message for broadcast
* bazelrc: --config=debug
2024-11-29 16:14:41 +01:00
Steeve Morin
02527de9b9
runtimes/cuda: fetch nccl from the pip package (#110)
Exactly as XLA does
2024-11-28 11:14:05 +01:00
Steeve Morin
6f6dfe9fe5
runtimes/cuda: rework sandboxing and dynamic symbol renaming (#109)
This change reworks the sandboxing and dynamic symbol renaming for the
cuda runtime.

Switch to the jax-cuda-pjrt pre-built PJRT plugin until we fix the pjrt
artifacts build. Most notably because our build of the plugin didn't
include the right compute capabilities.

Take the opportunity to bump the `cuda=12.6.2` `cudnn=9.5.1`.

Fixes #19
2024-11-27 19:39:17 +01:00
Hugo Mano
85685174c9
zml/pjrtx: use our own stablehlo version if plugin provided version is newer (#108)
Discovered use case:

- PJRT plugin attribute `stablehlo_current_version` is `1.7.8`
- `stablehloVersionFromCompatibilityRequirement(WEEK_12)` is `1.1.0`
-  `stablehloGetCurrentVersion` is `1.7.6`

The target version for artifact serialization is `1.7.8` but should be
`1.7.6`.

XLA ref:
2f99455cdf (diff-7f335a7892)
2024-11-27 18:29:04 +01:00
Hugo Mano
2f70fa4a00
workspace: log message before return NotFound (#107)
This allow us to find which NotFound is raised
2024-11-27 17:21:34 +01:00
Hugo Mano
db55475b17
examples/loader: missing stdx dep (#106) 2024-11-27 17:09:25 +01:00
Corentin Godeau
6a1183f8ad
zml/pjrtx: call bufferFromHostBuffer on the thread pool (#103)
This was causing very bad performance for weight loading on Neuron
because the calls are blocking
2024-11-26 18:57:59 +01:00
Corentin Godeau
58f5895d50
zml/platform: increase max device limit (#104)
Some platform (such as Neuron) can have up to 32 devices, one per core.
2024-11-26 18:57:21 +01:00