0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

70 commits

Author SHA1 Message Date
Corentin Kerisit
c30d9138a4 Upgrade toolchains_llvm_bootstrapped to 0.2.4 2025-07-11 14:18:02 +00:00
Hugo Mano
b4716dba03
ci: revamp commit workflow (#246)
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags 
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`

Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls (#258)
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.

### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅	Loaded weights in 397ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201  627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***

error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```

### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`

```logs
info(modernbert): ✅	Loaded weights in 416ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅	Loaded weights and compiled model in 2968ms
info(modernbert): 	Input text: [MASK] is the capital of France.
info(modernbert): ⏱️	Model inference in  347ms
info(modernbert): ✅	Top 5 predictions:
info(modernbert): 	  • score: 0.9755  word: 'Paris' token: 36062
info(modernbert): 	  • score: 0.0067  word: 'Nice' token: 29235
info(modernbert): 	  • score: 0.0063  word: 'It' token: 1147
info(modernbert): 	  • score: 0.0010  word: 'Or' token: 3980
info(modernbert): 	  • score: 0.0010  word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
f323979cb2
zml/buffer: various improvements (#248)
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
2025-06-06 18:12:49 +02:00
Corentin Godeau
7ac7a7f210
examples/llama: set model.norm.eps correctly (#256) 2025-06-06 18:12:25 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value (#255)
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Corentin Godeau
41e222f39f
workspace: updated default nvim configuration (#254)
This requires neovim >0.11.0
2025-06-06 11:20:03 +02:00
Steeve Morin
2502c44c0f
workspace: switch to toolchains_llvm_bootstrapped (#245)
Notably, this will enable using custom linkers or enable LTO.

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-06-05 21:45:15 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f (#238)
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily

XLA diff:
71c67e2a4f...cb67f2f7ce

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Steeve Morin
6df48e39de
workspace: make zls runner target explicit (#240)
Easier for future reference
2025-05-29 23:04:20 +02:00
Guillaume Wenzek
f2f37e6d8d
zml: add some buffer / hostbuffer helpers (#232)
* buffer/hostbuffer utils
* more precise f32 -> bf16 conversion
* visit should not read irrelevant fields
* HostBuffer.data -> HostBuffer.bytes()
* zml.aio loadBuffers: type inference
* store expected input shapes in the executable
* zml.meta.visit support std.BoundedArray
* stdx.json.TaggedUnion
* protection against undefined float hyperparams
* avoid meta.TagPayload which can be quite slow
* improve logging
2025-05-20 15:08:35 +02:00
Steeve Morin
a566e396d2
workspace: add debug/release configurations and various stdx (#234)
This PR bumps rules_zig to add command line to the zig compilation in
order to enable debug and release configs from the bazel command line
(and bazelrc).

Most notably, this is useful for passing `-mcpu` flags to Zig
(especially for SIMD support like AVX512).

While we're there, add `stdx.time.Duration.hz()` and
`stdx.math.roundeven()`.
2025-05-19 21:26:28 +02:00
Guillaume Wenzek
bf79263919
Fix llama3 rope scaling (#230) 2025-05-09 14:32:15 +02:00
Steeve Morin
7ccfeaf524
workspace: various fixes and bumps (#229)
This PR add various fixes and bumps in the workspace, notably to prevent
dual builds for LLVM tools.
2025-04-22 16:12:26 +02:00
Steeve Morin
1bdde64755
workspace: fix vscode-zig update (#217)
The extension checks that the ZLS path is absolute, else it will run
which to find it in the PATH.

Add a heading / so that the path looks absolute. This works because we
use ${workspaceFolder}.
2025-03-28 13:41:01 +01:00
Steeve Morin
e0336dc9b3
workspace: fetch llvm, stablehlo and triton from xla, bump XLA (#206)
This enables making sure versions are the same across the board.

Also, import `triton` repo for later use.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-19 10:00:27 +01:00
Victor Moin
123082de35
runtimes: add macOS x86_64 (#175)
This PR showcases an initial implementation to run ZML on MacOS Intel
(x86_64). While Mac Intel is *deprecated*, this work allowed me to
contribute to [modernbert example](https://github.com/zml/zml/pull/149).
It relies on a custom
[pjrt-cpu_darwin-amd64.tar.gz](https://github.com/vctrmn/pjrt-artifacts/releases/download/v3.0.0-rc/pjrt-cpu_darwin-amd64.tar.gz),
which is not officially supported by the ZML team.

This PR __is not intended for merging__ but serves as a reference for
users who want to experiment with ZML on older Mac.

https://github.com/zml/zml/issues/105
2025-03-18 16:52:18 +01:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Steeve Morin
5bb1d46640
workspace: rework runtimes sandboxing, bump PJRT plugins, cuda=12.8 and ROCm=6.3.4 (#199)
This PR significantly reworks the runtimes sandboxing to leverage
rules_distroless apt resolution, bump pjrt-artifacts=7.0.0 which inclues
Triton ROCm fixes.

Finally, bump CUDA=12.8 cuDNN=9.8, ROCm=6.3.4
2025-03-13 22:17:41 +01:00
jaco-bro
7e62b6551c
workspace: update rules_cc=0.1.1 and bazel=8.1.1 (#196)
This PR updates the `rules_cc` dependency from `0.1.0` to `0.1.1` and
the Bazel version from `8.0.1` to `8.1.1`.

### Changes:
1. Updated `rules_cc` version in `zml/MODULE.bazel` from 0.1.0 to 0.1.1:
    ```bazel
    bazel_dep(name = "rules_cc", version = "0.1.1")
    ```
2. Updated the Bazel version in `zml/examples/.bazelversion` from
`8.0.1` to `8.1.1`.

### Context:
1. Initially, running `bazel run -c opt //mnist` failed due to a Bazel
version mismatch (required `8.0.1` but the currently brew-installable
version is `8.1.1`). This was resolved by updating `.bazelversion`.
2. After fixing the Bazel version, the build failed again this time due
to a yanked version of `rules_cc` (`0.1.0`).

> ERROR: Error computing the main repository mapping: Yanked version
detected in your resolved dependency graph: rules_cc@0.1.0, for the
reason: rules_cc 0.1.0 is yanked due to incompatible change (prematurely
removing cc_proto_library from defs.bzl), please upgrade to 0.1.1.
> Yanked versions may contain serious vulnerabilities and should not be
used. To fix this, use a bazel_dep on a newer version of this module. To
continue using this version, allow it using the --allow_yanked_versions
flag or the BZLMOD_ALLOW_YANKED_VERSIONS env variable.

3. Updating `rules_cc` to `0.1.1` resolved this issue.

### Testing:
1. Verified the update by running:
    ```zsh
    git clone https://github.com/jaco-bro/zml
    cd zml/examples
    bazel run -c opt //mnist
    ```
2. The build and execution succeeded without errors.

    ```
    info(mnist): ✅ RECOGNIZED DIGIT:
               +-------------+
               |     ###     |
               |    #   #    |
               |   #     #   |
               |   #     #   |
               |   #     #   |
               |    #   #    |
               |     ###     |
               +-------------+
    ```
2025-03-03 16:46:50 +01:00
Guillaume Wenzek
a672bb8247
examples: add modernBERT (#192)
Modern bert implementation from Victor: @vctrmn

Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.

---------

Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
2025-02-27 18:07:42 +01:00
Hugo Mano
2fd9516773
workspace: query, build and test (#187)
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.

As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
2025-02-12 10:40:20 +01:00
Guillaume Wenzek
1c2580b3c2
zml: FnCache should handle non-tensor values for less confusing behavior (#186)
Currently if you used the FnCache (through zml.call) on a function with
non tensor fields you would get a struct with undefined memory.
This wasn't nice. So now we copy the value of the function returned by
the first zml.call, and use this as base for future calls, reusing
non-tensor fields, and adding correct mlir.values to the previous
tensors.
This is easy because now zml.call only supports fixed size structs (no
slice/pointer)

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2025-02-11 16:49:05 +01:00
Hugo Mano
ceec8a8c9a
workspace: bump xla to version 20250204.0-6789523 (#184)
Also improves Bazel 8 compatibility

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-02-07 15:36:15 +01:00
Guillaume Wenzek
5de2adcec3
Fix llama generations (#181)
Current implementation of llama is discarding the first outputted token,
and feeding the last prompt token a second time in the model.

Due to how self attention works this wasn't too disturbing for attention
but it was technically wrong.
Also it was doing a bit extra compute for nothing.

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2025-02-06 14:10:02 +01:00
Steeve Morin
cbd9e50a79
workspace: bump libxev to 20252401.0-31eed4e (#182)
Includes:
- https://github.com/mitchellh/libxev/pull/128
- https://github.com/mitchellh/libxev/pull/141
2025-02-05 13:40:03 +01:00
Rene Schallner
a9057737f3
llama: clean up llama3 prompt template encoding. (#172)
There was an unnecessary re-encoding of the newline string, and we added
a newline at the end.
2025-02-03 12:10:19 +01:00
Guillaume Wenzek
f1d0c64db3
Restore tinyllama models (#173)
This is a follow up on #162 

* restore homemade tokenizer, integrate it with the new zml/tokenizer
library. This will only be used for working with tinyllama, I could also
make it work to support GGUF if needed (couple lines of code).
* this allows us to use the original tinyllama files from karpathy repo
which generates actual sentences. This is important when working on
modeling because subtle bugs can degrade the output quality, which is
not detectable if the output is crap to begin with.
* fix casing of zml/tokenizer api: fromFile, tokenToId. Let's stick to
the Zig style guide. Function written in Rust follows rust convention,
but the Zig bindings should follow Zig convention.
* make tokenizer/main.zig a proper tokenizer test tool reusing
llama/test_tokenizer.zig

~~use native_binary instead of cc_binary~~
Reverted because apparently native_binary break the runfiles and
therefore the cuda sandbox
2025-02-03 12:06:33 +01:00
Corentin Kerisit
71fa3764d1
bazel: use prebuilt protoc binaries (#178)
This removes ~1300 build steps by depending on prebuilt binaries.

Note that this PR is blocked by
https://github.com/aspect-build/toolchains_protoc/pull/44 since
`rules_proto` depends on `protobuf@29.1` and
https://github.com/aspect-build/toolchains_protoc doesn't expose
`protoc@29.1` and we can't pin module versions unless adding an entry in
the local BCR which I think is overkill for this.
2025-01-31 23:12:04 +01:00
Rene Schallner
edf5d9a93d
llama: fix dims & dtypes -> fix #170 (#171)
This PR fixes issue #170 , making Llama run again on CUDA and RoCM.

- Latest Llama changes had introduced an unnecessary batching dimension:
fixed
- Ops returning indices return i32 yet we now use u32 as token indices
-> dtype conversion requred: fixed

Tested on CUDA.
2025-01-27 17:06:03 +01:00
Steeve Morin
d76e5b9cd4
examples: fix logFn usages (#167)
Ooops! Forgot to update the examples.
2025-01-24 21:07:36 +01:00
Rene Schallner
f41e023e22
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162)
# HF tokenizers & Llama improvements

1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example

## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers

`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.

To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.

## 2. examples/llama

Changes to the llama example:

- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
    - single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
Corentin Kerisit
52c5d4fe02
deps: Upgrade rules_distroless to 0.4.2 (#161)
The reason for the upgrade is because the //:lock target of the apt
repository checks for bazel marker files to write the lockfile at the
right relative path from the root module.

In 0.3.8, only WORKSPACE and WORKSPACE.bazel files were checked as
marker files, and since we are full bzlmod with a single MODULE.bazel
marker file, the run target would fail like so:

```bash
> bazel run @neuron_bookworm//:lock
Writing lockfile to runtimes/neuron/packages.lock.json
Error: neither WORKSPACE nor WORKSPACE.bazel file was found
```

0.4.2 brings that support.

# Tests

I only tested `cd examples; bazel run //mnist:image` to validate that it
still works. Let me know what else is needed.
2025-01-24 13:58:23 +01:00
Jean-Baptiste Dalido
3682e16da9
workspace: upgrade to Bazel 8 (#143)
This PR covers the migration to Bazel 8.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-01-23 18:36:57 +01:00
Guillaume Wenzek
b517170674
Llama3.2: fix tokenizer (#157)
Fix #134 , superseed #133

Basically the llama3 tokenizer.json has the following config: 

```
  "model": {
    "type": "BPE",
    "dropout": null,
    "unk_token": null,
    "continuing_subword_prefix": null,
    "end_of_word_suffix": null,
    "fuse_unk": false,
    "byte_fallback": false,
    ...
}
```

this it technically invalid because it says the model has no byte
fallback and no unknown token either.
This means that unknown emoji can't be converted to bytes nor to the
special unknown token.
This ended up in tokenizer.zig using the value of MAX_INT for the UNK
token then panicking for out of bound.

So I've added some logic to recognize that in the case of GPT2 encoded
vocab (like llama3) that actually contains individual bytes, we should
use the "byte_fallback" algorithm even if disabled.

I've also added a Llama3 "well known" normalizer because HF tokenizers
doesn't add a dummy prefix for llama3 tokenizer.

Technically this is still not a correct implementation of llama3
tokenizer because we don't do the regex based splitting they do. But
even for quite long snippets I tested it now produces exact same tokens.
2025-01-21 13:41:57 +01:00
Steeve Morin
f228ec2edc
async: fix async hangs (#154)
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.

In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.

Should have a slight performance increase, too.
2025-01-18 00:50:51 +01:00
Guillaume Wenzek
424bbc5dcb
zml: improve source locations (#148)
Put the name of the operator in the source location.
This seemed redundant at first because it's not needed when looking at
the stablehlo,
but it's nice to have it to understand the optimization passes.

I'm also including a few QoL:

* sdpa don't need a bias argument, attn_mask already play this role.
* gatherSlices should accept a shape literal `.{ .a = 5, .b = 128 }`
* Shape.outer to combine two shapes into the superset of their axes
* Tensor.all (since we already have Tensor.any)
* argMax doesn't need to specify the dtype, it can be inferred from the
size of the input tensors. If the user care about the dtype they can
manually convert.

This PR is stacked on top of #147
2025-01-09 17:18:20 +01:00
Guillaume Wenzek
b360a92f34
zml: rewrite simple transpose (#142)
Rewrite trivial transpose into reshapes.
Transpose over a 1-d axis are effectively no-op.
Normally the backend can detect this, but we observe that some backend
can get confused. So avoiding those extra transpose can help.

I also changed the default profiler size to 1_000_000 events because
above that the perfetto ui can't load the generated file anyway
2025-01-07 10:56:58 +01:00
Hugo Mano
80ec28f3c7
workspace: bump llvm / xla / stablehlo / pjrt plugins xla@20250103.0-5f1fe6a (#141) 2025-01-06 15:06:52 +01:00
Guillaume Wenzek
cf135bd428
zml: Reorganize the compilation output folders (#135)
* create one folder per target, then one sub-folder per mlir module.
* pass this folder to `xla_dump_to` so that all the xla files arrive
there.
* explicitly save the mlir and the serialized executable there

I'm also exposing James profile conversion tool into ZML as
`profiler.dumpDataAsJson(file)`
2024-12-17 16:55:11 +01:00
Steeve Morin
e0041974d2
workspace: rework code completion using a custom build runner (#131)
Following the new release of the VSCode Zig plugin, rework the code
completion logic to leverage a build runner. This is much lighter,
faster and more reliable.
2024-12-16 13:20:27 +01:00
Guillaume Wenzek
e8fa410537
pjrt/zml: expose client creation flags (#129)
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125 

More settings are not exposed ATM, will follow up with #132 

3d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)

AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.

**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
2024-12-11 13:38:04 +01:00
Allison Durham
88e0cc7da9
zml: Llama 3.2 (text only models) support (#117)
llama 3.2 1B and 3B do not have lm_head. 

I've added transposing over the embed_tokens as a replacement and set
lm_head as optional for backward compatibility with the other models.

I also added both repositories+executions to bazel and updated the
README.
2024-12-10 16:30:21 +01:00
Guillaume Wenzek
6578a3d11a
zml: emit func.call operations (#116)
Restore the machinery to emit individual functions and calls to them.

this should unblock composites, and generates cleaner MLIR in general

* functions are cached based on the input shapes including tags.
* donations/tags are propagated through function calls

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-12-05 17:45:07 +01:00
Guillaume Wenzek
d669bdc6bf
zml: cleanup module.zig (#115)
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
2024-12-05 17:33:19 +01:00
Corentin Godeau
bed7a57ae4
examples/llama+neuron: enable buffer donation (#112)
Donate verything except the `token_index` buffer as it messes up the
output
2024-12-04 10:51:43 +01:00
Guillaume Wenzek
a6ea0bd905
zml: revisit rope implementation (#114)
The current implementation first compute the full cos/sin matrix for the
full sentence,
then slice into it to extract the positions for the current token.
This is generally optimized by the compiler so it's not a performance
issue, but it's a readability issue,
because it means calling `rope` requires first a call to `ropeCosSin`.

I rewrote this to directly produce the optimized IR where we only
compute rope for given offsets.
2024-12-02 14:51:36 +01:00
Guillaume Wenzek
8de3d19bb3
zml: fix float8 <-> float32 conversions, support for Tensor.constant(.{}, .{ .f8 = 1.0}) (#113)
Mostly:
* fix float8 <-> float32 conversions
* support for `Tensor.constant(.{}, .{ .f8 = 1.0})` 

Misc:
* fix small inconsistencies between different versions of sdpa
* better error message for broadcast
* bazelrc: --config=debug
2024-11-29 16:14:41 +01:00
Steeve Morin
6f6dfe9fe5
runtimes/cuda: rework sandboxing and dynamic symbol renaming (#109)
This change reworks the sandboxing and dynamic symbol renaming for the
cuda runtime.

Switch to the jax-cuda-pjrt pre-built PJRT plugin until we fix the pjrt
artifacts build. Most notably because our build of the plugin didn't
include the right compute capabilities.

Take the opportunity to bump the `cuda=12.6.2` `cudnn=9.5.1`.

Fixes #19
2024-11-27 19:39:17 +01:00
Hugo Mano
db55475b17
examples/loader: missing stdx dep (#106) 2024-11-27 17:09:25 +01:00