0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

39 commits

Author SHA1 Message Date
Corentin Godeau
6f9d6df943 misc: improved benchmark 2024-10-17 15:53:07 +02:00
Corentin Godeau
d02ccc0486 misc: batched benchmark 2024-10-17 15:53:07 +02:00
Guillaume Wenzek
72a1c5954b
zml/aio/torch: rename (#52)
Applying a few principles from Zig style guide:

### [Redundant names in fully qualified
namespace](https://ziglang.org/documentation/0.13.0/#Avoid-Redundant-Names-in-Fully-Qualified-Namespaces)
:

ops.PickleOp -> pickle.Op
parser.Decoder -> parser.Parser

### [Avoid generic
names](https://ziglang.org/documentation/0.13.0/#toc-Avoid-Redundancy-in-Names)
:

removed "utils.zig" and inlined the code where used.

### allow to read file top to bottom

sorted functions in `aio/torch.zig` by the order of call.
extracted some part of the code into functions `parseTensor`,
`parseStorage`
2024-10-14 15:41:52 +02:00
Guillaume Wenzek
27add022a9
zml: fix testLayer (#49)
`testLayer` was broken because it wasn't directly used by `llama.zig`.

Remove the uneeded "compile_options" argument,
update the testing logic to the new sharded output.
2024-10-14 09:53:38 +02:00
Guillaume Wenzek
9cf68e0bec
Try to read normalizer setting from the huggingface json (#47)
We had issues with the CodeLlama tokenizer.
https://huggingface.co/codellama/CodeLlama-7b-hf

Looking at the config, it's actually a "sentencepiece" tokenizer, that
uses the special "▁" token to represent spaces.

```
"normalizer": {
    "type": "Sequence",
    "normalizers": [
      {
        "type": "Prepend",
        "prepend": "▁"
      },
      {
        "type": "Replace",
        "pattern": {
          "String": " "
        },
        "content": "▁"
      }
    ]
  },
```

I've added a `Normalizer.fromHfJson` that reads this json and tries to
convert this style to our options.
This config can express more things than we can, so it's a best effort.
I've also made the magic space token configurable, requiring adding a
small buffer in Normalizer to store it.

There is a small change that can have importance, is that now in most
cases the setting "remove_extra_whitespaces" will not be set, meaning
that user input will be tokenized differently in face of double spaces.
I generally don't think this is a good practice, but it respect HF
defaults.

I've also added a `debug` boolean on the tokenizer struct that shows the
individual merges of tokens.
It helped me previously when debugging tokenizer issues.
Before you needed to explicitly uncomment some code to use it, now it's
behind a runtime boolean.

---------

Co-authored-by: Rene Schallner <rene@zml.ai>
2024-10-11 15:27:51 +02:00
Guillaume Wenzek
164a2f2c70
zml.aio.json: fix loading empty array (#50)
safetensor loader was segfaulting when loading metadata with an empty
json array.

I also rewrote some of the torch loader code to improve the readability.
Notably every slice in `ops.zig` is now const. I've also realized that
our strategy to load Pytorch tensor is not future proof see the warning
at: https://pytorch.org/docs/stable/storage.html
2024-10-11 13:26:04 +02:00
Hugo Mano
f26a022aa2
zml/sharding: user sharding constraints api and hlo integration (#46)
- Introduce sharding hints from the user code to specify how operations
should be split across user's devices mesh leveraging the HLO sharding
annotations
- Remove options arguments in `zml.module` including
`opts.add_donations_attributes`: it's for the function caching because inner function can't do donation, so we don't want to put the donation attributes there but for now we it's removed and we will do it differently next time around
2024-10-10 13:56:49 +02:00
Guillaume Wenzek
9a895ac0e6
make zml/test_runner.zig public (#48)
The end goal is to allow zml users to use our async test runner in their
project.

This approach works but is a bit hacky because it relies on the fact
that `test_runner.zig` is part of the `zml` library
otherwise `zig_test` doesn't correctly put the test_runner into the
build sandbox.

I made this dependency explicit, but it was already the case because of
the glob pattern used in zml library.

And I think `test_runner` should be `zig_library` in the first place,
and not a filegroup.
See https://github.com/zml/rules_zig/issues/2 for follow up.
2024-10-09 11:02:16 +02:00
Steeve Morin
9b3491f4bf
workspace: be a little more precise regarding the huggingface token (#45)
Indicate that it's also the standard location used by the CLI, and add
the environment variable, too.

---------

Co-authored-by: Rene Schallner <rene@renerocks.ai>
2024-10-06 14:34:38 -07:00
Jon Stevens
a050b3f9ce
make the token directory first if it doesn't already exist (#44) 2024-10-04 11:52:57 -07:00
Jean-Baptiste Dalido
e5c43e2799
runtimes/cuda: fix versions in build (#43)
We missed a variable bump for runtimes/cuda in #ea0b204, making it
impossible to build CUDA.
2024-10-04 16:04:09 +02:00
Guillaume Wenzek
76a68980c8
zml: move tests next to the tested function (#42)
note: `dynamicSlice1d()` has been completely removed because the `test
dynamicSlice` has already very similar test.
2024-10-04 09:42:21 +02:00
Guillaume Wenzek
ec9f69add7
zml: remove pjrtx (#38)
pjrtx.zig was mean to be a ZML specific wrapper over pjrt.zig.
But pjrt.zig in its current form is pretty usable without wrapper,
the extra layer of indirection is IMHO confusing because it mostly
transfer the calls directly to pjrt.zig but make some call suspending.

I moved the few helpers left in pjrtx.zig where they are used, ie 

`pjrtx.Buffer.bufferFromHostBuffer`-> in `buffer.Buffer.from`
`pjrtx.Client.compile` -> in `module.compileModuleToPjrtExecutable`
`pjrtx.Event.await` -> `zml.Platform.awaitEvent()`

This also fix a bug in `pjrtx.Event.await` which was blocking for the
event before suspending for the event.
2024-10-04 09:32:53 +02:00
Steeve Morin
0049efdd0c
runtimes/cuda: bump cuda=12.6.2 and cudnn=9.4.0 (#40) 2024-10-03 09:42:17 -04:00
Steeve Morin
ea0b204af8
runtimes/rocm: bump ROCm to 6.2.2 (#39) 2024-10-03 09:41:57 -04:00
Guillaume Wenzek
db861fd5e2
zml.tokenizer: proper byte fallback support (#37)
Some tokenizers have bytes encoded in hex like this: "<0x40>".
This break the tokenization algorithm because the input text will
contain "@" not "<0x40>",
and if the input contains "<0x40>" it needs to not be treated as a
single byte.
So we replace byte fallbacks strings, by their corresponding character.
This enables the normal tokenization algorithm to work.

This PR also implement the main byte fallback logic, that split unknow
utf8 codepoint into bytes.

---------

Co-authored-by: cryptodeal <james@zml.ai>
2024-10-01 21:15:59 +02:00
Guillaume Wenzek
2251b1d815
zml: sharding (#26)
First rough support for in-process sharding.
The goal of sharding is to allow leveraging several GPUs to collaborate
to run one big model.

Current implementation:
* added `platform.sharding()` describing how the set of devices managed
by the current process should be used.
* added a `_sharding_info` to shape.
* A tensor / shape axis can be marked as needing sharding using
`.withSharding(.{ .a })` or `.withSharding(.{ 1 })`.
* When using `populateModelWeights` tensors with sharded axes, will be
populated by sharded buffers.
* when generating the MLIR bytecode for a ZML function the **input**
arguments, will see their `_sharding_info` converted into
`mhlo.sharding` attributes
2024-10-01 21:12:25 +02:00
Corentin Godeau
4f1d6fec3e
Add StableHLO bindings for functions related to versioning (#34)
These functions can be used to serialize stablehlo in a portable way.
2024-09-30 17:18:53 +02:00
Corentin Godeau
47674fbe31
PJRT: handle rank-0 special case in getDimensions (#33)
On some platforms, the returned pointer is null so we can't use it as a
slice, even if `num_dims` is 0.
2024-09-30 17:18:42 +02:00
Corentin Godeau
7ba81d39e5
Use select instead of log to generate attention mask (#32)
Using log on 0 values can lead to nans on some platforms
2024-09-30 17:18:27 +02:00
Guillaume Wenzek
6079d54927
Implement scatterSlices (#25) 2024-09-27 16:09:06 +02:00
Corentin Godeau
6553e31a76
zml: Use correct init values for Tensor.min() and Tensor.max() (#29)
When doing a `max` over a dimension, you have to start with the minimum
value possible, otherwise you might miss the maximum.

Example:
Let's imagine you have a Tensor with the following values: `[-5, -4, -3,
-2, -1]`
If you start with a value of `0`, the `max` in reduce will always return
0. Thus in the end, you get a maximum of `0` whereas `0` is not in the
set of values.

Same for `min`
2024-09-26 11:58:29 +02:00
Guillaume Wenzek
689751d7c4
zml: remove dead code (#27)
some dead code or commented out code we should have removed before
release.

copyslice is also broken as per the commented out tests.
it will need to be reimplemented if we need it again.
2024-09-26 11:03:41 +02:00
Guillaume Wenzek
6b6448e8ed
remove bugged Tensor.chunk (#28)
User should use either `chunkExact` or `chunkAllowTrailing` to clarify
what they want.

previous `chunk` was crashing on `Tensor(.{.a=10}).chunk(.a, 3)` 
`chunkAllowTrailing` will return **4** tensors: `.{.a=3}, .{.a=3},
.{.a=3}, .{.a=1}`

Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2024-09-26 11:01:20 +02:00
Guillaume Wenzek
1c7d7dacef
workspace: bump xla to enabled batching in gather (#21)
Since we were using an "old" version of XLA we where missing the
batching dims stablehlo attribute.
This was preventing the usage of batching in the gather operator.

Since I had already wrote the batching test before hand and was just
ignoring the compilation error,
this PR just bump xla and remove the error suppression code in tests.
2024-09-24 09:40:55 +02:00
Leonard Bogdonoff
da439255e7
README: Fix misspelled param (#23) 2024-09-24 09:07:49 +02:00
cryptodeal
890f5e6299
aio: fix refAllDecls for yaml/nemo (#20) 2024-09-21 01:02:50 -05:00
Guillaume Wenzek
714906265a
zml/nn: fix resize (#18)
Add more `refAllDecl` wich detected non compiling code.
I had to fix resizeBilinear / resizeBicubic.
(85 tests now)
2024-09-20 17:49:12 +02:00
Tyler Rockwood
36ecd6edcc
docs: fixup llama target names (#17) 2024-09-20 17:15:39 +02:00
cryptodeal
f20b0720d3
aio: cleanup torch model parsing (#5)
Improve readability parsing pytorch model; optimize for reading slice
w/o a single type as tuple.
2024-09-20 08:58:27 -05:00
Hugo Mano
7ef90a44fb
zml/tests: ensure all Zig tests are running (#14)
This PR restore skipped tests. 
It was skipped because we managed to have a CI infrastructure for the
public release. There was a problem regarding computation precision 
on ubuntu/cpu platform so it was skipped. It's now fixed using `f32` 
instead of `f16` precision so we have lower diffs.
I also added `std.testing.refAllDecls` on each main namespace to ensure
we run every declared tests in `zml/`.
We have to rework about how we want to reference all tests but for now
it does the job.
2024-09-20 15:26:35 +02:00
Hugo Mano
eb0c5b3e7f
zml/test_runner: add optional filtering on test functions list (#15)
**Motivation**

In some contexts we want to run only a subset of our test suite. 
This allow us to debug and trace more easily what's going on with
failing tests.
Since we maintain our own test runner mainly to run in a async stack,
we can easily filter the list filled by `builtin.test_functions`

**Implementation**

I purpose a simple filtering without flags management, based on the
first process argument.
- `bazel run -c opt //zml:test` will ran the full test suite as before
this pull request.
- `bazel run -c opt //zml:test -- sdpa` or `bazel run -c opt //zml:test
-- "sdpaMemEfficient without mask"` will run
only tests with these words in the function name.

For later, it will gives us the opportunity to have a testing strategy.
Eg: `test "sdpaMemEfficient without mask - [tag1, tag2]" { ...} `

**Caveats**

- While we are able to call a subset of test functions, the tests target
need to compile
every necessary deps and sources, so your code should compile without
failure.
- Maybe we will have to add cumulative arguments to create matrix tests
2024-09-20 12:05:27 +02:00
Ikko Eltociear Ashimine
c60f5bae72
zml/tensor: fix typo in uniform comment (#16)
substract -> subtract
2024-09-19 22:14:59 +02:00
Guillaume Wenzek
493f61d31b
zml/tensor: gatherValues (#12)
gather1d was quite limited by design.
gatherValues is a generalization of gather1d, that handles tuple of
indices and batching.

For each coordinate in `indices`,
`gatherValues` extracts a single value of the given tensor.

* axes_ is a single axis, or a tuple of axis: .b, or .{ .b, .c }
* indices is an integer tensor
* result is a tensor whose shape is similar to the input shape
where the gathered axes have been replaced by axes from 'indices'.

Some example input for the base case where we work on one axis:
- gatherValues(f:[a]->float, .a, ind:[n]->int)[n] == f[ind[n]]
- gatherValues(f:[a, b], .a, ind:[n])[n, b] == f[ind[n], b]
- gatherValues(f: [a,b,c], .{.b}, ind: [n,m])[a, n, m, c] == f[a, ind[n,
m], c]

If an axis in common between `self` and `indices`,
it is treated as a "batching" axis, meaning that semantically
the operator is doing a gatherValues one time per dimension of this
axis:
- gatherValues(f: [a,b,c], .{.b}, ind: [a,n])[a, n] == f[a, ind[a, n]]

It is an error to have an axis present in `self`, `axes_` and `indices`.

If several axes are passed, then the last axis of indices is treated as
coordinates:
- gatherValues(f: [a,b,c], .{.b, .c}, ind: [n,2])[a, n] == f[a,
ind[n][0], ind[n][1]]
- gatherValues(f: [a,b,c,d], .{.b, .c}, ind: [a, n,2])[a, n, d] == f[a,
ind[a, n][0], ind[a, n][1], d]

It is possible to use gatherValues without tags, but batching won't be
available.


> [!WARNING]
> The semantic of inserting the indices axis where we are dropping the
input axis is very useful in practice:
> `roll`, `nearest`, `resize` benefit greatly from it.
> But it doesn't always play nice with batching dimensions.
> In particular the following currently isn't accepted:
> `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .n = 8, .a = 10 })`
> because MLIR will produce `.{ .n = 8, .a = 10 }` following the indices
shape,
> while we produce `.{.a = 10, n=8}` following the semantic of
"replacing .b with axes from indices".
> If you do `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .a = 10, .n =
8 })` then both convention agrees and it's a non issue.
> I think this inconsistency is ok for now because it only happens when
you have both batch axes and extra-indices axes which act like second
set of batching dimensions.
> the issue is avoidable from the user side by eg broadcasting the
values along `.n`, or transposing one of the operand.

---------

Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2024-09-19 16:17:07 +02:00
Hugo Mano
0b136ef43a
zml_utils: fix CollectionOver scope (#13)
**Changes**

- zml_utils: fix `AttributeError: type object 'ActivationCollector' has
no attribute 'CollectionOver'`
- zml_utils: remove dead code and comments
- docs(howto_torch2zml): add explanation on why `output` variable can be
`None`
2024-09-19 13:54:21 +02:00
Rene Schallner
034c004012
docs: first model: fix const vs. var and 80col limit (#3)
There was a const/var bug in the code shown.

Also, after adhering code to roughly 80col limit, the code does not need
to scroll horizontally anymore.
2024-09-17 16:28:30 +02:00
Rene Schallner
3e6c148e7a
docs: add links to online docs to README (#2) 2024-09-17 13:16:01 +02:00
Guillaume Wenzek
bf8244a1ad
docs: fix Mnist typo (#1)
A wild typo appears!
2024-09-17 12:15:15 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00