0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

7 commits

Author SHA1 Message Date
Steeve Morin
cfe4581f85 examples,runtimes: add layers for each runtime
Some runtimes require additional layers to be built. This change adds
a generic `layers` filegroup to each runtime. It should be embedded into
OCI images.
2024-11-08 17:49:55 +01:00
Steeve Morin
ccd925309b
async: rename async_ to asyncc and add Generic slugs (#71)
`asyncc` is arguably nicer to write than `async_`, so use that.

Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
2024-11-01 17:36:08 +01:00
Corentin Godeau
7a35d53ea3
Add Llama 3.1 70B Instruct model (#56)
This can be handy to test beefier models on powerful accelerators
2024-10-17 17:05:24 +02:00
Corentin Godeau
cb31e4a749
Llama weights sharding (#58)
This PR introduces sharding attributes to Llama weights to take
advantage of Tensor Parallelism
2024-10-17 17:05:10 +02:00
Guillaume Wenzek
895492dee4
zml.aio.torch: rename zml.aio.Value -> zml.aio.Metadata (#55)
Part of ongoing improvements to the torch loader

* rename zml.aio.Value -> zml.aio.Metadata
* simplify the types zml.aio.Metadata can have
* pickle `parse` function moved to `pickle.zig`
* add dedicated test case
* fix small bug in parse where `short_binstring` was converted to
`bin_string`
* pickle `evaluate` function now returns `[]const zml.aio.torch.Value`

note: I didn't modified the core logic.
2024-10-17 16:16:05 +02:00
Guillaume Wenzek
493f61d31b
zml/tensor: gatherValues (#12)
gather1d was quite limited by design.
gatherValues is a generalization of gather1d, that handles tuple of
indices and batching.

For each coordinate in `indices`,
`gatherValues` extracts a single value of the given tensor.

* axes_ is a single axis, or a tuple of axis: .b, or .{ .b, .c }
* indices is an integer tensor
* result is a tensor whose shape is similar to the input shape
where the gathered axes have been replaced by axes from 'indices'.

Some example input for the base case where we work on one axis:
- gatherValues(f:[a]->float, .a, ind:[n]->int)[n] == f[ind[n]]
- gatherValues(f:[a, b], .a, ind:[n])[n, b] == f[ind[n], b]
- gatherValues(f: [a,b,c], .{.b}, ind: [n,m])[a, n, m, c] == f[a, ind[n,
m], c]

If an axis in common between `self` and `indices`,
it is treated as a "batching" axis, meaning that semantically
the operator is doing a gatherValues one time per dimension of this
axis:
- gatherValues(f: [a,b,c], .{.b}, ind: [a,n])[a, n] == f[a, ind[a, n]]

It is an error to have an axis present in `self`, `axes_` and `indices`.

If several axes are passed, then the last axis of indices is treated as
coordinates:
- gatherValues(f: [a,b,c], .{.b, .c}, ind: [n,2])[a, n] == f[a,
ind[n][0], ind[n][1]]
- gatherValues(f: [a,b,c,d], .{.b, .c}, ind: [a, n,2])[a, n, d] == f[a,
ind[a, n][0], ind[a, n][1], d]

It is possible to use gatherValues without tags, but batching won't be
available.


> [!WARNING]
> The semantic of inserting the indices axis where we are dropping the
input axis is very useful in practice:
> `roll`, `nearest`, `resize` benefit greatly from it.
> But it doesn't always play nice with batching dimensions.
> In particular the following currently isn't accepted:
> `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .n = 8, .a = 10 })`
> because MLIR will produce `.{ .n = 8, .a = 10 }` following the indices
shape,
> while we produce `.{.a = 10, n=8}` following the semantic of
"replacing .b with axes from indices".
> If you do `gatherValues(.{ .{ .a = 10, .b = 20 }, .b, .{ .a = 10, .n =
8 })` then both convention agrees and it's a non issue.
> I think this inconsistency is ok for now because it only happens when
you have both batch axes and extra-indices axes which act like second
set of batching dimensions.
> the issue is avoidable from the user side by eg broadcasting the
values along `.n`, or transposing one of the operand.

---------

Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2024-09-19 16:17:07 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00