0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

90 commits

Author SHA1 Message Date
Corentin Godeau
82a23317ad zml/module.zig: increase SerializedExecutable size limit
On Neuron, some artifacts can reach more than 100MB.
2024-11-13 15:01:15 +00:00
Corentin Godeau
26637d52ce zml/nn: hack for sdpaMemEfficient 2024-11-13 11:58:14 +00:00
Corentin Godeau
fef181e0e8 zml/platform.zig: increase max device limit 2024-11-13 11:58:14 +00:00
Guillaume Wenzek
5b13dc8f42 add more scatterSlices testcases 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
80d1c4bad1 zml: remove helpers.mapTensors in favor of zml.meta.map 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
68e9c0c8e3 fix shape when for loop is fully unrolled 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
d116292fa8 zml.nn: properly handle overflows in top_p
fixup! zml.nn: properly handle overflows in top_p
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
64eedebc8b zml.Tensor.pad should accept i64 to be compatible with dims 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
93276ff728 allow to have array of tensor inside model struct 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
87f772c942 implement advanced sampling logic
fixup! implement advanced sampling logic
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
74d8bee54f iota should only take one dtype
currently iota accept a dtype and a shape, making one wonder which dtype is used.
the `withDtype` allow to elegantly change dtype on the fly if needed
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
cdee3fa0bd add DotAlgorim support to dot_general
for now it's not exposed to users because the API is a bit limited.
in particular only a subset of algorithm are supported by openxla,
and only a subset of that is supported by chips.

I was able to successfully change the accumulation type for dot on Cuda,
but want to experiment more before deciding what to expose.
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
465a7a276e skip gather/scatter with batches tests 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
1a474379ea pjrtx: error instead of panic 2024-11-13 11:57:36 +00:00
Steeve Morin
cfe4581f85 examples,runtimes: add layers for each runtime
Some runtimes require additional layers to be built. This change adds
a generic `layers` filegroup to each runtime. It should be embedded into
OCI images.
2024-11-08 17:49:55 +01:00
Steeve Morin
c748ee155e runtimes/neuron: rework neuronx-cc sandboxing
- use codegen for python environment setup
- rework use of runfiles to do that properly
- switch to the script version of the py_binary rule
2024-11-08 14:35:34 +01:00
Steeve Morin
7e88f7cee1 runtimes/neuron: add compiler env and flags
Notably:
- `--enable-fast-loading-neuron-binaries
- `--optlevel=1`: 30% speed up in compilation, no performance penalty on llama
- NEURON_RT_STOCHASTIC_ROUNDING_EN=1 for more precision
2024-11-08 14:35:34 +01:00
Steeve Morin
9901933eb3 runtimes/neuron: disable pyc precompiling
Given the time it takes to compile, I'm not sure it's worth it.
2024-11-08 14:35:34 +01:00
Corentin Godeau
4469db42c3 zml/module.zig: increase loaded module size 2024-11-08 10:59:35 +00:00
Steeve Morin
30716dd802 zml: rework device assignment when compiling the modul
First, do not copy `DeviceAssignmentProto` but use `@TypeOf` to fetch it.
Then, use a named block to set it inline.
2024-11-05 20:00:50 +01:00
Corentin Godeau
acedc63bb8 zml/module: disable donation for Neuron 2024-11-05 19:28:38 +01:00
Corentin Godeau
b311070dc5 runtimes/neuron: Set required env before Python initialization 2024-11-05 19:28:38 +01:00
Steeve Morin
bef2d3f009 runtimes/neuron: bump requirements 2024-11-05 19:28:38 +01:00
Steeve Morin
1b036259f2 +neuron 2024-11-05 19:28:38 +01:00
Steeve Morin
2110847c9a +neuron 2024-11-05 19:28:38 +01:00
Steeve Morin
c13a8fe278 runtimes/neuron: wchar handling 2024-11-05 19:28:38 +01:00
Steeve Morin
54873b8476 neurontmp 2024-11-05 19:28:35 +01:00
Steeve Morin
e60ebe064f NEURON 2024-11-05 19:26:39 +01:00
Steeve Morin
38e3bd2a04 neuron 2024-11-05 19:26:39 +01:00
Steeve Morin
3efc78d273 neuron 2024-11-05 19:26:39 +01:00
Steeve Morin
27b7d31b40 neuron 2024-11-05 19:26:39 +01:00
Steeve Morin
fad7cc0876
zml: reintroduce pjrtx (#78)
In an async scenario, it is very easy to footgun yourself and block the
reactor, especially with Events.

Reintroduce pjrtx that deals with the specifics.
2024-11-05 19:25:50 +01:00
Steeve Morin
44d1b24beb
async: add threaded implementation (#77)
This implements the `async` module using a thread pool. This is now the
default.

Implementations can be swapped using the
`@zml//async:impl=[threaded,zigcoro]` flag.

Because io_uring is not enabled in the default seccomp profile, some
container implementations will block on io_uring calls. Use the threaded
implementation to work around this, for now.
2024-11-05 16:47:41 +01:00
Guillaume Wenzek
f661e1a45d
zml/aio: allow metadata to be read from index.json (#76)
For sharded safetensor files, we already have a separate json file we
need to read.
Let's allow writing metadata there, to avoid having a separate json file
with the model config.
2024-11-05 16:45:45 +01:00
Guillaume Wenzek
2febb10baa
zml.tensor: add triangular and toDiagonal (#75)
- triangular: zeroes out the upper right part of a matrix. Options
control how much is actually zeroes with offset from the diagonal.
- toDiagonal: equivalent to pytorch "diag_embed". The base case is
converting a vector into a diagonal matrix. We already have a `diag`
operator doing this, but the name was wrong cause in general `diag` does
the reverse operation of extracting the diagonal from a matrix into a
vector. Also it was using old style ZML, and was a bit too restrictive
on input shapes.
- elu: one more activation function. it's getting a bit out of hand, I've put it under zml.nn.Activation
2024-11-04 15:30:55 +01:00
Guillaume Wenzek
cb0bcf2f7b
zml.tensor: add cumulativeSum, small refactor of maxPoolND (#64)
This PR adds the "cumulative sum" operator.

This is based on what jax does, which is leveraging reduce window
operator,
where for each n in [0..N] you compute the some of elements [0..n] 
I'm not sure how this get optimized in complex programs, to be
investigated.

**BREAKING CHANGES**:
* `reduceWindow` is still pretty hard to used but I merged the
`padding_shape` and `padding_value` into one operator and made the
signature more explicit.
* this also lead me to change a bit how `maxPool1D` and `maxPool2D` take
arguments, using proper tuples instead of slices that need to be of
specific len.
* `pad` now take options using 
     * "tagged syntax":`x.pad(0, .{ .a = .{ .low = 1, .high = -1 }});` 
* or the "AOS syntax": `x.pad(0, &.{.{}, { .low = 1, .high = -1 }});`
* the "SOA syntax" is **REMOVED**: `x.pad(0, .{ .low = &.{1}, .high =
&.{-1} });`
2024-11-04 09:45:32 +01:00
Steeve Morin
09f8506763
runtimes: add intermediate abstraction layer (#74)
Add a tiny abstraction layer between ZML and PJRT.

Most notably, to implement custom logic to decide wether or not we
should/can load a PJRT plugin.

For instance, `libtpu` will block the program trying to reach the GCP
metadata server, which doesn't exist outside GCP.

This also allows for comptime detection of wether a runtime is linked
with the program, and its dependencies.
2024-11-01 18:07:33 +01:00
Steeve Morin
ee6aaea850
zml: configure runfiles env at context start (#73)
If we start bazel built binaries inside a ZML process, the environment
has to be configured properly for the sub processes to find their
runfiles properly.
Do it process wide at context start.
2024-11-01 17:44:47 +01:00
Steeve Morin
a3a5a6336c
zml,pjrt: account for runtimes that not implementing Events (#72)
Some runtimes do not implement `PJRT_Event`. For those, we must dispatch
on our own thread pool for buffer copies and wait on events only if the
runtime returns them.

If there are no events, assume all operations are synchronous.
2024-11-01 17:38:14 +01:00
Steeve Morin
ccd925309b
async: rename async_ to asyncc and add Generic slugs (#71)
`asyncc` is arguably nicer to write than `async_`, so use that.

Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
2024-11-01 17:36:08 +01:00
Steeve Morin
be510cbb6b
bazel,runtimes/rocm: use dynamic symbol import renaming (#70)
Currently, in ROCm, we use
`__attribute__((ifunc("_zml_rocm_resolve_dlopen")))` to make sure
`dlsym(RTLD_NEXT, "dlopen")` will resolve to the real `dlopen` and not
the always failing stub.

This is a bit brittle.

Instead, leverage `patchelf`'s `rename_dynamic_symbols` feature to
import a custom `dlopen` implementation, and forward to the real one
normally.
2024-11-01 17:33:57 +01:00
Hugo Mano
62a05cc2f8
zml/tensor: use stablehlo.logistic for silu and remove logistic to implement it in sigmoid [BREAKING CHANGE] (#69)
For `silu`, `logistic` is more precise than `sigmoid` since the first
one is bound to StableHLO while the second one is a custom
implementation. Implemented `logistic` implementation in `sigmoid`
and aliased `logistic` to `sigmoid`.
2024-10-29 11:23:33 +01:00
Guillaume Wenzek
468e432fca
zml/testing: handle all float types in tests (#66)
Handle all float types. Before we would crash if the reference tensor
was in bfloat16.
2024-10-23 17:39:26 +02:00
Corentin Godeau
c61ef6bd1d
Fix LSP in Neovim (#65)
`zig_exe_path` needs to be set for the LSP to work properly
2024-10-23 14:07:48 +02:00
Farookh Zaheer Siddiqui
7f6dbbe289
[Docs] : fix typos (#60) 2024-10-22 17:38:05 +02:00
Victor Moin
db665ba706
Add missing load in dockerize_models.md (#62)
Added the missing import `zig_cc_binary` in the simple layer example,
even though it seems obvious when you read the error
2024-10-22 17:35:51 +02:00
Guillaume Wenzek
4a5c579a10
zml: concatenate tag support (#61)
* add support for `Tensor.concatenate(&.{x, y}, .d)`
* start removing some `broadcastLeft` usage
* use f32 to compute head scaling constant in transformer. We should
always compute constant with max precision.
2024-10-18 12:33:46 +02:00
Guillaume Wenzek
edc0454d6c
zml.aio.torch: more refactor (#59)
* use more complicated test cases with long integers
* add tests for protocol 0. Pickle as evolve a lot since protocol 0 was
there,
and it's unlikely we receive such pickle, I thought it was justified
since a big part of the code is related to protocol 0 anyway.
* add tests to the "zipped pickle"
* simplify structs hierarchy -> `PickleData` has been merged with
`Parser` into `torch.File`.
* rename `torch/value.zig` -> `torch/py_object.zig`
* intermediary results: []pickle.Op, []py_object.AnyPy, should not be
stored inside objects but passed around function calls.
* we should use buffered reader to parse the pickle and the zip headers
* python Dict should not be represented by a singleton seq of seq of seq
of kv, just a single seq with keys and values following each other is
enough (like in pickle protocol)
* kwargs should not be mixed with args
* added harder pytorch test with a raw tensor and a `torch.nn.Conv2d`
module
* simplified a bit lifecylcle of `BufferStore` by passing the files
during init.
2024-10-18 11:32:38 +02:00
Corentin Godeau
7a35d53ea3
Add Llama 3.1 70B Instruct model (#56)
This can be handy to test beefier models on powerful accelerators
2024-10-17 17:05:24 +02:00
Corentin Godeau
cb31e4a749
Llama weights sharding (#58)
This PR introduces sharding attributes to Llama weights to take
advantage of Tensor Parallelism
2024-10-17 17:05:10 +02:00