0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

Compare commits

...

31 commits

Author SHA1 Message Date
Corentin Godeau
82a23317ad zml/module.zig: increase SerializedExecutable size limit
On Neuron, some artifacts can reach more than 100MB.
2024-11-13 15:01:15 +00:00
Corentin Godeau
26637d52ce zml/nn: hack for sdpaMemEfficient 2024-11-13 11:58:14 +00:00
Corentin Godeau
fef181e0e8 zml/platform.zig: increase max device limit 2024-11-13 11:58:14 +00:00
Guillaume Wenzek
5b13dc8f42 add more scatterSlices testcases 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
80d1c4bad1 zml: remove helpers.mapTensors in favor of zml.meta.map 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
68e9c0c8e3 fix shape when for loop is fully unrolled 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
d116292fa8 zml.nn: properly handle overflows in top_p
fixup! zml.nn: properly handle overflows in top_p
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
64eedebc8b zml.Tensor.pad should accept i64 to be compatible with dims 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
93276ff728 allow to have array of tensor inside model struct 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
87f772c942 implement advanced sampling logic
fixup! implement advanced sampling logic
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
74d8bee54f iota should only take one dtype
currently iota accept a dtype and a shape, making one wonder which dtype is used.
the `withDtype` allow to elegantly change dtype on the fly if needed
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
cdee3fa0bd add DotAlgorim support to dot_general
for now it's not exposed to users because the API is a bit limited.
in particular only a subset of algorithm are supported by openxla,
and only a subset of that is supported by chips.

I was able to successfully change the accumulation type for dot on Cuda,
but want to experiment more before deciding what to expose.
2024-11-13 11:57:36 +00:00
Guillaume Wenzek
465a7a276e skip gather/scatter with batches tests 2024-11-13 11:57:36 +00:00
Guillaume Wenzek
1a474379ea pjrtx: error instead of panic 2024-11-13 11:57:36 +00:00
Steeve Morin
cfe4581f85 examples,runtimes: add layers for each runtime
Some runtimes require additional layers to be built. This change adds
a generic `layers` filegroup to each runtime. It should be embedded into
OCI images.
2024-11-08 17:49:55 +01:00
Steeve Morin
c748ee155e runtimes/neuron: rework neuronx-cc sandboxing
- use codegen for python environment setup
- rework use of runfiles to do that properly
- switch to the script version of the py_binary rule
2024-11-08 14:35:34 +01:00
Steeve Morin
7e88f7cee1 runtimes/neuron: add compiler env and flags
Notably:
- `--enable-fast-loading-neuron-binaries
- `--optlevel=1`: 30% speed up in compilation, no performance penalty on llama
- NEURON_RT_STOCHASTIC_ROUNDING_EN=1 for more precision
2024-11-08 14:35:34 +01:00
Steeve Morin
9901933eb3 runtimes/neuron: disable pyc precompiling
Given the time it takes to compile, I'm not sure it's worth it.
2024-11-08 14:35:34 +01:00
Corentin Godeau
4469db42c3 zml/module.zig: increase loaded module size 2024-11-08 10:59:35 +00:00
Steeve Morin
30716dd802 zml: rework device assignment when compiling the modul
First, do not copy `DeviceAssignmentProto` but use `@TypeOf` to fetch it.
Then, use a named block to set it inline.
2024-11-05 20:00:50 +01:00
Corentin Godeau
acedc63bb8 zml/module: disable donation for Neuron 2024-11-05 19:28:38 +01:00
Corentin Godeau
b311070dc5 runtimes/neuron: Set required env before Python initialization 2024-11-05 19:28:38 +01:00
Steeve Morin
bef2d3f009 runtimes/neuron: bump requirements 2024-11-05 19:28:38 +01:00
Steeve Morin
1b036259f2 +neuron 2024-11-05 19:28:38 +01:00
Steeve Morin
2110847c9a +neuron 2024-11-05 19:28:38 +01:00
Steeve Morin
c13a8fe278 runtimes/neuron: wchar handling 2024-11-05 19:28:38 +01:00
Steeve Morin
54873b8476 neurontmp 2024-11-05 19:28:35 +01:00
Steeve Morin
e60ebe064f NEURON 2024-11-05 19:26:39 +01:00
Steeve Morin
38e3bd2a04 neuron 2024-11-05 19:26:39 +01:00
Steeve Morin
3efc78d273 neuron 2024-11-05 19:26:39 +01:00
Steeve Morin
27b7d31b40 neuron 2024-11-05 19:26:39 +01:00
35 changed files with 8502 additions and 2254 deletions