0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

1,078 commits

Author SHA1 Message Date
Hugo Mano
833f42029d
platforms: bump cpu + cuda plugins and align xla (#626)
Related PR: https://github.com/zml/monorepo/pull/304
2026-07-03 14:22:23 +02:00
elogir
2057d68556
platforms/metal: bump plugin & remove metadata from struct (#620)
- Bump Metal plugin version
- Drop the attention metadata fields from the llama and lfm2 Sessions. Prefill and decode now each build their own metadata inline and free it.
2026-07-03 11:23:43 +02:00
Corentin Godeau
7349826476
zml/nn: improve sampleTokensDynamic() (#625)
This PR improves the `zml.nn.sampleTokensDynamic()` function. Most
notably:
* Convert parameters to input data type
* Support scalar/batched parameters
* Correctly use the temperature parameter
2026-07-02 17:02:51 +02:00
Corentin Godeau
51db372455
third_party/iree: bump iree tokenizer to 4d4e97d00f099a21f38eeff26f82a6d9e3643a11 (#624) 2026-07-02 15:46:02 +02:00
Olivier Boulant
28e5d9ed7f
pjrt: fix misleading pjrt plugin load success log (#622)
Move the PJRT plugin “Loaded” log so it is only emitted after the
dynamic library `.so` is opened and the PJRT API is initialized
successfully.

Previously, `Api.loadFrom` used a `defer` for the success log. Failed
plugin load attempts could still print `Loaded: ...`. This was
misleading during platform auto-detection, for example when probing TPU
before falling back to CUDA. Example:

```
2026-07-01 13:12:17   ███████╗███╗   ███╗██╗             ██╗    ██╗     ██╗     ███╗   ███╗██████╗
2026-07-01 13:12:17   ╚══███╔╝████╗ ████║██║            ██╔╝    ██║     ██║     ████╗ ████║██╔══██╗
2026-07-01 13:12:17     ███╔╝ ██╔████╔██║██║           ██╔╝     ██║     ██║     ██╔████╔██║██║  ██║
2026-07-01 13:12:17    ███╔╝  ██║╚██╔╝██║██║  .ai     ██╔╝      ██║     ██║     ██║╚██╔╝██║██║  ██║
2026-07-01 13:12:17   ███████╗██║ ╚═╝ ██║███████╗    ██╔╝       ███████╗███████╗██║ ╚═╝ ██║██████╔╝
2026-07-01 13:12:17   ╚══════╝╚═╝     ╚═╝╚══════╝    ╚═╝        ╚══════╝╚══════╝╚═╝     ╚═╝╚═════╝
2026-07-01 13:12:17                                                          Copyright (c) 2026 ZML
2026-07-01 13:12:17 warning(zml/io/vfs/s3): S3 initialized without full credentials (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY). Access will be unauthenticated. Using default endpoint: https://s3.amazonaws.com (set AWS_ENDPOINT_URL or AWS_ENDPOINT_URL_S3). Using default region: us-east-1 (set AWS_REGION or AWS_DEFAULT_REGION).
2026-07-01 13:12:17 info(pjrt): Loading: libpjrt_tpu.so...
2026-07-01 13:12:17 info(llmd): Loading tokenizer
2026-07-01 13:12:17 error(pjrt): Unable to dlopen plugin: /llmd/llmd.runfiles/zml++tpu_packages+libpjrt_tpu/sandbox/lib/libpjrt_tpu.so
2026-07-01 13:12:17 info(pjrt): Loaded: libpjrt_tpu.so           #################### HERE ####################
2026-07-01 13:12:17 info(llmd): Resolving model repo
2026-07-01 13:12:17 info(llmd): Loading model config
2026-07-01 13:12:17 info(llmd): Loaded model config [67.405ms]
2026-07-01 13:12:18 info(pjrt): Loading: libpjrt_cuda.so...
2026-07-01 13:12:18 warning(zml/platforms/cuda): Detected NVIDIA GPU that requires CUDA compatibility libraries.
2026-07-01 13:12:18 info(zml/platforms/cuda): Loaded CUDA compatibility libraries.
2026-07-01 13:12:18 info(pjrt): Loaded: libpjrt_cuda.so
2026-07-01 13:12:18 info(llmd): Loaded tokenizer [334.168ms]
2026-07-01 13:12:20 info(llmd): Devices:
2026-07-01 13:12:20 info(llmd):         - NVIDIA RTX PRO 6000 Blackwell Server Edition (cuda:0)
2026-07-01 13:12:20 info: Using 31711 pages for the KvCache
2026-07-01 13:12:20 info: sizes: { 256 }
2026-07-01 13:12:27 info: Compiled all models [7.336s]
2026-07-01 13:12:50 info: Listening on 0.0.0.0:8000
2026-07-01 13:12:50 info(llama): Loaded weights [14.96GiB, 22.778s, 672.41MiB/s]
```

Moreover make `fromDynLib` return errors for missing `GetPjrtApi` and
failed `PJRT_Plugin_Initialize` instead of panicking or using `catch
unreachable`. So that `Api.loadFrom()` can catch it.

After the change, the output is:
```
2026-07-01 14:50:50   ███████╗███╗   ███╗██╗             ██╗    ██╗     ██╗     ███╗   ███╗██████╗
2026-07-01 14:50:50   ╚══███╔╝████╗ ████║██║            ██╔╝    ██║     ██║     ████╗ ████║██╔══██╗
2026-07-01 14:50:50     ███╔╝ ██╔████╔██║██║           ██╔╝     ██║     ██║     ██╔████╔██║██║  ██║
2026-07-01 14:50:50    ███╔╝  ██║╚██╔╝██║██║  .ai     ██╔╝      ██║     ██║     ██║╚██╔╝██║██║  ██║
2026-07-01 14:50:50   ███████╗██║ ╚═╝ ██║███████╗    ██╔╝       ███████╗███████╗██║ ╚═╝ ██║██████╔╝
2026-07-01 14:50:50   ╚══════╝╚═╝     ╚═╝╚══════╝    ╚═╝        ╚══════╝╚══════╝╚═╝     ╚═╝╚═════╝
2026-07-01 14:50:50                                                          Copyright (c) 2026 ZML
2026-07-01 14:50:51 warning(zml/io/vfs/s3): S3 initialized without full credentials (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY). Access will be unauthenticated. Using default endpoint: https://s3.amazonaws.com (set AWS_ENDPOINT_URL or AWS_ENDPOINT_URL_S3). Using default region: us-east-1 (set AWS_REGION or AWS_DEFAULT_REGION).
2026-07-01 14:50:51 info(llmd): Resolving model repo
2026-07-01 14:50:51 info(llmd): Loading model config
2026-07-01 14:50:51 info(llmd): Loaded model config [101.024ms]
2026-07-01 14:50:51 info(llmd): Loading tokenizer
2026-07-01 14:50:51 info(pjrt): Loading: libpjrt_tpu.so...
2026-07-01 14:50:51 error(pjrt): Unable to dlopen plugin: /llmd/llmd.runfiles/zml++tpu_packages+libpjrt_tpu/sandbox/lib/libpjrt_tpu.so
2026-07-01 14:50:52 info(llmd): Loaded tokenizer [349.499ms]
2026-07-01 14:50:53 info(pjrt): Loading: libpjrt_cuda.so...
2026-07-01 14:50:53 warning(zml/platforms/cuda): Detected NVIDIA GPU that requires CUDA compatibility libraries.
2026-07-01 14:50:53 info(zml/platforms/cuda): Loaded CUDA compatibility libraries.
2026-07-01 14:51:11 info(pjrt): Loaded: libpjrt_cuda.so
2026-07-01 14:51:14 info: Using 31711 pages for the KvCache
2026-07-01 14:51:14 info: sizes: { 256 }
2026-07-01 14:51:14 info(llmd): Devices:
2026-07-01 14:51:14 info(llmd):         - NVIDIA RTX PRO 6000 Blackwell Server Edition (cuda:0)
2026-07-01 14:51:22 info: Compiled all models [8.395s]
2026-07-01 14:51:46 info: Listening on 0.0.0.0:8000
2026-07-01 14:51:46 info(llama): Loaded weights [14.96GiB, 23.629s, 648.20MiB/s]
```
2026-07-02 14:01:18 +02:00
Corentin Godeau
001dbbbde8
zml/tokenizer: Have matching tokenizer output (#623)
The IREE tokenizer was not producing the same output as the HF
tokenizer, potentially degrading the performance of the models. I put a
clanker to the task (because the tokenizer is already full clanked).
Outputs are now matching without any performance drop.

I also fixed an edge case issue on the `Writer` of the tokenizer.
2026-07-02 12:26:27 +02:00
Corentin Godeau
e8eb3ca689
zml/moe/triton: reduce log level (#621) 2026-07-01 18:10:32 +02:00
Guillaume Wenzek
e04ccb9ecb
zml: introduce zml.Compiler (#618)
Move the type out of `zml.compile` signature so that we can call
`io.async` more easily.

Sample code:

```
var layer_compile_future = io.async(zml.Compiler(TransformerLayer.forward).compile, .{
    allocator,
    io,
    platform,
    layer_compile_opts,
    .{
        model.model.layers[0],
        hidden,
        device_state.kv_cache.layers[0],
    },
});
```
2026-07-01 16:21:04 +02:00
Guillaume Wenzek
d6e9b47391
zml/examples/llm: simplify dispatch code (#619)
`@unionInit` ftw
2026-07-01 16:20:48 +02:00
Louis Lechevalier
0e12e4c998
platforms/cuda: update pjrt plugin (#617)
related: https://github.com/zml/pjrt-artifacts/pull/91/changes
From 130 to 200 tok/s for Gemma 4 26B A4B on GH200
2026-06-30 17:31:14 +02:00
Olivier Boulant
cb12e2ea71
zml/io: update defaults for zml.io.load and read parallelism 2026-06-30 15:58:00 +02:00
Reese Chong
89b0908ca5
zml/attention/flashattn: make fa2 window_size_left configurable (#595)
Add a sliding_window field to the FA2 parameters and use it to set the window_size_left argument in zml.attention.attention().
2026-06-29 17:37:13 +02:00
Reed Meyerson
be684a6dce
docs/tutorials: fix 'write_first_model' example code (#612)
Fix in the `write_first_model.md` tutorial: Update main() code section in to match full example
2026-06-29 17:32:32 +02:00
Damien Mehala
d1e4ef9e29
zml/tensor: fix segmentation fault when using split Tensor (#614)
Previously, split tensors where allocated using an arena that would be
free at the end of the function scope. This commit uses the arena
context instead.

Signed-off-by: dmehala <damien@zml.ai>
2026-06-29 14:54:07 +02:00
Corentin Kerisit
4c2393e5cf
bazel: scope Zig cache sandbox mounts to Linux (#615)
## Summary

- Limit Zig cache sandbox mount flags to Linux, where linux-sandbox and
Bazel's Docker sandbox consume host path mounts.
- Remove the macOS mount entry because sandbox-exec uses a symlinked
execroot and ignores bind-mount settings.
- Mark `/tmp` and `/var/tmp` writable alongside the Linux mount pairs.
2026-06-29 14:43:42 +02:00
elogir
6251953668
platforms/metal: add moe support (#613) 2026-06-29 10:43:40 +02:00
Corentin Kerisit
076417363a
workspace: add missing loads (#606)
nits
2026-06-27 00:51:55 +02:00
Corentin Kerisit
4792fba3c1
bazel: improve remote build resilience (#610)
Enable lost-input rewinding and local fallback for the remote
downloader.
2026-06-27 00:26:50 +02:00
Olivier Boulant
1dbe39d3e8
zml/io/vfs: add parallel read to GCS and S3 (#604)
# Summary

This PR extends the support to chunked parallel read introduced
initially for HF in the VFS, to GCS and S3.

It:
- Adds a shared `zml/io/vfs/parallel_read.zig` helper for chunked
parallel HTTP range reads
- Add configurable read pool options for S3/GCS initialization
- Register the `gs://` VFS backend in `examples/llm`.


# Tests

## S3

We used [MINIO](https://hub.docker.com/r/minio/minio) docker image from
the `5090` GPU machine. From there, we mirror the
`meta-llama/Llama-3.1-8B-Instruct` model through the S3 server.

Then from the `4090` GPU machine, we run 

```bash
bazel run --@zml//platforms:cuda=true --@zml//platforms:cpu=false --config=release --config=native //examples/llm -- --model=s3://zml-test/Llama-3.1-8B-Instruct --prompt="hello, how are you ?"
```

It works both on 1 and 2 GPU. 

## GCS

In the same spirit, we put the `meta-llama/Llama-3.1-8B-Instruct` in a
GCS bucket we own. Then, on the `4090`, we run

```bash
bazel run --@zml//platforms:cuda=true --@zml//platforms:cpu=false --config=release --config=native //examples/llm -- --model=gs://zml-mirror2/models/meta-llama/Llama-3.1-8B-Instruct --prompt="hello, how are you ?"
```

It works both on 1 and 2 GPU. 

### Performance

As a reference, we run:
```bash
gcloud storage cp gs://zml-mirror2/models/meta-llama/Llama-3.1-8B-Instruct/model-00001-of-00004.safetensors gs://zml-mirror2/models/meta-llama/Llama-3.1-8B-Instruct/model-00002-of-00004.safetensors gs://zml-mirror2/models/meta-llama/Llama-3.1-8B-Instruct/model-00003-of-00004.safetensors gs://zml-mirror2/models/meta-llama/Llama-3.1-8B-Instruct/model-00004-of-00004.safetensors /tmp/gcs-gcloud-llama-shards/
```

It gives:
```
Completed files 4/4 | 15.0GiB/15.0GiB | 942.7MiB/s 
Average throughput: 918.8MiB/s
```

Using the VFS in various setups:

* `dma_chunks = 32`, `dma_chunk_size = 128`, `parallelism = 16`,
`chunk_size = 10 MiB`, `num_workers = 16`, `queue_capacity = 64`

```bash
info (llama): Loaded weights [14.96GiB, 21.302s, 719.02MiB/s]
```

* `dma_chunks = 32`, `dma_chunk_size = 256`, `parallelism = 16`,
`chunk_size = 10 MiB`, `num_workers = 16`, `queue_capacity = 64`

```bash
info (llama): Loaded weights [14.96GiB, 18.227s, 840.32MiB/s]
```

* `dma_chunks = 32`, `dma_chunk_size = 256`, `parallelism = 16`,
`chunk_size = 16 MiB`, `num_workers = 16`, `queue_capacity = 64`

```bash
info (llama): Loaded weights [14.96GiB, 17.376s, 881.43MiB/s]
```

* `dma_chunks = 32`, `dma_chunk_size = 256`, `parallelism = 16`,
`chunk_size = 16 MiB`, `num_workers = 32`, `queue_capacity = 64`

```bash
info (llama): Loaded weights [14.96GiB, 17.28s, 886.36MiB/s]
```

* `dma_chunks = 32`, `dma_chunk_size = 512`, `parallelism = 16`,
`chunk_size = 16 MiB`, `num_workers = 16`, `queue_capacity = 64`

```bash
info (llama): Loaded weights [14.96GiB, 18.658s, 820.90MiB/s]
```
2026-06-26 19:24:31 +02:00
Corentin Kerisit
3099106f17
platforms: set linux amd64 legacy cpu (#607)
part of highway XLA target selects which header to include based on
legacy --cpu flags but the rest not...
So even if we targeted linux amd64, the top part of the highway would be
used correctly, but the extension specific part would fallback to --cpu
darwin_arm64 and would not include hh_avx2.h resulting in compilation
errors.

The normal solution here is to use platform_mappings but since we pass
`--force_pic=true` in platforms, this takes precedence and the mappings
are ignored.

The temporary solution while https://github.com/openxla/xla/pull/44823
is merged, is to pass `--cpu` in all platforms we use for cross
compilation.
2026-06-26 15:35:27 +02:00
Louis Lechevalier
c2cbcb82f1
zml/attention: Optimize heuristic and parameters choices of triton attention (#605)
Limits the number of segments for the 3d kernel. Effect : reduce the
cost of the reduce kernel which follows.
2026-06-26 14:44:48 +02:00
elogir
a4993854da
platforms/metal: sandbox Metal toolchain (#598)
Fetch Apple's encrypted Metal toolchain and only exposes the minimal
compile tools + runtime needed
2026-06-26 14:37:56 +02:00
Corentin Kerisit
aa90f92832
workspace: bump to bazel 9.1.1 (#601)
A few missing loads
Remove all flipped flags or deperacted flags from .bazelrc

Important:
- Keep autoload rules_cc and rules_python because XLA still has
transitive deps that are missing loads, PR in progress.
2026-06-25 19:37:51 +02:00
Kevin Barre
33ced8fa07
zml/backend: add oneAPI as Triton backend selector for MoE (#603) 2026-06-25 15:57:35 +02:00
Corentin Godeau
c023e26f2c
zml/moe: enable moe for any compute capability (#600) 2026-06-25 14:32:44 +02:00
Corentin Kerisit
f9ec63fd4e
workspace: Update to XLA @ 5e83525729fd21c4ad8d55073a3d047360d13f9c (#599)
Updates to latest XLA which added support for bazel 8.x workspace and
fixed a bunch of things that made bazel 9 impossible.

3 problematic changes with this bump:
- PJRT headers were again incompatible with translate-c so a patch is
made and PR has been done in XLA
- PJRT introduced a proper type fo PJRT_Error so I used the InnerMixin
pattern for those.
- //tsl/platform/default package force evaluates Label("@cuda_nvcc")
even for CPU builds and we don't have this repo. I commented for now
until I find a solution on XLA side.

Bumped a few deps necessary for this and additional llvm deps that are
now evaluated as part of the llvm-project graph on XLA side (llvm_zlib
and llvm_zstd).

Next will be to investigate bazel 9 for real.
2026-06-25 14:17:46 +02:00
Olivier Boulant
d19949e255
zml/io/vfs(hf): introduce parallel read (#594)
---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-06-24 17:45:39 +02:00
elogir
1d71ee7bd3
zml/safetensors: limit reads to 1GB (#597)
On macOS, `pread` returns INT_MAX when the requested size is larger than 2GB
2026-06-24 16:50:01 +02:00
elogir
33a3b37900
platforms/metal: add Metal support (#593)
Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-06-24 09:54:11 +02:00
Louis Lechevalier
d942ce014a
platforms/tpu: bump plugin from v0.0.40 to v0.0.42 (#596) 2026-06-23 17:51:37 +02:00
elogir
f586cbfc3d
third_party/iree: match all added tokens (#591)
IREE's tokenizer only registered added tokens for pre-segmentation
matching if they were flagged special or normalized. Tokens that are
neither (e.g. <think> was dropped)

The fix is to match all added tokens
2026-06-23 09:16:21 +02:00
Kevin Barre
a1acbd36d5
zml/onapi: set defaults for oneAPI PjRT create options (#592)
same as https://github.com/zml/zml/pull/579
And update the pjrt plugin with the collective fix
2026-06-23 05:41:40 +02:00
elogir
5131467252
zml/attention: tune triton kernel for oneAPI performance (#588)
Speeds up paged-attention decode on the Intel/oneAPI backend
2026-06-22 19:06:48 +02:00
Louis Lechevalier
7c3b60202d
zml/sharding : introduce shardableDim api (#583)
This PR introduces shardableDim helper in Sharding to handle the case
where : (dimension % number of sharding axes) != 0

Useful to stutter and shard correctly kv heads dim for example.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-06-22 18:52:35 +02:00
Tristan Gautié
6ca94f34bd
attention: support is_causal option in Triton and Mosaic paged-attn kernels (#587)
We add suport for `is_causal` option in Triton and Mosaic paged-attention
kernels, by adapting the mask-generation logic
2026-06-22 15:25:02 +02:00
Corentin Godeau
984b60b75e
zml: various QoL improvements (#585)
These are QoL improvements to problems I stumbled upon while working on
#586
2026-06-20 17:58:18 +02:00
Kevin Barre
4cd7139477
zml/platform: add hermetic oneAPI 2026 runtime deps (#582)
Fetch oneAPI runtime libraries from locked DEB packages and wire the oneAPI sandbox packaging to use those repositories.
2026-06-19 18:30:57 +02:00
Kevin Barre
2cc6a541b3
zml/build: bump bazel to 8.7.0 (#584) 2026-06-19 15:15:41 +02:00
Hugo Mano
56a6a4db64
zml/moe: enable triton moe kernels on ROCm (#580) 2026-06-19 11:38:58 +02:00
Hugo Mano
ec776c53e0
zml/platform: set defaults for ROCm PjRT create options (#579)
ROCm was using only 0.75 of available memory. Align with CUDA, this
permit us to have Gemma 4 26B A4B on 2xRTX 9700 for example.
2026-06-18 16:17:12 +02:00
Hugo Mano
700b57466b
zml/io: expose DMA chunk windows to readers from DirectMemoryWriter (#576)
DirectMemoryWriter used to expose the next shard-sized slice as its writable
window. With HF/VFS this often produced tiny reads, even when adjacent tensor
bytes could be streamed through a larger direct-load window.

This PR changes direct loading to use an alias-window over the active primary
shard DMA buffer. The public `std.Io.Writer` buffer still points at real DMA
memory, but its visible length can coalesce across dispatch spans until a DMA
chunk, shard buffer, or tensor boundary is reached. On commit, the first active
primary segment stays zero-copy, while crossed spans and mirrors are scattered
before any buffer is submitted or rotated.

`DirectShardWriter` keeps ownership of DMA pool rotation, flip-flop events, and
PJRT transfer state. `DirectMemoryWriter` only walks the precomputed dispatch
spans with a small cursor.
2026-06-18 15:35:09 +02:00
Reese Chong
85e71e405c
zml/moe: fix - move allReduce into manual computation block (#577)
Move the MoE allReduce inside the manual computation block.

Previously, zml.moe.forwardMoe placed allReduce outside the manual
computation block which caused XLA SPMD partitioning to throw the 
following error with certain models.

"Cross-partition allreduce must be in (partial) manual partitioning mode."
2026-06-16 17:58:35 +02:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
Louis Lechevalier
be29637c07
zml/moe: Mosaic TPU MoE backend. (#571)
This PR add the TPU backend for MoE. It uses the GMM Expert parallelism
kernel to perform the grouped gemms on gate_up and down weights (these
weights must have a precise shape to run optimally).

This backend is currently optimized for small batches. A specialized
mosaic tpu kernel will be introduced to handle bigger ones.

This implementation come with few fixes concerning the TPU sharding and
the Qwen 3.5 moe model which uses the backend :
- TPU sharding : PJRT reports TPU device coordinates in a max rank form.
So it breaks folding. To avoid this we compute the Rank by not counting
where coords =0.
- Linear attention was reading padding token in the Qwen 3.5 model. So
we introduce masking of them.

n.b. : The kv head repetition to satisfy the sharding will be handled a
more generic way in a next PR

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2026-06-16 10:06:52 +02:00
Benjamin Rabier
1278805b87
zml/pjrtx: alignCast the host buffer for host offloading (#575) 2026-06-12 15:03:34 +02:00
Benjamin Rabier
5d4e4db01c
zml/profiling/tracer: Add simpler infaillible span (#573)
Renamed `scope` to `span` to match OpenTelemetry convention I'm more
familiar with. Traceme doesn't seem to use `Scope` explicitly but talks
more about a scope object AFAIK.

Now `span` fails at compile time. if it cannot compute the appropriate
buffer size for the name. In that case the caller should use
`formatSpanName` which allocates. This makes `span` infaillible at
runtime.
2026-06-09 16:10:34 +02:00
Corentin Godeau
c5bdcac632
zml/sharding: fix max_rank for TPU (#572)
Topology coords on TPU have rank 4 and not 3
2026-06-04 19:04:34 +02:00
Corentin Godeau
39ffd27f6f
zml/module: expose autotune cache dir through an env var (#570)
This opt-in flag allows XLA to save the autotuning resulting from
previous runs in a folder, greatly speeding up the compilation.

It's not meant to be used at all times, so it's undocumented and opt-in.
2026-06-04 15:32:17 +02:00
Louis Lechevalier
4b0c7ed1c0
zml/moe : Mosaic GMM EP TPU kernel. DSL update. (#569)
This PR introduce the new GMM Expert parallelism kernel (basis grouped
gemm for moe on TPU). It induces some updates into the DSL mosaic TPU,
such as Builders helpers or Pipelines and schedulers :
- Buffer counts (for scheduling overlapping) > 2 allowed (until 7)
- source_reshape_shape reshaped at DMA slice/use sites to match Pallas
IR placement
- More grid dimensionality for pipeline (up to 4)
2026-06-03 15:58:16 +02:00
Steeve Morin
f9a282162a zml: fix multiple enabled platforms
more than one platform can be enabled at the same time
2026-06-01 15:53:48 +00:00