0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

16 commits

Author SHA1 Message Date
Guillaume Wenzek
b2f0e03043
zml/Tensor: explicit sharding in Tensor code (#868)
In Buffer code we already have explicit sharding, but not for tensor.

I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.

This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.

There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.

`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.

This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.

`Placement` logic also got a bit simpler.
2026-10-06 15:37:22 +02:00
Corentin Godeau
f37e845672
zml/safetensors: support hf snapshots (#872)
This PR adds support for models downloaded via `hf download` without the
`--local-dir` flag.

This is done by keeping the given path instead of getting it from the
`std.Io.File`. The problem with `File.realpath` is that in the case of
models downloaded from HF, the files points to blobs whose name are hex
strings, thus losing the required file extension. Unfortunately, we rely
on the file extension to detect the kind of repository the path
represents.

I also added `//examples/io:playground` to the `//:completion` target so
that we have completion when working on it.
2026-09-29 18:23:08 +02:00
Raphael Nahoum
3178484b27
zml/io: record computed bindings in the store (#749)
A binding created through `maybeCreateBinding` is computed from its
sources by an executable and must never be streamed from the checkpoint.
`Loader.defaultCallback` inferred that from `sources.len != 1`, which a
single-source binding fails resulting to the raw bytes being loaded and
producing wrong results
2026-09-02 12:51:09 +02:00
Guillaume Wenzek
2b9320fe72
//zml/io/vfs: promote to //vfs (#680)
Simplify vfs architecture: merge `//zml/io` and `//zml/io/vfs`. They move to top level `@zml//vfs`

The PR also include two bug fix:
- S3: when no content length we were trying to read exactly 1M, while we
should read up to 1M
- GCP: restore the commented code for parsing bucket info

## Test plan:

```
bazel run --config=debug //examples/io:playground -- ls gs://gcp-public-data-landsat/
bazel run --config=debug //examples/io:playground -- tree s3://noaa-goes19/ABI-Flood-Day-Shapefiles/2025/08
```
2026-07-30 14:21:36 +02:00
Corentin Godeau
577d6452bf
zml/io: rework zml.io.load (#586)
The old `zml.io.load` was plagued with problems. This PR is an attempt
at improving the state of things.

One major problem of `zml.io.load` was that it was not very composable.
It was either using it to load all the model or doing everything by
hand.

Another problem was that it didn't offer a way to apply transformations
at load time to the tensors stored in a safetensors file. For example,
it happens that for MoE models, instead of the experts weights to be
tightly packed in a nice 3D tensor, the weight file contain one tensor
for each expert. This is nice until your implementation required a 3D
tensor as input. Using the old `zml.io.load` API this resulted in very
convoluted way to do concatenation at load time. With this new API, we
offer convenience so that it can be handled nicely and as fast as
possible.

The main change is the introduction of the `Loader` struct. It's a
struct that you instantiate once and reuse whenever you want to load
tensors. It relies on `Binding`, which are a way to store which tensor
are required and how they need to be combined (if it's a complex binding
involving more than one source tensor).

One advantage of this `Loader` is that it's composable. Loading a struct
using `Loader.auto(Struct)` or loading each individual field of `Struct`
has essentially the same behavior. This solved the major problem of the
old solution.

For simplicity I kept the old model loading code almost identical, as
this new solution is a drop-in replacement.

# Update

After some discussions with @gwenzek I've changed my mind. Instead of
having a declarative API, we'll provide helper to do transformation
easily. The main changes that this PR now bring are:
* Introduce `Loader` struct
* `Loader` exposes `load` which is the composable replacement of
`zml.io.load`
* `Loader` exposes `loadExecute` that allows you to execute an
executable with the buffers loaded from the sources associated with a
tensor. This allows for example to concatenate tensors.

We'll continue to work on it, this is just a preliminary work that
allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
Olivier Boulant
cb12e2ea71
zml/io: update defaults for zml.io.load and read parallelism 2026-06-30 15:58:00 +02:00
Hugo Mano
700b57466b
zml/io: expose DMA chunk windows to readers from DirectMemoryWriter (#576)
DirectMemoryWriter used to expose the next shard-sized slice as its writable
window. With HF/VFS this often produced tiny reads, even when adjacent tensor
bytes could be streamed through a larger direct-load window.

This PR changes direct loading to use an alias-window over the active primary
shard DMA buffer. The public `std.Io.Writer` buffer still points at real DMA
memory, but its visible length can coalesce across dispatch spans until a DMA
chunk, shard buffer, or tensor boundary is reached. On commit, the first active
primary segment stays zero-copy, while crossed spans and mirrors are scattered
before any buffer is submitted or rotated.

`DirectShardWriter` keeps ownership of DMA pool rotation, flip-flop events, and
PJRT transfer state. `DirectMemoryWriter` only walks the precomputed dispatch
spans with a small cursor.
2026-06-18 15:35:09 +02:00
Guillaume Wenzek
0979289ffe
zml/sharding: pass sharding by pointer and simplify replicated usage (#514)
This PR simplifies the common replicated case while making sharding
intent explicit and use sharding by pointer ( sharding struct is quite
big: ~600 bytes).

Using pointers for shardings also avoids unnecessary sharding copies and
makes it easier to share stable sharding instances across compilation,
loading, and runtime paths.

- `Platform` owns the canonical replicated sharding
- `Buffer` APIs now take explicit sharding specs:
    - `.replicated`
    - `.{ .sharded = sharding_ptr }`
- `createTensor` / `maybeCreateTensor `now require explicit partitioning
instead of accepting null
- compilation/loading now use pointer-based sharding lists

This makes simple cases like mnist and benchmark much cleaner: callers
no longer need to manually construct and thread a replicated sharding
just to use the default case, but they still state placement explicitly
at the call site.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-05-06 12:46:15 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
KB
8edf02d0c3
zml/chore: bump Zig to 0.16.0-dev.3133 and adapt codebase (#462)
Update Zig compiler to 0.16.0-dev.3133+5ec8e45f3 and adjust code
for breaking standard library changes:

- Replace ArrayListUnmanaged/ArrayList init with .empty
- Switch to unmanaged AutoArrayHashMap/StringArrayHashMap
- Use debug.GeneralPurposeAllocator (renamed upstream)
- Replace std.Io.File.ReadStreamingError with error.EndOfStream
- Refactor meta.visit to use structured visitor pattern
- Fix alignment allocation in mem.bufferizeInner

Also update third-party dependencies: flashattn 0.0.6-rc8, arocc,
uucode, libvaxis, and zigimg patches.

Co-authored-by: Kevin Barre <kevin@zml.ai>
2026-04-13 11:56:32 +02:00
Corentin Godeau
ac9bc32d89
zml/attention: add Triton Unified Attention backend (#417)
This PR introduces a Triton paged attention backend, unlocking ROCm
support.

It relies on a Python sandbox to generate the TTIR.

The major change is that the KVCache now needs to be per-layer for paged
attention. The `layer_index` is now superfluous and will probably be
removed in the future.
2026-04-13 11:39:08 +02:00
Steeve Morin
adee932efa
zml/io/vfs: add GCS VFS (#453)
This pull request adds support for Google Cloud Storage (GCS) to the
virtual file system (VFS) module, alongside some build configuration
updates. The main changes introduce the new `GCS` backend, make it
available in the VFS API, and register it in the example application.

Authentication is handled like other GCS libraries via the
`GOOGLE_APPLICATION_CREDENTIALS` env or application default account
files.

**IMPORTANT NOTE**: Only Authorized User and Metadata Server auth are
implemented. Service Account auth isn't implemented yet.
2026-04-07 19:05:43 +02:00
Corentin Godeau
2eb8423e8a
Update doc for V2 (#428) 2026-03-24 17:00:10 +01:00
Benjamin Rabier
a011fb0e52
Green CI (#419)
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
2026-03-20 10:31:57 +01:00
Corentin Godeau
70e4c9953d
Bump zig to 0.16.0-dev.2722+f16eb18ce (#409)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-10 15:58:48 +01:00
Hugo Mano
abde0a04dd
zml: sharding (#407)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-10 11:17:57 +01:00