0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

246 commits

Author SHA1 Message Date
Louis Lechevalier
7c3b60202d
zml/sharding : introduce shardableDim api (#583)
This PR introduces shardableDim helper in Sharding to handle the case
where : (dimension % number of sharding axes) != 0

Useful to stutter and shard correctly kv heads dim for example.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-06-22 18:52:35 +02:00
Hugo Mano
700b57466b
zml/io: expose DMA chunk windows to readers from DirectMemoryWriter (#576)
DirectMemoryWriter used to expose the next shard-sized slice as its writable
window. With HF/VFS this often produced tiny reads, even when adjacent tensor
bytes could be streamed through a larger direct-load window.

This PR changes direct loading to use an alias-window over the active primary
shard DMA buffer. The public `std.Io.Writer` buffer still points at real DMA
memory, but its visible length can coalesce across dispatch spans until a DMA
chunk, shard buffer, or tensor boundary is reached. On commit, the first active
primary segment stays zero-copy, while crossed spans and mirrors are scattered
before any buffer is submitted or rotated.

`DirectShardWriter` keeps ownership of DMA pool rotation, flip-flop events, and
PJRT transfer state. `DirectMemoryWriter` only walks the precomputed dispatch
spans with a small cursor.
2026-06-18 15:35:09 +02:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
Louis Lechevalier
be29637c07
zml/moe: Mosaic TPU MoE backend. (#571)
This PR add the TPU backend for MoE. It uses the GMM Expert parallelism
kernel to perform the grouped gemms on gate_up and down weights (these
weights must have a precise shape to run optimally).

This backend is currently optimized for small batches. A specialized
mosaic tpu kernel will be introduced to handle bigger ones.

This implementation come with few fixes concerning the TPU sharding and
the Qwen 3.5 moe model which uses the backend :
- TPU sharding : PJRT reports TPU device coordinates in a max rank form.
So it breaks folding. To avoid this we compute the Rank by not counting
where coords =0.
- Linear attention was reading padding token in the Qwen 3.5 model. So
we introduce masking of them.

n.b. : The kv head repetition to satisfy the sharding will be handled a
more generic way in a next PR

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2026-06-16 10:06:52 +02:00
Hugo Mano
cd941143dc
examples/llm: remove useless sharding constraints (#560)
Perf iso to https://github.com/zml/zml/pull/555
2026-05-28 20:47:02 +02:00
Hugo Mano
950c80dee8
examples/llm: split Llama and Qwen 3.5 with LFM cleanup (#555)
This PR splits Llama in `examples/llm` into separate `embed_tokens`,
`layer`, and `lm_head` kernels for both prefill and decode, and brings
LFM2 onto the same composed execution path. And done same work for Qwen
3.5 .

It removes the old LFM2 `--single` mode, reworks Llama KV-cache access
to use explicit layer indices, and reuses more decode-time state through
a persistent runner and prebuilt token index buffers.

It also tightens sharding/partitioning in LFM2 for multi-device runs,
adds clearer per-phase compile logging/program names, removes the unused
`--single` CLI flag.
2026-05-28 10:10:29 +02:00
loupicaaa
b6e2b026bc
zml/moe: Add sharding for triton MoE (#552)
This PR introduces the sharding as expert parallelism for MoE :

- Introduce a .experts logical sharding (required to express an other
semantic than TP (.model))
- Wrap the whole MoE impl into a manualComputation
- Implement a manual stableHlo all reduce since shardy can not infer
the correct communication after a manualComputation (blackbox)
- Partition qwen 3.5 moe example with TP for the rest of the model (non
MoE)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
2026-05-27 10:13:38 +02:00
Benjamin Rabier
a06265264f
zml/Sharding: Rename Placement.Shard to Placement (#549)
Keeping `devices_in_canonical_order: []Device` in the `PhysicalMesh` and
inside `Sharding.Data` I've changed `physical: PhysicalMesh` to
`physical: *const PhysicalMesh` to have the platform own it now that
PhysicalMesh has a `deinit()`.
2026-05-26 09:29:49 +02:00
Kevin Barre
73fab92393
zml/platforms: add experimental oneapi (intel) (#550)
Add experimental Intel GPU backend

Add oneAPI as a ZML platform target with Bazel flags, module extension
repositories, and PJRT runtime packaging for Linux Intel GPUs.

- package libpjrt_oneapi with oneAPI runtime libraries, Level Zero
loader,
  libsycl, and a zmlxoneapi dlopen shim for versioned shared libraries
- register the oneAPI platform loader and detect available render
devices
  before loading the packaged PJRT plugin from runfiles
- thread the oneAPI target through platform selection, sharding,
placement,
  buffers, execution, memory, IO, tensor behavior, and XLA compile flags
- document oneAPI prerequisites and example LLM usage
- update Bazel module lockfile entries for the new runtime dependencies

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Steeve Morin <steeve@9960x-b70x2.tailb57cfe.ts.net>
2026-05-25 15:52:10 +02:00
Benjamin Rabier
5d372aa4c0
zml/Sharding: Reduce Placement size by allocating shards on the stack. (#547)
`Placement` is 27KiB big, it led to stack allocation failure with
gigantic struct sizes (~400KiB) in llmd. So changing the shards to be
allocated on the heap with functions to avoid allocations when we don't
need it (`format`, etc.).
2026-05-20 16:34:47 +02:00
loupicaaa
dd0ab6c334
zml/moe : Moe backend triton zig, qwen3_5_moe model (#529)
### Overview
Context: Modern models utilize a MLP called Mixture of Experts (MoE). In
a MoE, the MLP is organized into "experts," where specific sub-tensors
are specialized to process particular linguistic patterns or elements.
During inference, the hidden states of each token are routed to the most
relevant experts, ensuring that matrix multiplications (matmul) are
performed only on the active parameters.

Problem: Due to the sparse and dynamic nature of MoE where token
assignment is determined at runtime via routing, treating an MoE layer
as a standard matmul results in significant performance degradation.
Standard kernels are not optimized for the irregular memory access
patterns in sparse routing.

Approach: MoE is implemented as a primitive in the ZML operation set.
The generic MoE Backend is designed to support various specialized
kernels tailored to specific hardware and weight precisions. The primary
implementation currently features a Triton MoE backend optimized for
NVIDIA Hopper architecture using BF16 and FP8 weights. This
implementation is utilized in the Qwen 3.5 MoE model.

### Features
**Generic MoE Backend (zml/moe/moe.zig)**
- Integrate specialized MoE backends (e.g., Triton).
- Select the optimal backend based on hardware and precision.
- Implements essential utilities and structures for registration,
parameter loading, and metadata management.
- The forwardMoe() function acts as the entry point to invoke the
appropriate backend.

**Triton MoE Backend (zml/moe/triton.zig)**
- Generates TTIR (Triton Intermediate Representation) at compile time.
- Leverages Triton's custom calls exposed via XLA.
Execution Flow:
1. Select the optimal configuration for the fused MoE kernel.
2. Determine token distribution: use naive assignment if the
distribution is sparse; otherwise, sort and align tokens for efficiency.
3. Quantization: quantize activations if using FP8 weights.
4. Fused Kernel: execute the fused MoE kernel for the gate and
up-projection layers.
5. Post-Processing: split the output and apply the activation function.
6. Quantization : re-quantize activations for the next stage if required
by the FP8 path.
7. Fused Kernel: execute the fused MoE kernel for the down-projection
layer.
8. Aggregation: sum the Top-K results for each token.

Qwen 3.5 MoE
- Fully utilizes the Triton MoE backend.
- BF16 weights implementation (also tested with FP8 but not implemented
here)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
Co-authored-by: Raphael NAHOUM <raphael@zml.ai>
2026-05-15 11:06:02 +02:00
Benjamin Rabier
ca8599aea0
zml/attention: attnd (#450)
Simple reference implementation of attnd client as a attention backend
relying on POSIX. I've tried various things with `sendmmsg`/`recvmmsg`
and `io_uring` out of curiosity but they weren't worth the extra code
here.

The integration as a attention backend is a bit clunky, we're still
allocating a KV cache and there are a few panics here and there because
the current initialization mechanism of attention backend doesn't work
for attnd, but that's something for later.
2026-05-12 17:27:37 +02:00
elogir
144f3eb0af
platforms/neuron: bump toolchain and stabilize support (#476)
- bump Neuron dependencies in `platforms/neuron/requirements.lock.txt`
- `libneuronxla` `2.2.15515.0+50c26cbd` -> `2.2.16408.0+50c26cbd`
- `neuronx-cc` `2.23.6484.0+3b612583` -> `2.24.8799.0+6f62ff7c`
- make `neuronx-cc` invocation more robust by fixing path resolution,
releasing the Python GIL during compilation, checking the compiler
process exit status, and propagating NEFF wrapping/read
failures
- resolve PJRT devices and memories by PJRT handles instead of assuming
stable numeric ids during platform initialization, including Neuron
cases where memory ids are reused across devices
- avoid unsupported Neuron synchronization/event paths by always
requesting execution events on Neuron, deinitializing them without
awaiting, and removing the explicit await from the sharding
example
- avoid unsupported Neuron runtime paths by:
- falling back to host-backed buffer initialization for
`Buffer.uninitialized`
- using passthrough DMA allocation on Neuron
- skipping `Tensor.print()` custom calls on Neuron
- add NRT-backed Neuron physical mesh topology construction, including
chip/core placement, ring/torus/point-to-point topology inference, and
Neuron local-hardware-id based shard placement
- let the sharding example pick the default partitioner from the current
target
2026-05-12 09:58:32 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Guillaume Wenzek
0979289ffe
zml/sharding: pass sharding by pointer and simplify replicated usage (#514)
This PR simplifies the common replicated case while making sharding
intent explicit and use sharding by pointer ( sharding struct is quite
big: ~600 bytes).

Using pointers for shardings also avoids unnecessary sharding copies and
makes it easier to share stable sharding instances across compilation,
loading, and runtime paths.

- `Platform` owns the canonical replicated sharding
- `Buffer` APIs now take explicit sharding specs:
    - `.replicated`
    - `.{ .sharded = sharding_ptr }`
- `createTensor` / `maybeCreateTensor `now require explicit partitioning
instead of accepting null
- compilation/loading now use pointer-based sharding lists

This makes simple cases like mnist and benchmark much cleaner: callers
no longer need to manually construct and thread a replicated sharding
just to use the default case, but they still state placement explicitly
at the call site.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-05-06 12:46:15 +02:00
Guillaume Wenzek
039a638aeb
examples/llm: cleanups (#512)
I'm doing a pass on the example code we have.

* llama/model.zig, qwen/model.zig: change layer_index from being a
tensor to being an integer.
It's clearer this way what is happening. Note that lfm doesn't have the
issue because the layer index is passed everywhere.

* tokenizePrompt: the code was doing unnecessary copies of data, I added
some helper to tokenizer.encoder to streamline the code by removing the
extra copies and allocations.

* `.stdout().writer() -> .stdout().writerStreaming()` this one is weird
but apparently using stdout by position isn't always allowed, in
particular when running the exe from within SublimeText.

* runDecode -> show case bake
2026-04-29 10:38:19 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
Hugo Mano
f80f925409
examples/llm: add profiling option (#497)
This allow use to easily trace what going on
2026-04-22 16:23:36 +02:00
Benjamin Rabier
f258280ab8
platforms/cuda+rocm: update pjrt artifact (#487)
Update with the XLA patches for improved parallel compilation.

`//examples/llm` goes from 27s to 8s for llama3.1 8B compilation
2026-04-20 18:24:01 +02:00
Benjamin Rabier
df86cdfd96
zml/tokenizer/iree: Improve IREE tokenizer with upstream fix (#482)
I've reworked the Tokenizer API to actually work with all tokenizers and
not just IREE and moved the `Writer` part into the IREE tokenizer. I'm
now avoiding intermediate copies as much as possible now.

I'm also more defensive now to avoid the same infinite loop and put more
effort into providing appropriate output buffers to IREE. It fails with
empty output buffers and may return a `ResourceExhausted` if it's too
small. Also added a bunch of tests.

IREE tokenizer itself had a bug which I fixed here
https://github.com/iree-org/iree/pull/24150. In the meantime, I'm
pointing bazel to our fork.
2026-04-20 11:15:37 +02:00
Hugo Mano
8b12d2ef1e
bazel: make repo-wide cquery work across host and linux target analysis (#465)
This PR fixes several Bazel analysis regressions so both of these
commands work reliably:

- `bazel cquery //...`
- `bazel cquery --platforms=@zml//platforms:linux_amd64 //...`

After recent changes and the rebase, repo-wide cquery was failing for
three separate reasons:

- ZLS toolchains were over-constrained by target platform: updated ZLS
toolchain registration to constrain only the exec platform.
This avoids cross-target analysis failures when host-side completion
tooling is analyzed under a foreign target platform.
- OCI packaging targets were analyzed on host platforms where their base
images do not apply: marked Linux OCI image targets as Linux x86_64
compatible only. Also switched the Debian base image pull from Docker
Hub to the GCR mirror to avoid CI fetch failures when resolving
packaging images manifests due to rate limiting
- Triton Python deps were fetched for the host Python platform instead
of the Linux wheel platform the lockfile was generated for: restored
Linux x86_64 wheel resolution for `triton_py_deps` in `MODULE.bazel`.
This prevents macOS host analysis from attempting to fetch a
`triton==3.6.0` wheel for an unsupported host platform.


Also added a step in ci to prevent regression
2026-04-17 11:48:09 +02:00
Benjamin Rabier
1e79469136
examples/llm: fix prompt tokenization (#480)
Tested with IREE and HF on all three models.
2026-04-16 11:09:47 +02:00
Benjamin Rabier
91f58097cc
zml/tokenizer: use correct API for decoding with HF (#478)
The current API only worked with IREE. Reworked it to work with both HF
and IREE
2026-04-14 17:13:26 +02:00
KB
8edf02d0c3
zml/chore: bump Zig to 0.16.0-dev.3133 and adapt codebase (#462)
Update Zig compiler to 0.16.0-dev.3133+5ec8e45f3 and adjust code
for breaking standard library changes:

- Replace ArrayListUnmanaged/ArrayList init with .empty
- Switch to unmanaged AutoArrayHashMap/StringArrayHashMap
- Use debug.GeneralPurposeAllocator (renamed upstream)
- Replace std.Io.File.ReadStreamingError with error.EndOfStream
- Refactor meta.visit to use structured visitor pattern
- Fix alignment allocation in mem.bufferizeInner

Also update third-party dependencies: flashattn 0.0.6-rc8, arocc,
uucode, libvaxis, and zigimg patches.

Co-authored-by: Kevin Barre <kevin@zml.ai>
2026-04-13 11:56:32 +02:00
Corentin Godeau
ac9bc32d89
zml/attention: add Triton Unified Attention backend (#417)
This PR introduces a Triton paged attention backend, unlocking ROCm
support.

It relies on a Python sandbox to generate the TTIR.

The major change is that the KVCache now needs to be per-layer for paged
attention. The `layer_index` is now superfluous and will probably be
removed in the future.
2026-04-13 11:39:08 +02:00
Steeve Morin
adee932efa
zml/io/vfs: add GCS VFS (#453)
This pull request adds support for Google Cloud Storage (GCS) to the
virtual file system (VFS) module, alongside some build configuration
updates. The main changes introduce the new `GCS` backend, make it
available in the VFS API, and register it in the example application.

Authentication is handled like other GCS libraries via the
`GOOGLE_APPLICATION_CREDENTIALS` env or application default account
files.

**IMPORTANT NOTE**: Only Authorized User and Metadata Server auth are
implemented. Service Account auth isn't implemented yet.
2026-04-07 19:05:43 +02:00
Benjamin Rabier
448084908a
zml/tokenizer: add IREE tokenizer (#416)
Adding the [IREE
tokenizer](https://github.com/iree-org/iree/blob/main/runtime/src/iree/tokenizer/README.md).Being
made from the ground up with streaming (de-)tokenization in mind, I've
extended the APIs to expose a `std.Io.Writer` for the tokenization.

I've quickly benchmarked IREE's tokenizer vs HuggingFace's one with a
request I retrieved from a coding agent (opencode) in the middle of a
session. The request body was 360KB encoded in 95K tokens. I also did
another test with a similarly sized request with the same results as
shown hereafter.

I've tested two cases:

- oneshot: the template is rendered all at once and only then we encode
the whole string.
- stream: the template rendering and encoding are pipelined to avoid the
intermediate allocation.

There is a slight difference in the generated tokens between HuggingFace
and IREE. Some tokens like `introdu` were encoded as `i`, `n`, `t`, etc.
Decoding them returns the same text. It's very close in terms of total
token count:

- hf: 95779
- iree/stream: 95705
- iree/oneshot: 95587

| Model | IREE avg (oneshot / stream) ms | HuggingFace avg (ms) | Uplift
vs HF (oneshot / stream) |
|---|---:|---:|---|
| liquidai/LFM2.5-1.2B-Instruct | 8.5 / 7.3 | 40.5 | 4.8x / 5.5x |
| Qwen/Qwen3.5-9B | 6.0 / 8.2 | 59.4 | 9.9x / 7.2x |
| mistralai/Ministral-3-3B-Reasoning-2512 | 7.4 / 6.9 | 59.2 | 8x / 8.6x
|


It's unclear why the Qwen's chat template is somehow slower in the
stream case, maybe because there are more writes. Adding non-empty
buffer in the interface helps a bit, but tends to slow down mistral
case. So well

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2026-04-07 16:42:55 +02:00
Tristan Gautié
333ec93c47
Fix slice lifetime issue (#451)
We fix a small slice lifetime issue in the prompt template of the Qwen3.5 model in LLM CLI
2026-04-03 13:11:45 +02:00
Benjamin Rabier
c3c0a69689
ci: update actions (#447)
- Update setup-bazel & checkout to latest
- Add a bunch of `tags = ["manual"]` for docker image related targets
2026-04-01 10:48:10 +02:00
Hugo Mano
65e9b4487e
examples/llm: replicate Qwen 3.5 KV heads when TP exceeds KV head count (#444)
Handle Qwen3.5 full-attention K/V heads like vLLM when
num_key_value_heads < tp: replicate K/V heads across the model mesh
instead of sharding the KV head axis.

**Motivation**

Qwen3.5 9B has 4 KV heads. On 8-way TP, sharding the KV head axis over
`.model` is invalid because the dimension is not divisible by the
tensor-parallel degree.

This is a placement issue, not a model-shape issue. The model still has
4 logical KV heads, so the right fix is to replicate those heads across
ranks rather than materialize fake extra heads.

This also matches cost modeling better: replication represents
duplicated placement of the same logical data, while repeating would
incorrectly make the tensor look larger than the model actually defines.

**Changes**

- Add a policy helper to detect when KV heads should be replicated
instead of model-sharded.
- Apply that policy to projected K/V tensors in Qwen3.5 self-attention.
- Apply the same policy to KV cache shape partitioning.
- Thread model partition count through the Qwen3.5 inference compilation
path so cache placement follows the same rule.

**Why replicate instead of repeat**

Replicating:

- preserves the logical model shape
- matches grouped-query attention semantics
- keeps cost modeling honest by representing placement, not model
expansion

Repeating:

- would change the graph from 4 KV heads to 8 KV heads
- would increase KV cache size and bandwidth for no new information
- would distort cost modeling by turning a placement choice into fake
tensor growth

**Result**

Qwen3.5 now supports TP setups such as 8 GPUs where KV head count is
smaller than the tensor-parallel degree.
2026-03-31 18:07:02 +02:00
Hugo Mano
6e7cb90728
zml/platforms/rocm: update to ROCm 7.2.1 with full hermeticity (#439)
Fix ROCm runtime hermeticity in the packaged LLM sandbox by patching transitive ROCm libraries to use zmlxrocm_dlopen and by adding missing libdw/liblzma/libbz2 runtime deps for distroless execution. Also update the ROCm packaging to 7.2.1 and refresh the pinned libpjrt_rocm
2026-03-26 17:52:48 +01:00
Benjamin Rabier
9fa7768808
examples/llm: better single prompt UX + --topk (#438)
- Same UX as the chat
- Add `--topk` argument
2026-03-26 15:58:03 +01:00
Benjamin Rabier
268ef6e058
examples/llm: fix --help & relative/home paths (#437) 2026-03-25 15:14:22 +01:00
Hugo Mano
22db5548ec
zml/examples/llm: add sharding to Qwen 3.5 (#435)
Using tensor parallelism strategy. From 86 toks/s to 119 tok/s for Qwen
3.5 on 2x5090
2026-03-25 13:55:06 +01:00
Benjamin Rabier
0d0d5020cf
example/llm: Further simplifications (#433) 2026-03-25 10:16:26 +01:00
Benjamin Rabier
8de2c42fe6
examples/llm: Simplifications (#430) 2026-03-24 18:16:25 +01:00
Corentin Godeau
2eb8423e8a
Update doc for V2 (#428) 2026-03-24 17:00:10 +01:00
Benjamin Rabier
81299c67c8
examples/llm: Common model example (#425) 2026-03-24 16:55:29 +01:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Steeve Morin
58b2a4b552 examples/llama: fix bazel_lib and tar imports 2026-03-21 16:21:57 +00:00
Benjamin Rabier
a011fb0e52
Green CI (#419)
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
2026-03-20 10:31:57 +01:00
Hugo Mano
619fea8b18
zml: introduce profiler (#413)
Move profiling behind a zml-owned primitive instead of exposing PJRT
details in user code. Add the XLA/TSL Bazel deps needed to convert
collected XSpace protobufs into Perfetto traces with XLA/TLS APIs
instead of having a custom fallible converter.
2026-03-17 18:14:24 +01:00
Corentin Godeau
70e4c9953d
Bump zig to 0.16.0-dev.2722+f16eb18ce (#409)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-10 15:58:48 +01:00
Hugo Mano
abde0a04dd
zml: sharding (#407)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-10 11:17:57 +01:00
Benjamin Rabier
569bd1dfd7
examples/transfer-benchmark: Use VFS & full model for transfer-benchmark (#390)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-02-24 09:52:40 +01:00
Hugo Mano
a9f2687237
zml/io/vfs: fixes for Zig 0.16.0-dev.2611+f996d2866 + new S3 VFS (#388) 2026-02-19 10:48:15 +01:00
Tristan Gautié
a175578eba
examples/llama: improve defer pattern (#386)
We improve the defer pattern in the Llama example to:
- avoid double deinit of `prefill_exe` and `decode_exe` by cleaning at
future-definition time,
- using defer instead of errdefer for future-allocated resources cleanup
in the main function scope,
- unload buffers from the arena allocator.
2026-02-19 09:28:36 +01:00
Benjamin Rabier
48df9c85e9
examples/benchmark: Add transfer-benchmark & fix benchmark (#387)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-02-19 09:27:53 +01:00
Benjamin Rabier
82c9c25836
examples/lfm: Add LiquidAI LFM2.5 (#376) 2026-02-19 09:27:10 +01:00
Corentin Godeau
2bd356e0f9 zml/nn: added common rope_theta parameter to rope variants 2026-02-16 20:52:23 +00:00