0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

1,078 commits

Author SHA1 Message Date
elogir
8c03af9987
stdx/flags: Update typeInfo tag to Zig 0.16 .@"union". (#565) 2026-06-01 17:51:42 +02:00
Guillaume Wenzek
807a9116d4
zml: make Buffer.from faster (#556)
`Buffer.from` is in the hot path everytime we want to send smth to the
GPU.
Sometimes we can overlap it with GPU compute but not always, and
sometimes it's quite complicated to do so.

I've rewrote Sharding.Placement to be device agnostic instead of device
specific.
This allow to reuse most of the logic for multiple devices.

According to LLVM MCA (with script included here), we got from:

```
Iterations:        100
Instructions:      324200
Total Cycles:      175191
Total uOps:        369700
```

to 

```
    Iterations:        100
    Instructions:      162900
    Total Cycles:      85585
    Total uOps:        182600
```

So about twice as fast for this part of the code. I would need some help
to confirm the numbers on TPU.


(Once the PR is approved, I'll remove the mca script as well as the
build.zig)

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-06-01 10:51:32 +02:00
Kevin Barre
02c1a90c08
bin/smi: add oneapi smi monitoring (#557)
Add a oneAPI backend for `zml-smi` that detects Intel DRM render devices
and reports GPU telemetry from sysfs, hwmon, and fdinfo.

The new backend collects utilization, memory usage, power, temperature,
clocks, PCIe link data, and per-process usage for Intel GPUs. It is
wired into device detection, Bazel builds, Prometheus metrics, and the
TUI, including Intel GPU imagery and GPU detail/process table support.

Also clean up related TUI handling for GPU clock/PCIe fields and process
table lifetime management.
2026-05-29 18:14:38 +02:00
loupicaaa
2530747949
zml/attention: Fix the triton attention kernel by always masking the padding tokens (#563)
Fix the triton attention kernel by avoiding the case where `TILE_SIZE ==
BLOCK_SIZE`. Now we enforce masking this way : `const tile_mask: Value =
seq_offset.lt(max_seq_prefix_len);`.

Co-authored-by: loupicaaa <louis@zml.ai>
2026-05-29 17:42:35 +02:00
Hugo Mano
cd941143dc
examples/llm: remove useless sharding constraints (#560)
Perf iso to https://github.com/zml/zml/pull/555
2026-05-28 20:47:02 +02:00
Hugo Mano
950c80dee8
examples/llm: split Llama and Qwen 3.5 with LFM cleanup (#555)
This PR splits Llama in `examples/llm` into separate `embed_tokens`,
`layer`, and `lm_head` kernels for both prefill and decode, and brings
LFM2 onto the same composed execution path. And done same work for Qwen
3.5 .

It removes the old LFM2 `--single` mode, reworks Llama KV-cache access
to use explicit layer indices, and reuses more decode-time state through
a persistent runner and prebuilt token index buffers.

It also tightens sharding/partitioning in LFM2 for multi-device runs,
adds clearer per-phase compile logging/program names, removes the unused
`--single` CLI flag.
2026-05-28 10:10:29 +02:00
Corentin Godeau
3db711aa6f
zml/ops: fix typo in type name (#553)
Names of types should be `PascalCase`
2026-05-27 17:52:33 +02:00
Corentin Godeau
6d596964eb
workspace: bump llvm to 0.8.3 (#554) 2026-05-27 15:00:47 +02:00
loupicaaa
b6e2b026bc
zml/moe: Add sharding for triton MoE (#552)
This PR introduces the sharding as expert parallelism for MoE :

- Introduce a .experts logical sharding (required to express an other
semantic than TP (.model))
- Wrap the whole MoE impl into a manualComputation
- Implement a manual stableHlo all reduce since shardy can not infer
the correct communication after a manualComputation (blackbox)
- Partition qwen 3.5 moe example with TP for the rest of the model (non
MoE)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
2026-05-27 10:13:38 +02:00
Benjamin Rabier
a06265264f
zml/Sharding: Rename Placement.Shard to Placement (#549)
Keeping `devices_in_canonical_order: []Device` in the `PhysicalMesh` and
inside `Sharding.Data` I've changed `physical: PhysicalMesh` to
`physical: *const PhysicalMesh` to have the platform own it now that
PhysicalMesh has a `deinit()`.
2026-05-26 09:29:49 +02:00
Benjamin Rabier
289c29f365
workspace: update AGENTS.md (#548)
More concise AGENTS.md and add a command to help the agent find the
sources of the Zig library it regularly struggles with it
2026-05-25 17:14:28 +02:00
Kevin Barre
73fab92393
zml/platforms: add experimental oneapi (intel) (#550)
Add experimental Intel GPU backend

Add oneAPI as a ZML platform target with Bazel flags, module extension
repositories, and PJRT runtime packaging for Linux Intel GPUs.

- package libpjrt_oneapi with oneAPI runtime libraries, Level Zero
loader,
  libsycl, and a zmlxoneapi dlopen shim for versioned shared libraries
- register the oneAPI platform loader and detect available render
devices
  before loading the packaged PJRT plugin from runfiles
- thread the oneAPI target through platform selection, sharding,
placement,
  buffers, execution, memory, IO, tensor behavior, and XLA compile flags
- document oneAPI prerequisites and example LLM usage
- update Bazel module lockfile entries for the new runtime dependencies

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Steeve Morin <steeve@9960x-b70x2.tailb57cfe.ts.net>
2026-05-25 15:52:10 +02:00
elogir
cebc4cebf8
kernels: extend Mosaic TPU + Triton kernel DSL (#545)
This PR changes the following:

- Name-based `output_operand_aliases` for MosaicTPU and Triton DSLs
- New `kernels/mosaic_tpu/pipeline.zig`, a Zig port of `pltpu.emit_pipeline`
- Some fixes to MosaicTPU
2026-05-25 09:41:08 +02:00
Corentin Godeau
36300dee2e
zml/attention: small changes required for llmd (#551)
* Made the `MAX_NUM_SPLITS` public
* Inject max num pages from args in `Options.fromBackend()` instead of
computing it
2026-05-22 10:58:27 +02:00
Benjamin Rabier
5d372aa4c0
zml/Sharding: Reduce Placement size by allocating shards on the stack. (#547)
`Placement` is 27KiB big, it led to stack allocation failure with
gigantic struct sizes (~400KiB) in llmd. So changing the shards to be
allocated on the heap with functions to avoid allocations when we don't
need it (`format`, etc.).
2026-05-20 16:34:47 +02:00
Corentin Kerisit
77711b949d
workspace: support zig-only local execution (#546)
This PR bumps rules_zig back to upstream/main where we now have all the
necessary
features to enable zig local actions while the reste on remote.

It also adds the necessary configurable to actually enable this.

Additionally, since rules_zig now ships ZLS support, we remove zls
entirely from zml.

### How the Bazel resolution works

Bazel resolves execution platforms and toolchains during analysis for
each
target that requires toolchains. If a rule defines exe, resolution
happens separately for the default target execution group and for each
`exec_group`;
actions assigned to an exec group use that group’s selected execution
platform
and toolchains.

Execution platforms come from an ordered set. In this repository’s
remote
configuration, the relevant order is:

1. `//:rbe_platform`, supplied by
`--extra_execution_platforms=@zml//:rbe_platform`
2. `//bazel/zig_toolchain_config:local_execution_platform`, registered
by this PR
3. the host platform, included automatically by Bazel

Earlier platforms are selected when they can satisfy the requested
toolchains.

For each target or exec group, Bazel takes the required toolchain types,
the
target platform, the ordered execution platforms, and the ordered
registered
toolchains. It filters execution platforms by execution constraints,
then tries
to find a matching toolchain for every required toolchain type. A
toolchain
matches when:

- its `exec_compatible_with` constraints are present on the candidate
execution
platform
- its `target_compatible_with` constraints are present on the target
platform
- its `target_settings` match the current configuration

Any execution platform missing a required toolchain is rejected. The
first
remaining execution platform wins, and the first matching toolchain for
each type
is selected.

### How this makes Zig local-only

This PR defines a custom execution constraint:

- `//bazel/zig_toolchain_config:local_zig_execution`

It then defines a host-derived execution platform:

- `//bazel/zig_toolchain_config:local_execution_platform`

That platform inherits the host OS/CPU constraints from
`@platforms//host`, adds
`local_zig_execution`, and sets:

```starlark
exec_properties = {
    "no-remote-exec": "1",
}
```

The zig.toolchain(...) module extension call now passes:

extra_exec_compatible_with =
["//bazel/zig_toolchain_config:local_zig_execution"]

rules_zig appends that extra execution constraint to every generated Zig
SDK
toolchain. As a result, when rules_zig resolves
@rules_zig//zig:toolchain_type
for its zig execution group:

- the remote execution platform is considered first, but it does not
have
local_zig_execution, so it cannot match the Zig SDK toolchain
- the custom local execution platform has both the host OS/CPU
constraints and
local_zig_execution, so it matches
- the selected platform carries no-remote-exec, so Zig actions execute
locally
even in remote builds

Other toolchains are not constrained by local_zig_execution, so they can
continue
to resolve to the remote execution platform first. This keeps the build
remote by
default while carving out Zig actions as local-only.

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-05-20 16:13:59 +02:00
Reese Chong
3f2666dba6
zml/testing: resolve type of forward in testLayer() (#540)
Change made in response to the following error:

```
zml/testing.zig:299:43: error: root source file struct 'meta' has no member named 'FnArgs'
    const ArgsT = stdx.meta.Tail(stdx.meta.FnArgs(forward));
                                 ~~~~~~~~~^~~~~~~
stdx/meta.zig:1:1: note: struct declared here
```
2026-05-19 09:19:56 +02:00
Reese Chong
986532bb33
tensor/zml: fix SwiGLU (#539)
Update SwiGLU implementation to match the SwiGLU paper.

- Formula: `(xW + b) * sigmoid(β(xW + b)), outer product with (xV + c)`
- Add test for SwiGLU
2026-05-18 16:52:05 +02:00
Corentin Godeau
61ed004556
zml/attention: big rework of paged attention (#536)
List of changes:
* Removed `Context`
* Simplified metadata to not contain scratch buffers. They are created
uninitialized on the fly instead (thanks to #535)
* Added helper for memory movement (host_pinned -> device) (thanks to
#534)
* FA2/FA3 don't read the `max_seqlen_q` from host anymore. This was
error prone. Instead rely on a fixed value for now. (It's error prone
because it's baked in the command buffer and can result in wrong value
if the value changes but the command buffer is not recorded again).
2026-05-18 15:40:04 +02:00
Corentin Godeau
57424e6c41
zml/buffer: added fromBytesOpts (#544)
Since there is `fromSliceOpts`, there should be `fromBytesOpts`
2026-05-18 15:31:00 +02:00
Corentin Kerisit
55474ea31f
zls: resolve translate-c toolchain in zls rules (#538)
This one slipped from #537 

It worked because if there is no translate-c toolchain, zig uses zig
translate-c.
But we want to respect user choice here too.
2026-05-16 09:58:34 +02:00
Corentin Kerisit
886d9000be
workspace: use latest version of upstreamable rules_zig (#537)
The upstreamable branch now contains only 2 minor commits. Everything
else inccluding test-obj, translatec and bootstrap have been upstreamed.
2026-05-15 21:01:50 +02:00
loupicaaa
dd0ab6c334
zml/moe : Moe backend triton zig, qwen3_5_moe model (#529)
### Overview
Context: Modern models utilize a MLP called Mixture of Experts (MoE). In
a MoE, the MLP is organized into "experts," where specific sub-tensors
are specialized to process particular linguistic patterns or elements.
During inference, the hidden states of each token are routed to the most
relevant experts, ensuring that matrix multiplications (matmul) are
performed only on the active parameters.

Problem: Due to the sparse and dynamic nature of MoE where token
assignment is determined at runtime via routing, treating an MoE layer
as a standard matmul results in significant performance degradation.
Standard kernels are not optimized for the irregular memory access
patterns in sparse routing.

Approach: MoE is implemented as a primitive in the ZML operation set.
The generic MoE Backend is designed to support various specialized
kernels tailored to specific hardware and weight precisions. The primary
implementation currently features a Triton MoE backend optimized for
NVIDIA Hopper architecture using BF16 and FP8 weights. This
implementation is utilized in the Qwen 3.5 MoE model.

### Features
**Generic MoE Backend (zml/moe/moe.zig)**
- Integrate specialized MoE backends (e.g., Triton).
- Select the optimal backend based on hardware and precision.
- Implements essential utilities and structures for registration,
parameter loading, and metadata management.
- The forwardMoe() function acts as the entry point to invoke the
appropriate backend.

**Triton MoE Backend (zml/moe/triton.zig)**
- Generates TTIR (Triton Intermediate Representation) at compile time.
- Leverages Triton's custom calls exposed via XLA.
Execution Flow:
1. Select the optimal configuration for the fused MoE kernel.
2. Determine token distribution: use naive assignment if the
distribution is sparse; otherwise, sort and align tokens for efficiency.
3. Quantization: quantize activations if using FP8 weights.
4. Fused Kernel: execute the fused MoE kernel for the gate and
up-projection layers.
5. Post-Processing: split the output and apply the activation function.
6. Quantization : re-quantize activations for the next stage if required
by the FP8 path.
7. Fused Kernel: execute the fused MoE kernel for the down-projection
layer.
8. Aggregation: sum the Top-K results for each token.

Qwen 3.5 MoE
- Fully utilizes the Triton MoE backend.
- BF16 weights implementation (also tested with FP8 but not implemented
here)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
Co-authored-by: Raphael NAHOUM <raphael@zml.ai>
2026-05-15 11:06:02 +02:00
Guillaume Wenzek
37ef0a87c0
mlir: fluent api (#533) 2026-05-15 10:53:54 +02:00
Steeve Morin
c8cc476042 third_party/zls: adapt to rules_zig changes
notably when using with_cfg targets
2026-05-13 11:35:34 +00:00
Corentin Godeau
3b718a5a89
zml/tensor: add Tensor.uninitialized() (#535)
This PR exposes the XLA-specific custom calls that allow to create
uninitialized tensors (thus not paying the cost of a kernel to
essentially do a memset).

The `stablehlo` part of it was already done, so I added a wrapper in
`tensor.zig`
2026-05-13 10:40:20 +02:00
Corentin Godeau
8418c97624
zml/tensor: add Tensor.onMemory() (#534)
It is now required with newer plugin to annotate the executable
arguments with their location.

Like `Tensor.reuseBuffer()`, you can now use `Tensor.onMemory()` to
specify on which memory a tensor is located.
2026-05-13 10:39:07 +02:00
Benjamin Rabier
ca8599aea0
zml/attention: attnd (#450)
Simple reference implementation of attnd client as a attention backend
relying on POSIX. I've tried various things with `sendmmsg`/`recvmmsg`
and `io_uring` out of curiosity but they weren't worth the extra code
here.

The integration as a attention backend is a bit clunky, we're still
allocating a KV cache and there are a few panics here and there because
the current initialization mechanism of attention backend doesn't work
for attnd, but that's something for later.
2026-05-12 17:27:37 +02:00
elogir
c917acb572
kernels/triton: add LLVM dialect, pipelining hints and fixes (#525)
This PR introduces the following:

- Wire up the LLVM dialect so the Triton builder can emit
`llvm.intr.assume` so we can match
[`triton.language.assume`](https://triton-lang.org/main/python-api/generated/triton.language.assume.html#triton.language.assume)
- Thread `tt.num_stages` through `scf.for` for pipelining hints
- Add an inline-return mode to `ReturnIfScope`. Not necessary but better
for IR matching with Python reference
- Build DSL kernels IR in a new MLIR context

Also fixes i1 extension to use `extui` instead of `extsi`.
2026-05-12 14:54:42 +02:00
Hugo Mano
eae31a51b8
zml/sharding: fix segfault in tensorShardingAttr for gspmd (#532) 2026-05-12 13:34:08 +02:00
Corentin Kerisit
e67f60fe25
workspace: remove custom registry (#531)
Unused and noisy
2026-05-12 10:32:51 +02:00
Corentin Kerisit
31887689a9
workspace: add several bazel 9 required changed (#530)
Just a few stuff to get us closer to Bazel 9
XLA is the main blocker and i'll tackle this as soon as other
dependencies merge my PRs.
2026-05-12 10:27:18 +02:00
elogir
144f3eb0af
platforms/neuron: bump toolchain and stabilize support (#476)
- bump Neuron dependencies in `platforms/neuron/requirements.lock.txt`
- `libneuronxla` `2.2.15515.0+50c26cbd` -> `2.2.16408.0+50c26cbd`
- `neuronx-cc` `2.23.6484.0+3b612583` -> `2.24.8799.0+6f62ff7c`
- make `neuronx-cc` invocation more robust by fixing path resolution,
releasing the Python GIL during compilation, checking the compiler
process exit status, and propagating NEFF wrapping/read
failures
- resolve PJRT devices and memories by PJRT handles instead of assuming
stable numeric ids during platform initialization, including Neuron
cases where memory ids are reused across devices
- avoid unsupported Neuron synchronization/event paths by always
requesting execution events on Neuron, deinitializing them without
awaiting, and removing the explicit await from the sharding
example
- avoid unsupported Neuron runtime paths by:
- falling back to host-backed buffer initialization for
`Buffer.uninitialized`
- using passthrough DMA allocation on Neuron
- skipping `Tensor.print()` custom calls on Neuron
- add NRT-backed Neuron physical mesh topology construction, including
chip/core placement, ring/torus/point-to-point topology inference, and
Neuron local-hardware-id based shard placement
- let the sharding example pick the default partitioner from the current
target
2026-05-12 09:58:32 +02:00
Benjamin Rabier
532c22955d
zml/ops: Merge the two customCall and add compute_on_host (#511)
Add `compute_on_host` to force host offloading and merging the
`customCallInternal` and `typeCustomCall` function into one to avoid
duplication. I've slightly simplified the signature of some of the
functions I wrote initially to avoid passing the types explicitly.
2026-05-11 18:15:42 +02:00
Hugo Mano
10536817a4
zml/sharding: fix leak and use mlir.Attribute instead of concrete types (#527)
Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2026-05-11 11:16:56 +02:00
Reese Chong
86aeed775e
docs: fix excerpts don't match final code (#524)
Two fixes to the `write_first_model.md` tutorial:

- `allocator` on line 221 is deinitialized, however, `io` is not. `io`
should be deinitialized as well.

- `sharding` on line 216, is an undeclared identifier. It should be
replaced with `.replicated`, which is used in the full code example.
2026-05-11 10:37:52 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Guillaume Wenzek
5d94f9b83c
mlir: add shardy dialect (#526)
Add bindings to shardy dialect,
remove the string generation code from zml.Sharding, and replace it with
mlir calls.
2026-05-11 09:37:05 +02:00
Guillaume Wenzek
0979289ffe
zml/sharding: pass sharding by pointer and simplify replicated usage (#514)
This PR simplifies the common replicated case while making sharding
intent explicit and use sharding by pointer ( sharding struct is quite
big: ~600 bytes).

Using pointers for shardings also avoids unnecessary sharding copies and
makes it easier to share stable sharding instances across compilation,
loading, and runtime paths.

- `Platform` owns the canonical replicated sharding
- `Buffer` APIs now take explicit sharding specs:
    - `.replicated`
    - `.{ .sharded = sharding_ptr }`
- `createTensor` / `maybeCreateTensor `now require explicit partitioning
instead of accepting null
- compilation/loading now use pointer-based sharding lists

This makes simple cases like mnist and benchmark much cleaner: callers
no longer need to manually construct and thread a replicated sharding
just to use the default case, but they still state placement explicitly
at the call site.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-05-06 12:46:15 +02:00
Guillaume Wenzek
c38179513d
bump xprof-nightly (#523)
the previous nightly we depend on is not on pypi anymore.
I wanted to switch to stable release but 2.23 isn't public yet:
https://pypi.org/project/xprof/#history
2026-05-06 12:26:09 +02:00
Hugo Mano
92b8e34c7b
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522)
This change fixes profile timestamp handling so saved xplane.pb data is
post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the
Zig/C++ FFI used by the profile export bridge.

Before this, we were writing raw profiler output from the PJRT extension
directly to `profiling.xplane.pb`, which bypassed XLA’s single-host
post-processing step done by JAX that we can find here:
efac222ebf/tsl/profiler/lib/profiler_session.cc (L100)

That made ZML behave differently from JAX, including wrong timeline
origins in downstream viewers like XProf.
To fix that, we now run
`tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the
protobuf, and we treat that post-processed xplane.pb as the source for
subsequent Perfetto conversion.
We also register start/stop timestamps in `zml/profiler.zig` so they
match XLA’s expectations for session timestamps with `real` clock.

While touching this path, I tightened the FFI boundary as well. 
Error text returned from the xspace export bridge is now allocated with
the provided Zig allocator instead of malloc, C receives a pointer to
the `zig_allocator wrapper` to avoid lifetime issues from passing
allocator state by value, and error propagation in
`tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly
instead of building std::string error objects just to copy them back
out. This removes mixed allocation ownership between C++ and Zig and
fixes the allocator lifetime bug that could crash across the FFI
boundary.
2026-05-05 14:52:15 +02:00
Steeve Morin
57d191d092 zml/attention: make triton the default paged attention provider on cuda 2026-05-04 13:04:11 +00:00
Steeve Morin
b230be2f3f platforms: bump PJRT=manual-2026-04-30T17-13-00Z
notably, this bumps fixes TTIR gen problems now that we switched away
from python based TTIR
2026-04-30 17:34:49 +00:00
Steeve Morin
5e4d1b1d05 zml: bump allocation to 90% of the GPU memory
up from 85%

since we now usually compile _before_ loading models (at least on llmd)
2026-04-30 17:34:06 +00:00
Steeve Morin
e2ab3022de platforms: for aarch64 cpu flag when selecting the linux_arm64 platform
this fixes highwayhash outdated build files that select on this

sadly we can't update highwayhash just yet
2026-04-30 17:32:56 +00:00
elogir
8e4c333007
zml: replace Python Triton/Mosaic IR generation with native Zig DSLs (#517)
Drops the Python build-time dependency for Triton and Mosaic TPU
kernels. ZML now emits TTIR and Mosaic TPU IR directly from Zig, against
the same MLIR context as the rest of the compiler. So no more JAX,
Triton at build time and a lighter sandbox.

Split into two commits:

- **`c72f44884` — Add Triton and Mosaic Zig DSL.** Additive: IR builders
(`kernels/triton/`, `kernels/mosaic_tpu/`), upstream dialect bindings
(`ttir`, `mosaic_tpu`, plus `arith`/`scf`/`math`/`memref`/`vector`/
`affine`/`cf`), and the high-level
`zml.kernel.{triton,mosaic_tpu}.Kernel(Config, spec)` declarative form.
Comes with author guides in each `README.md`.

- **`cb61e529f` — Remove Python and replace with Zig kernels.** Migrates
the two production kernels (`unified_attention` for Triton,
`ragged_paged` for Mosaic TPU) and deletes everything Python related in
the sandbox and what was generating the IR.

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-04-30 16:13:48 +02:00
Olivier Boulant
1c3b505b9e
swig(tokenizer): remove unsupported SWIG optimal hint (#501)
This PR removes `optimal="1"` from the `[const std::string&]` out
typemap in `zml/tokenizer/sentencepiece/sentencepiece.i`.

SWIG cannot apply the requested optimal code path for the `%extend
std::string { data() }` wrapper and emitted `Warning 474: ` during
SwigC. Removing the unsupported hint preserves the generated behavior
while eliminating the warning.

**Reproduce**
```
$ bazel build //examples/llm
[...]
zml/tokenizer/sentencepiece/sentencepiece.i:41: Warning 474: Method std::string::data() const usage of the optimal attribute ignored
zml/tokenizer/sentencepiece/sentencepiece.i:25: Warning 474: in the out typemap as the following cannot be used to generate optimal code: cppresult = (std::string *) &std_string_data__SWIG((std::string const *)arg1);
[...]
```

**Validation**

* Rebuilt `//examples/llm:llm` and a clean
* Confirmed `Warning 474: ` no longer appears
* Confirmed build still succeeds
* Warning 524 remains and is not addressed in this PR
2026-04-29 18:26:58 +02:00
Olivier Boulant
df4bb4c3dd
bazel: suppress SWIG c-target warning due to intended behavior (#504)
This suppresses SWIG Warning 524 in `bazel/swig.bzl`. 

The custom SWIG rule intentionally parses C++ interfaces while
generating a C ABI wrapper for Zig consumption (in
[sentencepiece.zig](zml/tokenizer/sentencepiece/sentencepiece.zig)), so
this warning is expected noise from an intentional -c backend selection
rather than a build problem.

**Reproduce**
```
$ bazel build //examples/llm
[...]
SWIG:1: Warning 524: Experimental target language. Target language C specified by -c is an experimental language. See the 'Target Languages' section in the Introduction chapter of the SWIG documentation.
[...]
```

**Validation**
* Built `//examples/llm:llm` after a clean
* Confirmed Warning 524 no longer appears
* Build still succeeds without changing generated binding behavior
2026-04-29 18:24:43 +02:00
Steeve Morin
b0b7e02692
zml: remove hftokenizers (#518)
This is the second take of #463, now that #482 is merged.
2026-04-29 18:15:26 +02:00
Guillaume Wenzek
039a638aeb
examples/llm: cleanups (#512)
I'm doing a pass on the example code we have.

* llama/model.zig, qwen/model.zig: change layer_index from being a
tensor to being an integer.
It's clearer this way what is happening. Note that lfm doesn't have the
issue because the layer index is passed everywhere.

* tokenizePrompt: the code was doing unnecessary copies of data, I added
some helper to tokenizer.encoder to streamline the code by removing the
extra copies and allocations.

* `.stdout().writer() -> .stdout().writerStreaming()` this one is weird
but apparently using stdout by position isn't always allowed, in
particular when running the exe from within SublimeText.

* runDecode -> show case bake
2026-04-29 10:38:19 +02:00