This PR uses the new release of rules_zig where we upstreamed all the
work from the zml rules_zig fork.
Not much changes from the POV of the public API except for `copts`
becoming `zigopts`.
Also, the global `c` module is only available to Zig modules that depend
on a non empty `cc_library`.
This is behavior that can probably be modified in `rules_zig` later on.
Finally, ZLS support had to be completely rethought to comply with the
change in shape of ZigModuleInfo providers and subsequent rules.
This PR is blocked until `rules_zig` exposes `ZigModuleInfo` for
`zig_binaries`.
Revamp the `gather` api to match `scatter`
(prerequesite of #329 )
## Gather new api
Now the syntax is to give a named list of indices. Each name must
correspond to one axis in the gathered tensor.
This match `scatter`, and avoid the user to have to concatenate indices
in the same order than the input axes,
which is error prone cause any screw up will compile but index at the
wrong offsets.
For 1 axis it doesn't change much:
```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc}, .{ idx }, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx }, .{});
```
But for two it helps a lot:
```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc, .d}, .{ .concatenate(&.{idx_voc, idx_d}, .last)}, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx_voc, .d = idx_d }, .{});
```
Also adds a `gather_` that offers the same apis but with axis position
instead of names.
This one is more suitable inside library functions when dealing with
unknown names, making it complicated to create the "indices" struct.
I think we could generalize this pattern of offering "non-tagged" api
with an underscore suffix, simplifying the job of LSP.
## topk new API
I realized that most of the code calling topk was renaming the output
axis.
The idea is that logprob.topk(.voc) doesn't return you a tensor with voc
entry but only top_voc entries.
For the values it doesn't make much a difference but for the indices
it's embarrassing and it break usages of gather/scatter.
Now the user of `topK` can either use the unnamed api and pass a number,
of if using the name api must give a new name for the indexed axis.
## Yarn
Support Yarn rope embeddings. It's equivalent to the "llama3" embedding
except they do the interpolation in time space instead of freq space.
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.
As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
Put the name of the operator in the source location.
This seemed redundant at first because it's not needed when looking at
the stablehlo,
but it's nice to have it to understand the optimization passes.
I'm also including a few QoL:
* sdpa don't need a bias argument, attn_mask already play this role.
* gatherSlices should accept a shape literal `.{ .a = 5, .b = 128 }`
* Shape.outer to combine two shapes into the superset of their axes
* Tensor.all (since we already have Tensor.any)
* argMax doesn't need to specify the dtype, it can be inferred from the
size of the input tensors. If the user care about the dtype they can
manually convert.
This PR is stacked on top of #147
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125
More settings are not exposed ATM, will follow up with #1323d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)
AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.
**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.
API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)
Mostly, also, it's pretty fast.
This one is a chonky.
This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.
Also, clean the logs in the executables.
`asyncc` is arguably nicer to write than `async_`, so use that.
Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).