mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Revamp the `gather` api to match `scatter` (prerequesite of #329 ) ## Gather new api Now the syntax is to give a named list of indices. Each name must correspond to one axis in the gathered tensor. This match `scatter`, and avoid the user to have to concatenate indices in the same order than the input axes, which is error prone cause any screw up will compile but index at the wrong offsets. For 1 axis it doesn't change much: ``` // before self.weight.withTags(.{ .voc, .d }).gather(.{.voc}, .{ idx }, .{}); // after self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx }, .{}); ``` But for two it helps a lot: ``` // before self.weight.withTags(.{ .voc, .d }).gather(.{.voc, .d}, .{ .concatenate(&.{idx_voc, idx_d}, .last)}, .{}); // after self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx_voc, .d = idx_d }, .{}); ``` Also adds a `gather_` that offers the same apis but with axis position instead of names. This one is more suitable inside library functions when dealing with unknown names, making it complicated to create the "indices" struct. I think we could generalize this pattern of offering "non-tagged" api with an underscore suffix, simplifying the job of LSP. ## topk new API I realized that most of the code calling topk was renaming the output axis. The idea is that logprob.topk(.voc) doesn't return you a tensor with voc entry but only top_voc entries. For the values it doesn't make much a difference but for the indices it's embarrassing and it break usages of gather/scatter. Now the user of `topK` can either use the unnamed api and pass a number, of if using the name api must give a new name for the indexed axis. ## Yarn Support Yarn rope embeddings. It's equivalent to the "llama3" embedding except they do the interpolation in time space instead of freq space. |
History
|
|
|---|---|---|
| .. | ||
| BUILD.bazel | ||
| mnist.zig | ||