0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/examples
Raphael Nahoum 46bd32de2f zml/tokenizer: grow the decoder output to fit any token
IREE writes a token's text whole or not at all, so `feedOne` asserted
whenever the caller's buffer was smaller than the token, e.g. Laguna's
415-byte whitespace tokens in a 128-byte buffer. The decoder now owns
its output buffer and grows it the way `decode` already did, so
`feedOne` and `finalize` no longer take a buffer.
History 2026-11-09 10:56:20 +01:00
..
benchmark zml/Tensor: explicit sharding in Tensor code (#868) 2026-10-06 15:37:22 +02:00
io zml/Tensor: explicit sharding in Tensor code (#868) 2026-10-06 15:37:22 +02:00
llm zml/tokenizer: grow the decoder output to fit any token 2026-11-09 10:56:20 +01:00
mnist zml.Compiler: allow to propagate tensor store names (#1018) 2026-11-04 16:04:17 +01:00
sharding zml/ops: add manualComputation2 with explicit partitioning (#1013) 2026-11-03 18:22:59 +01:00