mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
IREE writes a token's text whole or not at all, so `feedOne` asserted whenever the caller's buffer was smaller than the token, e.g. Laguna's 415-byte whitespace tokens in a 128-byte buffer. The decoder now owns its output buffer and grows it the way `decode` already did, so `feedOne` and `finalize` no longer take a buffer. |
History
|
|
|---|---|---|
| .. | ||
| benchmark | ||
| io | ||
| llm | ||
| mnist | ||
| sharding | ||