0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

4 commits

Author SHA1 Message Date
Guillaume Wenzek
5aa1a22d9f
async should only use sync logging (#225)
currently using async.LogFn inside async runtime itself panics.
So I'm avoiding that by forcing async.zig logging to go through
std.log.defaultLog which is synchronous.

you can reproduce the crash by changing log level to .debug
2025-04-16 09:35:18 +02:00
Rene Schallner
f41e023e22
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162)
# HF tokenizers & Llama improvements

1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example

## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers

`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.

To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.

## 2. examples/llama

Changes to the llama example:

- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
    - single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
Steeve Morin
f228ec2edc
async: fix async hangs (#154)
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.

In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.

Should have a slight performance increase, too.
2025-01-18 00:50:51 +01:00
Steeve Morin
5309a25420
async: vendor zigcoro and unify APIs (#152)
This PR merges zigcoro inside the repo and reworks its internals to play
well with `stdx.meta`.

It also adds `Channel.try_{send,recv}` methods as well as dynamically
sized channels (with a `comptime` capacity).

Although not used, it also introduces a `PoolStackAllocator` for
allocating coroutine stacks from a pool
2025-01-14 19:33:16 +01:00