0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

18 commits

Author SHA1 Message Date
Rene Schallner
01fd58b46a
fixed example prompts & bazel target name 2025-02-04 14:26:27 +01:00
Rene Schallner
9c98537a84
llama: add tinyllama back into the docs 2025-01-28 11:43:11 +01:00
Rene Schallner
f41e023e22
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162)
# HF tokenizers & Llama improvements

1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example

## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers

`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.

To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.

## 2. examples/llama

Changes to the llama example:

- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
    - single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
Jean-Baptiste Dalido
3682e16da9
workspace: upgrade to Bazel 8 (#143)
This PR covers the migration to Bazel 8.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-01-23 18:36:57 +01:00
Victor Moin
72c56a21bc
Tutorial updates for write_first_model.md (#136)
Quick doc fixes
2024-12-16 21:04:36 +01:00
Guillaume Wenzek
e8fa410537
pjrt/zml: expose client creation flags (#129)
The Pjrt api allows to passe string flags to the plugin on client
creation.
Since we don't like working with strings I've converted the strings
supported by the cuda plugin into a struct.
For now I've focused on the strings corresponding to the allocator
settings following #125 

More settings are not exposed ATM, will follow up with #132 

3d31c48c71/xla/pjrt/c/pjrt_c_api_gpu_internal.cc (L76-L86)

AFAICT the cpu plugin doesn't read flags at all.
Neuron doesn't have known flags, TPU requires more investigation.

**Breaking change**: `context.autoPlatform()` ->
`context.autoPlatform(.{})`
2024-12-11 13:38:04 +01:00
Rene Schallner
754587f6bb
docs: move image in concepts.md (#128) 2024-12-09 19:48:10 +01:00
Guillaume Wenzek
38ee9df735
update instructions (#123)
following #115, `prepare` doesn't alloc anymore, `ExeWithWeights` is
`ModuleExe`
2024-12-06 15:28:56 +01:00
Steeve Morin
43c4348de9
runtimes/neuron: add support for AWS neuron / trainium / inferentia 2 (#102)
This PR adds support for AWS Neuron / Trainium 1 / Inferentia 2.

It can be enabled via the `--@zml//runtimes:neuron=true` flag.

Everything should work as expected.

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2024-11-26 14:20:47 +01:00
Steeve Morin
db88158518
async: rework async runtime to use coroutines and cross thread signaling (#100)
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.

API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)

Mostly, also, it's pretty fast.
2024-11-22 17:56:22 +01:00
Steeve Morin
ccd925309b
async: rename async_ to asyncc and add Generic slugs (#71)
`asyncc` is arguably nicer to write than `async_`, so use that.

Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
2024-11-01 17:36:08 +01:00
Farookh Zaheer Siddiqui
7f6dbbe289
[Docs] : fix typos (#60) 2024-10-22 17:38:05 +02:00
Victor Moin
db665ba706
Add missing load in dockerize_models.md (#62)
Added the missing import `zig_cc_binary` in the simple layer example,
even though it seems obvious when you read the error
2024-10-22 17:35:51 +02:00
Steeve Morin
9b3491f4bf
workspace: be a little more precise regarding the huggingface token (#45)
Indicate that it's also the standard location used by the CLI, and add
the environment variable, too.

---------

Co-authored-by: Rene Schallner <rene@renerocks.ai>
2024-10-06 14:34:38 -07:00
Jon Stevens
a050b3f9ce
make the token directory first if it doesn't already exist (#44) 2024-10-04 11:52:57 -07:00
Hugo Mano
0b136ef43a
zml_utils: fix CollectionOver scope (#13)
**Changes**

- zml_utils: fix `AttributeError: type object 'ActivationCollector' has
no attribute 'CollectionOver'`
- zml_utils: remove dead code and comments
- docs(howto_torch2zml): add explanation on why `output` variable can be
`None`
2024-09-19 13:54:21 +02:00
Rene Schallner
034c004012
docs: first model: fix const vs. var and 80col limit (#3)
There was a const/var bug in the code shown.

Also, after adhering code to roughly 80col limit, the code does not need
to scroll horizontally anymore.
2024-09-17 16:28:30 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00