mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Bump to resolve a compiler bug encountered on LFM3.
LFM3 broadcasts one sigmoid gate per attention head. This generated code
of sigmoid introduced a BF16 conversion and conditional select. The
compiler then lower it to a scalar computation which had a
register-aligment bug. The fix introduced in 2.41.11
<
|
History
|
|
|---|---|---|
| .. | ||
| BUILD.bazel | ||
| libzml_oneapi.BUILD.bazel | ||
| oneapi.bzl | ||
| oneapi.h | ||
| oneapi.zig | ||
| packages.lock.json | ||
| packages.yaml | ||
| README.md | ||
| zmlxoneapi.zig | ||
EXPERIMENTAL: oneAPI
About
This directory contains the oneAPI backend for ZML. It is currently in an early stage of development, and only supports a subset of ZML's features. The oneAPI backend is only available on Linux, and requires an Intel GPU.
The sharding is not yet supported, so only models that fit in the GPU memory can be run.
Getting Started
Prerequisites
- An Intel GPU
- bazel see Getting Started for installation instructions.
Linux Only
One GPU is supported on Linux. ONEAPI_DEVICE_SELECTOR environment variable must be set to select the GPU device.
For example, to select the first GPU:
export ONEAPI_DEVICE_SELECTOR=level_zero:0
Run
To run the LLM example with oneAPI and the Llama 3.2 1B model:
ONEAPI_DEVICE_SELECTOR=level_zero:0 \
bazel run //examples/llm \
--config=release \
--@zml//platforms:cpu=false \
--@zml//platforms:oneapi=true \
-- \
--model=hf://meta-llama/Llama-3.2-1B-Instruct \
--topk=1 \
--prompt="Tell me a story about a cat in 2 lines"
To run the LLM example with oneAPI and a local model:
ONEAPI_DEVICE_SELECTOR=level_zero:0 \
bazel run //examples/llm \
--config=release \
--@zml//platforms:cpu=false \
--@zml//platforms:oneapi=true \
-- \
--model=path/to/your/model/Llama-3.1-8B-Instruct \
--topk=1 \
--prompt="Tell me a story about a cat in 2 lines"