mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Add experimental Intel GPU backend Add oneAPI as a ZML platform target with Bazel flags, module extension repositories, and PJRT runtime packaging for Linux Intel GPUs. - package libpjrt_oneapi with oneAPI runtime libraries, Level Zero loader, libsycl, and a zmlxoneapi dlopen shim for versioned shared libraries - register the oneAPI platform loader and detect available render devices before loading the packaged PJRT plugin from runfiles - thread the oneAPI target through platform selection, sharding, placement, buffers, execution, memory, IO, tensor behavior, and XLA compile flags - document oneAPI prerequisites and example LLM usage - update Bazel module lockfile entries for the new runtime dependencies --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Steeve Morin <steeve@9960x-b70x2.tailb57cfe.ts.net>
1.4 KiB
1.4 KiB
EXPERIMENTAL: oneAPI
About
This directory contains the oneAPI backend for ZML. It is currently in an early stage of development, and only supports a subset of ZML's features. The oneAPI backend is only available on Linux, and requires an Intel GPU.
The sharding is not yet supported, so only models that fit in the GPU memory can be run.
Getting Started
Prerequisites
- An Intel GPU
- bazel see Getting Started for installation instructions.
Linux Only
One GPU is supported on Linux. ONEAPI_DEVICE_SELECTOR environment variable must be set to select the GPU device.
For example, to select the first GPU:
export ONEAPI_DEVICE_SELECTOR=level_zero:0
Run
To run the LLM example with oneAPI and the Llama 3.2 1B model:
ONEAPI_DEVICE_SELECTOR=level_zero:0 \
bazel run //examples/llm \
--config=release \
--@zml//platforms:cpu=false \
--@zml//platforms:oneapi=true \
-- \
--model=hf://meta-llama/Llama-3.2-1B-Instruct \
--topk=1 \
--prompt="Tell me a story about a cat in 2 lines"
To run the LLM example with oneAPI and a local model:
ONEAPI_DEVICE_SELECTOR=level_zero:0 \
bazel run //examples/llm \
--config=release \
--@zml//platforms:cpu=false \
--@zml//platforms:oneapi=true \
-- \
--model=path/to/your/model/Llama-3.1-8B-Instruct \
--topk=1 \
--prompt="Tell me a story about a cat in 2 lines"