0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

[LLMD] llmd oneapi image: Qwen2.5-1.5B and Qwen2.5-3B crash on first request ("OpenCL device not found") #1080

Open
opened 2026-11-13 20:14:21 +00:00 by davlgd · 1 comment
davlgd commented 2026-11-13 20:14:21 +00:00 (Migrated from github.com)

Summary

With zmlai/llmd:oneapi on an Arc Pro B60, some models load fine but the server aborts on the first chat completion request, because oneDNN looks for an OpenCL device and the image ships no Intel OpenCL ICD. Other models, including one from the same family, work with the exact same command.

Configuration:

  • Model: Qwen/Qwen2.5-1.5B-Instruct, Qwen/Qwen2.5-3B-Instruct
  • OS: Debian 13, kernel 7.2.12
  • Platform: oneapi
  • Accelerator: Intel Arc Pro B60 24 GB
  • Driver version: bundled in the image (Level Zero GPU driver 26.31.39395.14); kernel driver xe
  • Version: docker image sha256:5a4af1e7545249f8bff581745f362f5a23625f4c9737246796242d89d32f7281 (tag 20261112.0-oneapi)
  • Command line: docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct

Environment

  • Image: zmlai/llmd:oneapi (tag 20261112.0-oneapi, digest sha256:5a4af1e7545249f8bff581745f362f5a23625f4c9737246796242d89d32f7281)
  • Discrete GPU: Intel Arc Pro B60 24 GB (ASRock, PCI 8086:e211), xe driver, GuC 70.72.1
  • Integrated GPU: Intel Arrow Lake-S graphics (PCI 8086:7d67), i915 driver, CPU Core Ultra 7 270K Plus
  • Host: Debian 13, kernel 7.2.12 (trixie-backports), Docker 26.1.5
  • No Intel compute runtime installed on the host: the container only gets the device nodes

Steps to reproduce

The container is restricted to the Arc card with ZE_AFFINITY_MASK=0 (see issue 2 for why):

docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 \
  zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen2.5-1.5B-Instruct","messages":[{"role":"user","content":"What is the capital of France?"}],"max_tokens":24}'

Actual result

The server starts, detects the card and loads the weights:

info(llmd): Devices:
info(llmd): 	- Intel(R) Arc(TM) Pro B60 Graphics (oneapi:0)
info(models): Loaded weights [2.88GiB, ...]
info(llmd): Loaded a model of type llama
info(llmd): Listening on 0.0.0.0:8000

On the first request it aborts (exit code 134):

onednn_verbose,v1,info,gpu,engine,0,backend:Level Zero,name:Intel(R) Arc(TM) Pro B60 Graphics,driver_version:1.17.39395,binary_kernels:enabled
onednn_verbose,v1,common,error,runtime,Intel(R) Arc(TM) Pro B60 Graphics OpenCL device not found,src/gpu/intel/sycl/utils.cpp:270
warning(pjrt): [PJRT_Event_Await] Internal: oneDNN matmul failed: could not create a primitive
thread 7 panic: Device I/O error

Which models are affected

Same command, same card, one request each, no OpenCL ICD provided:

Model model_type hidden / intermediate heads / KV heads Result
Qwen/Qwen2.5-0.5B-Instruct qwen2 896 / 4864 14 / 2 works
Qwen/Qwen2.5-1.5B-Instruct qwen2 1536 / 8960 12 / 2 crash
Qwen/Qwen2.5-3B-Instruct qwen2 2048 / 11008 16 / 2 crash
Qwen/Qwen3-0.6B qwen3 1024 / 3072 16 / 8 works
Qwen/Qwen3-1.7B qwen3 2048 / 6144 16 / 8 works
Qwen/Qwen3-4B qwen3 2560 / 9728 32 / 8 works

All six are reported by llmd as "a model of type llama". The pattern does not follow the hidden size alone (Qwen3-1.7B and Qwen2.5-3B share 2048). I could not identify what makes the two failing models reach the oneDNN matmul that needs an OpenCL device.

Analysis

  • The image contains libOpenCL.so (loader), libze_intel_gpu.so.1 (reports 26.31.39395.14), libigc.so.2 and libigdfcl.so.2, but no libigdrcl.so, no *.icd file and no /etc/OpenCL/vendors. platforms/oneapi/packages.yaml lists libze-intel-gpu1, libigc2, libigdfcl2, libigdgmm12 and libze1, but not intel-opencl-icd.
  • oneDNN's sycl_dev2ocl_dev (src/gpu/intel/sycl/utils.cpp) looks up the OpenCL device matching the Level Zero device and raises this error when no OpenCL device is found.
  • Not related to virtualization: reproduced on bare metal and inside a KVM guest using an SR-IOV VF of the same card, with identical logs.

Workaround

Providing an Intel OpenCL ICD to the container fixes it. The bundled OpenCL loader honours OCL_ICD_FILENAMES:

docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 \
  -v /path/to/intel-opencl:/usr/lib/x86_64-linux-gnu/intel-opencl:ro \
  -e OCL_ICD_FILENAMES=/usr/lib/x86_64-linux-gnu/intel-opencl/libigdrcl.so \
  zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct

Tested with libigdrcl.so from Intel's intel-opencl-icd 26.40.40196.8 and from 26.31.39395.13 (the closest published release to the driver in the image). Both work with Qwen2.5-1.5B, with the same throughput (about 75 tokens/s for a single request on this card).

Possible fixes

  • Add intel-opencl-icd (matching the bundled Level Zero driver) to the oneAPI image, or
  • avoid the code path that requires an OpenCL device, or
  • document the requirement and the OCL_ICD_FILENAMES workaround.
### Summary With `zmlai/llmd:oneapi` on an Arc Pro B60, some models load fine but the server aborts on the first chat completion request, because oneDNN looks for an OpenCL device and the image ships no Intel OpenCL ICD. Other models, including one from the same family, work with the exact same command. **Configuration:** - Model: `Qwen/Qwen2.5-1.5B-Instruct`, `Qwen/Qwen2.5-3B-Instruct` - OS: Debian 13, kernel 7.2.12 - Platform: `oneapi` - Accelerator: Intel Arc Pro B60 24 GB - Driver version: bundled in the image (Level Zero GPU driver 26.31.39395.14); kernel driver `xe` - Version: docker image `sha256:5a4af1e7545249f8bff581745f362f5a23625f4c9737246796242d89d32f7281` (tag `20261112.0-oneapi`) - Command line: `docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct` ### Environment - Image: `zmlai/llmd:oneapi` (tag `20261112.0-oneapi`, digest `sha256:5a4af1e7545249f8bff581745f362f5a23625f4c9737246796242d89d32f7281`) - Discrete GPU: Intel Arc Pro B60 24 GB (ASRock, PCI `8086:e211`), `xe` driver, GuC 70.72.1 - Integrated GPU: Intel Arrow Lake-S graphics (PCI `8086:7d67`), `i915` driver, CPU Core Ultra 7 270K Plus - Host: Debian 13, kernel 7.2.12 (trixie-backports), Docker 26.1.5 - No Intel compute runtime installed on the host: the container only gets the device nodes ### Steps to reproduce The container is restricted to the Arc card with `ZE_AFFINITY_MASK=0` (see issue 2 for why): ```bash docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 \ zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct ``` ```bash curl -X POST http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"Qwen2.5-1.5B-Instruct","messages":[{"role":"user","content":"What is the capital of France?"}],"max_tokens":24}' ``` ### Actual result The server starts, detects the card and loads the weights: ``` info(llmd): Devices: info(llmd): - Intel(R) Arc(TM) Pro B60 Graphics (oneapi:0) info(models): Loaded weights [2.88GiB, ...] info(llmd): Loaded a model of type llama info(llmd): Listening on 0.0.0.0:8000 ``` On the first request it aborts (exit code 134): ``` onednn_verbose,v1,info,gpu,engine,0,backend:Level Zero,name:Intel(R) Arc(TM) Pro B60 Graphics,driver_version:1.17.39395,binary_kernels:enabled onednn_verbose,v1,common,error,runtime,Intel(R) Arc(TM) Pro B60 Graphics OpenCL device not found,src/gpu/intel/sycl/utils.cpp:270 warning(pjrt): [PJRT_Event_Await] Internal: oneDNN matmul failed: could not create a primitive thread 7 panic: Device I/O error ``` ### Which models are affected Same command, same card, one request each, no OpenCL ICD provided: | Model | `model_type` | hidden / intermediate | heads / KV heads | Result | |---|---|---|---|---| | `Qwen/Qwen2.5-0.5B-Instruct` | qwen2 | 896 / 4864 | 14 / 2 | works | | `Qwen/Qwen2.5-1.5B-Instruct` | qwen2 | 1536 / 8960 | 12 / 2 | **crash** | | `Qwen/Qwen2.5-3B-Instruct` | qwen2 | 2048 / 11008 | 16 / 2 | **crash** | | `Qwen/Qwen3-0.6B` | qwen3 | 1024 / 3072 | 16 / 8 | works | | `Qwen/Qwen3-1.7B` | qwen3 | 2048 / 6144 | 16 / 8 | works | | `Qwen/Qwen3-4B` | qwen3 | 2560 / 9728 | 32 / 8 | works | All six are reported by llmd as "a model of type llama". The pattern does not follow the hidden size alone (Qwen3-1.7B and Qwen2.5-3B share 2048). I could not identify what makes the two failing models reach the oneDNN matmul that needs an OpenCL device. ### Analysis - The image contains `libOpenCL.so` (loader), `libze_intel_gpu.so.1` (reports `26.31.39395.14`), `libigc.so.2` and `libigdfcl.so.2`, but no `libigdrcl.so`, no `*.icd` file and no `/etc/OpenCL/vendors`. `platforms/oneapi/packages.yaml` lists `libze-intel-gpu1`, `libigc2`, `libigdfcl2`, `libigdgmm12` and `libze1`, but not `intel-opencl-icd`. - oneDNN's `sycl_dev2ocl_dev` (`src/gpu/intel/sycl/utils.cpp`) looks up the OpenCL device matching the Level Zero device and raises this error when no OpenCL device is found. - Not related to virtualization: reproduced on bare metal and inside a KVM guest using an SR-IOV VF of the same card, with identical logs. ### Workaround Providing an Intel OpenCL ICD to the container fixes it. The bundled OpenCL loader honours `OCL_ICD_FILENAMES`: ```bash docker run -p 8000:8000 --device=/dev/dri -e ZE_AFFINITY_MASK=0 \ -v /path/to/intel-opencl:/usr/lib/x86_64-linux-gnu/intel-opencl:ro \ -e OCL_ICD_FILENAMES=/usr/lib/x86_64-linux-gnu/intel-opencl/libigdrcl.so \ zmlai/llmd:oneapi --model=hf://Qwen/Qwen2.5-1.5B-Instruct ``` Tested with `libigdrcl.so` from Intel's `intel-opencl-icd` 26.40.40196.8 and from 26.31.39395.13 (the closest published release to the driver in the image). Both work with Qwen2.5-1.5B, with the same throughput (about 75 tokens/s for a single request on this card). ### Possible fixes - Add `intel-opencl-icd` (matching the bundled Level Zero driver) to the oneAPI image, or - avoid the code path that requires an OpenCL device, or - document the requirement and the `OCL_ICD_FILENAMES` workaround.
steeve commented 2026-11-13 20:28:13 +00:00 (Migrated from github.com)

@davlgd we might have an update later today that fixes it

@davlgd we might have an update later today that fixes it
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
DKB/zml#1080
No description provided.