0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

[LLMD] OutOfMemory on rocm #1086

Closed
opened 2026-11-14 21:08:13 +00:00 by barddoo · 0 comments
barddoo commented 2026-11-14 21:08:13 +00:00 (Migrated from github.com)

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Configuration:

  • Model: Qwen/Qwen3.8-27B
  • OS: Omarchy 4.5.0-3
  • Platform: rocm
  • Accelerator: AMD Radeon RX 7900 XTX
  • Driver version: ROCM-SMI version: 4.0.0+unknown / ROCM-SMI-LIB version: 7.8.0
  • Version: zmlai/llmd@sha256:7f354b8d1f5290234e33ea88d3c42e919eb3ebf33bc7030c426e439ce77e980c
  • Command line: docker run --shm-size=256GB -p 8000:8000 --device=/dev/kfd --device=/dev/dri -e HF_TOKEN xx -it zmlai/llmd:rocm --model=hf://Qwen/Qwen3.8-27B

Additional context

info(llmd): Reserved 0.0.0.0:8000
warning(zml/vfs/hf): No Hugging Face authentication token found in environment or home config; proceeding without authentication.
warning(zml/vfs/s3): S3 initialized without full credentials (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY). Access will be unauthenticated. Using default endpoint: https://s3.amazonaws.com (set AWS_ENDPOINT_URL or AWS_ENDPOINT_URL_S3). Using default region: us-east-1 (set AWS_REGION or AWS_DEFAULT_REGION).
info(llmd): Resolving model repo
info(zml/vfs/hf): Fetching from HF: https://huggingface.co/api/models/Qwen/Qwen3.8-27B/tree/main/?expand=false&recursive=true&limit=1000
info(llmd): Loading model config
info(llmd): Loaded model config [2.517s]
info(llmd): Loading tokenizer
info(pjrt): Loading: libzml_rocm.so...
info(pjrt): Loaded: libzml_rocm.so
info(llmd): Devices:
info(llmd):     - AMD Radeon RX 7900 XTX (rocm:0)
info(llmd): Loaded tokenizer [2.341s]
error: OutOfMemory
**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Configuration:** - Model: Qwen/Qwen3.8-27B - OS: Omarchy 4.5.0-3 - Platform: rocm - Accelerator: AMD Radeon RX 7900 XTX - Driver version: ROCM-SMI version: 4.0.0+unknown / ROCM-SMI-LIB version: 7.8.0 - Version: zmlai/llmd@sha256:7f354b8d1f5290234e33ea88d3c42e919eb3ebf33bc7030c426e439ce77e980c - Command line: docker run --shm-size=256GB -p 8000:8000 --device=/dev/kfd --device=/dev/dri -e HF_TOKEN xx -it zmlai/llmd:rocm --model=hf://Qwen/Qwen3.8-27B **Additional context** ``` info(llmd): Reserved 0.0.0.0:8000 warning(zml/vfs/hf): No Hugging Face authentication token found in environment or home config; proceeding without authentication. warning(zml/vfs/s3): S3 initialized without full credentials (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY). Access will be unauthenticated. Using default endpoint: https://s3.amazonaws.com (set AWS_ENDPOINT_URL or AWS_ENDPOINT_URL_S3). Using default region: us-east-1 (set AWS_REGION or AWS_DEFAULT_REGION). info(llmd): Resolving model repo info(zml/vfs/hf): Fetching from HF: https://huggingface.co/api/models/Qwen/Qwen3.8-27B/tree/main/?expand=false&recursive=true&limit=1000 info(llmd): Loading model config info(llmd): Loaded model config [2.517s] info(llmd): Loading tokenizer info(pjrt): Loading: libzml_rocm.so... info(pjrt): Loaded: libzml_rocm.so info(llmd): Devices: info(llmd): - AMD Radeon RX 7900 XTX (rocm:0) info(llmd): Loaded tokenizer [2.341s] error: OutOfMemory
Sign in to join this conversation.
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
DKB/zml#1086
No description provided.