Hugo Mano
69e6f3f176
tensor output memory
2025-02-03 14:30:57 +00:00
Hugo Mano
00ac82ab55
dataInMemory
2025-01-31 14:14:08 +00:00
Hugo Mano
743b6ac74b
wip
2025-01-31 14:14:08 +00:00
Hugo Mano
be97bf79ea
wip
2025-01-31 14:14:08 +00:00
Rene Schallner
4592115638
llama: fix dims & dtypes -> fix #170 ( #171 )
...
This PR fixes issue #170 , making Llama run again on CUDA and RoCM.
- Latest Llama changes had introduced an unnecessary batching dimension:
fixed
- Ops returning indices return i32 yet we now use u32 as token indices
-> dtype conversion requred: fixed
Tested on CUDA.
2025-01-31 14:14:08 +00:00
Guillaume Wenzek
bc514c191d
revert zml: exe should collect events
2025-01-30 18:14:34 +01:00
Guillaume Wenzek
6050d4f032
zml: remove api from Shard
2025-01-29 10:28:55 +01:00
Guillaume Wenzek
01c54e0e72
zml: exe should collect events
2025-01-29 10:28:49 +01:00
Corentin Godeau
d9f2fa8d19
zml/tensor.zig: add maximum as scatter operation
2025-01-28 11:09:01 +00:00
Steeve Morin
f4eb7594b4
zml/module: disable CUDA libnvptx for older plugin
2025-01-26 12:16:32 +00:00
Steeve Morin
6e3879546b
zml/hostbuffer: is buffer is ready, mark it ready
2025-01-26 12:16:04 +00:00
Steeve Morin
39a64d1041
zml/buffer: await constant buffer creation
2025-01-26 12:15:35 +00:00
Steeve Morin
b3fdf197fe
zml: add asynchronous HostBuffer event
2025-01-24 21:15:50 +01:00
Steeve Morin
1141fa7f9d
zml,mnist: fix async buffers handling
2025-01-24 21:11:03 +01:00
Rene Schallner
9a683a87b2
zml(buffer): fix const bug in toHostAlloc
2025-01-24 21:10:40 +01:00
Steeve Morin
85698b515e
zml/exe: don't await for shard after execution
2025-01-24 21:10:36 +01:00
Steeve Morin
2612952a44
zml: add Buffer.Shard type
2025-01-24 21:10:25 +01:00
Steeve Morin
9517504d02
zml/aio: await weights transfers
2025-01-24 21:09:50 +01:00
Rene Schallner
9ca43972d8
llama: fix async toHost usage when fetching token from device.
2025-01-24 21:09:43 +01:00
Steeve Morin
8a5b73e70b
zml/aio: wait for buffer readiness in loadBuffers
2025-01-24 21:09:40 +01:00
Steeve Morin
7980fb51ff
zml: make buffers and execute asynchronous
...
This change makes buffer and execution completely asynchronous. This
enable easy pipelining of multiple operations. It doesn't map well to
`async` because the PJRT API is already asynchronous, so might as well
use it as is.
2025-01-24 21:09:12 +01:00