0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/examples/io/main.zig

366 lines
14 KiB

2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-15 16:12:43 +00:00
2026-01-07 12:08:31 +01:00
2026-01-15 16:12:43 +00:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
zml/io: rework `zml.io.load` (#586) The old `zml.io.load` was plagued with problems. This PR is an attempt at improving the state of things. One major problem of `zml.io.load` was that it was not very composable. It was either using it to load all the model or doing everything by hand. Another problem was that it didn't offer a way to apply transformations at load time to the tensors stored in a safetensors file. For example, it happens that for MoE models, instead of the experts weights to be tightly packed in a nice 3D tensor, the weight file contain one tensor for each expert. This is nice until your implementation required a 3D tensor as input. Using the old `zml.io.load` API this resulted in very convoluted way to do concatenation at load time. With this new API, we offer convenience so that it can be handled nicely and as fast as possible. The main change is the introduction of the `Loader` struct. It's a struct that you instantiate once and reuse whenever you want to load tensors. It relies on `Binding`, which are a way to store which tensor are required and how they need to be combined (if it's a complex binding involving more than one source tensor). One advantage of this `Loader` is that it's composable. Loading a struct using `Loader.auto(Struct)` or loading each individual field of `Struct` has essentially the same behavior. This solved the major problem of the old solution. For simplicity I kept the old model loading code almost identical, as this new solution is a drop-in replacement. # Update After some discussions with @gwenzek I've changed my mind. Instead of having a declarative API, we'll provide helper to do transformation easily. The main changes that this PR now bring are: * Introduce `Loader` struct * `Loader` exposes `load` which is the composable replacement of `zml.io.load` * `Loader` exposes `loadExecute` that allows you to execute an executable with the buffers loaded from the sources associated with a tensor. This allows for example to concatenate tensors. We'll continue to work on it, this is just a preliminary work that allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
zml/io: rework `zml.io.load` (#586) The old `zml.io.load` was plagued with problems. This PR is an attempt at improving the state of things. One major problem of `zml.io.load` was that it was not very composable. It was either using it to load all the model or doing everything by hand. Another problem was that it didn't offer a way to apply transformations at load time to the tensors stored in a safetensors file. For example, it happens that for MoE models, instead of the experts weights to be tightly packed in a nice 3D tensor, the weight file contain one tensor for each expert. This is nice until your implementation required a 3D tensor as input. Using the old `zml.io.load` API this resulted in very convoluted way to do concatenation at load time. With this new API, we offer convenience so that it can be handled nicely and as fast as possible. The main change is the introduction of the `Loader` struct. It's a struct that you instantiate once and reuse whenever you want to load tensors. It relies on `Binding`, which are a way to store which tensor are required and how they need to be combined (if it's a complex binding involving more than one source tensor). One advantage of this `Loader` is that it's composable. Loading a struct using `Loader.auto(Struct)` or loading each individual field of `Struct` has essentially the same behavior. This solved the major problem of the old solution. For simplicity I kept the old model loading code almost identical, as this new solution is a drop-in replacement. # Update After some discussions with @gwenzek I've changed my mind. Instead of having a declarative API, we'll provide helper to do transformation easily. The main changes that this PR now bring are: * Introduce `Loader` struct * `Loader` exposes `load` which is the composable replacement of `zml.io.load` * `Loader` exposes `loadExecute` that allows you to execute an executable with the buffers loaded from the sources associated with a tensor. This allows for example to concatenate tensors. We'll continue to work on it, this is just a preliminary work that allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
zml/io: rework `zml.io.load` (#586) The old `zml.io.load` was plagued with problems. This PR is an attempt at improving the state of things. One major problem of `zml.io.load` was that it was not very composable. It was either using it to load all the model or doing everything by hand. Another problem was that it didn't offer a way to apply transformations at load time to the tensors stored in a safetensors file. For example, it happens that for MoE models, instead of the experts weights to be tightly packed in a nice 3D tensor, the weight file contain one tensor for each expert. This is nice until your implementation required a 3D tensor as input. Using the old `zml.io.load` API this resulted in very convoluted way to do concatenation at load time. With this new API, we offer convenience so that it can be handled nicely and as fast as possible. The main change is the introduction of the `Loader` struct. It's a struct that you instantiate once and reuse whenever you want to load tensors. It relies on `Binding`, which are a way to store which tensor are required and how they need to be combined (if it's a complex binding involving more than one source tensor). One advantage of this `Loader` is that it's composable. Loading a struct using `Loader.auto(Struct)` or loading each individual field of `Struct` has essentially the same behavior. This solved the major problem of the old solution. For simplicity I kept the old model loading code almost identical, as this new solution is a drop-in replacement. # Update After some discussions with @gwenzek I've changed my mind. Instead of having a declarative API, we'll provide helper to do transformation easily. The main changes that this PR now bring are: * Introduce `Loader` struct * `Loader` exposes `load` which is the composable replacement of `zml.io.load` * `Loader` exposes `loadExecute` that allows you to execute an executable with the buffers loaded from the sources associated with a tensor. This allows for example to concatenate tensors. We'll continue to work on it, this is just a preliminary work that allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
zml/io: rework `zml.io.load` (#586) The old `zml.io.load` was plagued with problems. This PR is an attempt at improving the state of things. One major problem of `zml.io.load` was that it was not very composable. It was either using it to load all the model or doing everything by hand. Another problem was that it didn't offer a way to apply transformations at load time to the tensors stored in a safetensors file. For example, it happens that for MoE models, instead of the experts weights to be tightly packed in a nice 3D tensor, the weight file contain one tensor for each expert. This is nice until your implementation required a 3D tensor as input. Using the old `zml.io.load` API this resulted in very convoluted way to do concatenation at load time. With this new API, we offer convenience so that it can be handled nicely and as fast as possible. The main change is the introduction of the `Loader` struct. It's a struct that you instantiate once and reuse whenever you want to load tensors. It relies on `Binding`, which are a way to store which tensor are required and how they need to be combined (if it's a complex binding involving more than one source tensor). One advantage of this `Loader` is that it's composable. Loading a struct using `Loader.auto(Struct)` or loading each individual field of `Struct` has essentially the same behavior. This solved the major problem of the old solution. For simplicity I kept the old model loading code almost identical, as this new solution is a drop-in replacement. # Update After some discussions with @gwenzek I've changed my mind. Instead of having a declarative API, we'll provide helper to do transformation easily. The main changes that this PR now bring are: * Introduce `Loader` struct * `Loader` exposes `load` which is the composable replacement of `zml.io.load` * `Loader` exposes `loadExecute` that allows you to execute an executable with the buffers loaded from the sources associated with a tensor. This allows for example to concatenate tensors. We'll continue to work on it, this is just a preliminary work that allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
zml/io: rework `zml.io.load` (#586) The old `zml.io.load` was plagued with problems. This PR is an attempt at improving the state of things. One major problem of `zml.io.load` was that it was not very composable. It was either using it to load all the model or doing everything by hand. Another problem was that it didn't offer a way to apply transformations at load time to the tensors stored in a safetensors file. For example, it happens that for MoE models, instead of the experts weights to be tightly packed in a nice 3D tensor, the weight file contain one tensor for each expert. This is nice until your implementation required a 3D tensor as input. Using the old `zml.io.load` API this resulted in very convoluted way to do concatenation at load time. With this new API, we offer convenience so that it can be handled nicely and as fast as possible. The main change is the introduction of the `Loader` struct. It's a struct that you instantiate once and reuse whenever you want to load tensors. It relies on `Binding`, which are a way to store which tensor are required and how they need to be combined (if it's a complex binding involving more than one source tensor). One advantage of this `Loader` is that it's composable. Loading a struct using `Loader.auto(Struct)` or loading each individual field of `Struct` has essentially the same behavior. This solved the major problem of the old solution. For simplicity I kept the old model loading code almost identical, as this new solution is a drop-in replacement. # Update After some discussions with @gwenzek I've changed my mind. Instead of having a declarative API, we'll provide helper to do transformation easily. The main changes that this PR now bring are: * Introduce `Loader` struct * `Loader` exposes `load` which is the composable replacement of `zml.io.load` * `Loader` exposes `loadExecute` that allows you to execute an executable with the buffers loaded from the sources associated with a tensor. This allows for example to concatenate tensors. We'll continue to work on it, this is just a preliminary work that allows us to do transformation at load time in LLMD.
2026-07-17 18:41:18 +02:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00
2026-01-07 12:08:31 +01:00