mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Makes kernels written in the DSLs read alike, without changing what they emit. - Every DSL target is `//dsl/<name>:builder`, and `dsl/common` is imported as common. - Scopes without values end with `close()`. - The scf DSLs (triton, mosaic, cute, fly) share `openFor`/`openIf`/`openIfElse`/`openWhile` in `dsl/common`. cuda_tile keeps its own scopes with the same shapes. - Scopes take tuples or arrays, and result types can be dtypes (.f32), Values, or MLIR types. - Typed constants are `constant(dtype, value)`, alongside constIndex and constantLike. - Comparisons and bitcasts have the same names in every DSL: Fly gains `lt`/`eq`/…, and every DSL spells it `value.bitCast(dtype)`. - One block stack and one `common.verify`, which logs the module that fails to verify. - READMEs updated.
6.7 KiB
6.7 KiB
Porting a FlyDSL Python kernel to Zig
The Zig DSL builds Fly IR directly. Work from the revision and patches in
the FlyDSL repository rule, reading
Python sources with git -C <FlyDSL checkout> show <pin>:<path>. Check the
ROCm plugin pin as well: the emitter and
runtime must agree on the dialect version. FlyDSL HEAD may differ from both.
Cheat sheet
Python (flydsl.expr) |
Zig |
|---|---|
@flyc.kernel def k(A: Tensor, ...) |
fly.Kernel(Cfg, .{ .inputs = &.{"a"}, .outputs = &.{"c"}, .run = run }); const a = K.args(b) |
kernel arg tiled_copy: fx.TiledCopy |
build it inside run (static values cannot be arguments) |
fx.thread_idx.x, fx.block_idx.x |
b.threadId(.x), b.blockId(.x) (i32) |
A.shape.unpack() |
a.a.shapeStatic().at(i).leaf.s |
fx.make_layout((8,16),(16,1)) |
fly.L(.{ 8, 16 }, .{ 16, 1 }) (comptime) or b.static(layout) |
fx.make_ordered_layout((8,16),(1,0)) |
fly.ordered(.{ 8, 16 }, .{ 1, 0 }) |
fx.make_tile(64, 8) / tuple divisor (64, 8) |
fly.tile(.{ 64, 8 }) / .{ 64, 8 } |
fx.make_int_tuple((m, tid)), coords |
b.intTuple(.{ m, tid }) (Values become ? leaves) |
t[None, None, bx, by] |
t.slice(.{ null, null, bx, by }) |
t[(0, None), None, None] |
t.slice(.{ .{ 0, null }, null, null }) |
t[i] (no None) / t[i] = v |
t.at(i) / t.set(i, v) |
fx.flat_divide(t, tile) etc. |
t.flatDivide(tile), .logicalDivide, .zippedDivide, .tiledDivide |
fx.raked_product(a, b) etc. |
a.rakedProduct(b), .logicalProduct, ... |
fx.right_inverse(l), fx.composition(l, r) |
l.rightInverse(), l.composition(r) |
fx.get_shape(l), fx.size(l) (emitted) |
l.emitShape(), l.emitSize() |
fx.size(t.shape).unpack() (host int) |
t.sizeStatic() |
fx.select(tup, [0]), fx.get(tup, [1]) |
tup.selectModes(&.{0}), tup.get(&.{1}) |
fx.make_view((0,0), fx.make_identity_layout((M,N))) |
b.identity(.{ m, n }) |
fx.make_view(ptr, layout) |
ptr.view(layout) |
fx.make_copy_atom(fx.UniversalCopy128b(), fx.Float32) |
b.copyAtom(.{ .universal = 128 }, .f32) |
fx.rocdl.BufferCopy32b() / BufferCopyLDS128b() |
.{ .buffer_copy = 32 } / .{ .buffer_copy_lds = 128 } |
fx.make_mma_atom(fx.rocdl.MFMA(16,16,4,fx.Float32)) |
b.mmaAtom((try fly.fly_rocdl.MmaOpCDNA3MFMAType.get(b.ctx, .{ .m = 16, .n = 16, .k = 4, ... })).type_()) |
fx.make_tiled_mma(atom, layout) |
b.tiledMma(atom, layout, null) |
fx.make_tiled_copy_tv(atom, thr, val) |
b.tiledCopyTV(atom, thr, val) |
fx.make_tiled_copy_A(atom, tiled_mma) |
b.tiledCopyA(atom, tiled_mma) |
tiled_copy.tile_mn |
tc.tileMN() |
tiled_mma.tile_size_mnk etc. |
@import("mlir/dialects/fly").expect(tm.value.type_(), .tiled_mma).tileSizeMNK() |
tiled_copy.get_slice(tid).partition_S(t) |
tc.getSlice(tid).partitionS(t) |
thr_copy.retile(t) |
thr.retile(t) |
tiled_mma.make_fragment_A(t) |
tm.makeFragmentA(t) |
fx.make_fragment_like(t[, dtype=fx.Boolean]) |
t.makeFragmentLike(null) / t.makeFragmentLike(.i1) |
fx.make_rmem_tensor(8, fx.Float16) |
b.rmemTensor(fly.L(8, 1), .f16) |
fx.copy(atom, src, dst, pred=p) |
b.copy(atom, src, dst, .{ .pred = p }) |
fx.copy_atom_call(atom, src, dst) |
b.copyAtomCall(atom, src, dst) |
fx.gemm(atom, d, a, b, c) |
b.gemm(atom, d, a, bb, c) |
t.load(), t.store(v), t.fill(0) |
same names |
fx.elem_less(coord, (M, N)) |
coord.elemLess(.{ m, n }) |
fx.rocdl.make_buffer_tensor(t) |
b.bufferTensor(t) |
fx.SharedAllocator().allocate(fx.Array[f32, n, 16]) |
b.sharedArray(.f32, n, 16) (returns an n:1 tensor) |
gpu.barrier() |
b.barrier() |
x.shuffle_xor(off, 64) |
x.shuffleXor(off, 64) |
v.reduce(ReductionOp.ADD) |
v.reduce(.add) |
v.to(fx.Float32) |
v.to(.f32) |
x * rrms (vector × scalar) |
x.mul(rrms) |
fmath.rsqrt(x, fastmath=fast) |
b.setFastMath(true); b.rsqrt(x) |
lane < RED_SLOTS, cond.select(a, b) |
lane.lt(n), cond.select(a, b) |
if tid == 0: ... |
var s = b.openIf(tid.eq(0)); ...; s.close(); |
for i in range_constexpr(n) |
a plain Zig for (0..n) (unrolled at emit time) |
for i in range(lo, hi) with carried values |
b.openFor(lo, hi, 1, .{ init }) + .yield(.{ ... }) |
launch(grid=..., block=(256,1,1)) |
K.call(inputs, outputs, .{ .grid = ..., .threads = 256 }) |
Method
- Read the kernel and its helpers at the pin; note every
fx.*primitive. - Write the Zig
runtop to bottom, preserving operation order and numeric conversions. For matrix loops, use SSA copy and MMA calls as described under control flow and register tensors:b.copyAtomLoadreturns vectors,b.mmaAtomCallreturns the accumulator, andb.copyAtomStorewrites vectors. Carry accumulator values throughopenFor; do not put mutable register tensors across control-flow regions. bazel test //dsl/fly:test(or aK.emittest): the module must verify and re-parse. A panic fromfly.<op>: cannot create operationlists the operand types; compare them with the Python trace.- Run on the ROCm box against a host reference
(
bazel test //zml:test --@zml//platforms:rocm=true ...).
Things that differ
- Python config (
N,dtype_str,const_expr(...)branches) becomes the kernelConfig; branch on it with plainifinrun. - Python lists of vectors across an unrolled loop are Zig arrays of
Value.v.insert(i, x)returns a new vector;v.extract(i)reads an element.v.bitCast(dtype)preserves total bits and adjusts vector length. - Keep unsigned semantics explicit for packed bytes and offsets with
toUnsigned,divUnsigned,remUnsignedandcmpUnsigned. v.shuffleVector(rhs, indices)rearranges vector elements within a thread;v.shuffle(.up, offset, width)andshuffleXorexchange values across lanes.- Runtime
whileloops useb.openWhile(inits, after_types),yieldBeforeandyieldAfter; use a plain Zigwhileonly for loops unrolled at emit time. - Python
Int32(x)/Float32(x)constants areb.constant(.i32, x); most arithmetic methods also take a literal directly (tid.add(256)). - Cached scalars from LDS are read with
t.at(i); there is no implicit load. - bf16 outputs use
arith.truncf; the Python's manual round-to-nearest bit trick for pre-gfx950 parts is not reproduced.
Model kernels
The MoE kernels include native routing and
MXFP4 GEMV/MFMA implementations. llmd's sparse MLA kernel
(llmd/kernels/mla/fly/sparse_mla.zig, downstream) uses SSA
matrix accumulators through attention loops. These are examples of the
builder APIs with model tensor layouts and launch wrappers.