0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

Compare commits

..

51 commits

Author SHA1 Message Date
Hugo Mano
6b6152a291 cleanup from tpu 2025-08-08 12:42:00 +00:00
Hugo Mano
fe0b994273 fix norm weight sharding 2025-08-06 15:25:46 +00:00
Hugo Mano
bf9369abcc restore writeShardingRepresentation 2025-08-06 15:20:02 +00:00
Hugo Mano
8fc2a7e939 another attempt 2025-08-06 15:14:33 +00:00
Hugo Mano
df2e6eb884 rollback some stuffs 2025-08-06 15:06:49 +00:00
Hugo Mano
ac6fd43f08 tpu sharding fix 2025-08-06 14:06:21 +00:00
Hugo Mano
ca98ce0bea llama tpu 2025-08-06 13:22:44 +00:00
Hugo Mano
8c7afd6b08 physical mesh 2025-08-06 12:34:02 +00:00
Hugo Mano
717b0c405e 100 toks/s 2025-08-05 18:57:31 +00:00
Hugo Mano
f0f11930e2 8b okay 70b ko 2025-08-05 18:20:48 +00:00
Hugo Mano
5294d0b77c cleanup 2025-08-04 15:38:38 +00:00
Hugo Mano
0acedf2ba3 restore uninitialized 2025-07-31 14:44:35 +00:00
Hugo Mano
adc28fa325 remove check 2025-07-31 13:57:26 +00:00
Hugo Mano
acfea5c6ae fixes for tpu 2025-07-31 13:54:32 +00:00
Hugo Mano
c687600280 device description 2025-07-31 11:32:23 +00:00
Hugo Mano
6e885435e6 fix flags 2025-07-31 09:39:01 +00:00
Hugo Mano
b3f84f3179 back to something working 2025-07-31 09:29:32 +00:00
Hugo Mano
77e68344d5 add flags for nccl 2025-07-31 09:29:32 +00:00
Hugo Mano
7d3a2d0441 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
07c58ff185 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
beaf19824b try 2025-07-31 09:29:32 +00:00
Hugo Mano
585b51db44 gather right dim 2025-07-31 09:29:32 +00:00
Hugo Mano
933445a76b fix 70B 2025-07-31 09:29:32 +00:00
Hugo Mano
3d179606ad wip 2025-07-31 09:29:32 +00:00
Hugo Mano
a6b5e550e3 sharding output 2025-07-31 09:29:32 +00:00
Hugo Mano
4897398a82 new sharding 2025-07-31 09:29:32 +00:00
Hugo Mano
a6aa94c6e3 rollback 2025-07-31 09:29:32 +00:00
Hugo Mano
050b98d0de wip 2025-07-31 09:29:32 +00:00
Hugo Mano
c38ea4553c attempt 2025-07-31 09:29:32 +00:00
Hugo Mano
4751c3d666 use allGather 2025-07-31 09:29:32 +00:00
Hugo Mano
c5a87e0ba4 just test nccl 2025-07-31 09:29:32 +00:00
Hugo Mano
78aecbcb30 setup pjrt profiler 2025-07-31 09:29:32 +00:00
Hugo Mano
4874ad497e working llama 2025-07-31 09:29:32 +00:00
Hugo Mano
8b571abbcd collectives 2025-07-31 09:29:32 +00:00
Hugo Mano
214f78cb43 restore zml call 2025-07-31 09:29:32 +00:00
Hugo Mano
3dfca85029 deactivate zml call for fusion 2025-07-31 09:29:32 +00:00
Hugo Mano
354c8cfe0a simpler 2025-07-31 09:29:32 +00:00
Hugo Mano
f027119199 transpose k 2025-07-31 09:29:32 +00:00
Hugo Mano
966da15cc2 correct mlp sharding 2025-07-31 09:29:32 +00:00
Hugo Mano
044d7c4702 another version 2025-07-31 09:29:32 +00:00
Hugo Mano
229a1c3159 attempt 2025-07-31 09:29:32 +00:00
Hugo Mano
8380847390 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
980587e011 restore zml.call 2025-07-31 09:29:32 +00:00
Hugo Mano
7bfc85baa5 fix MLP 2025-07-31 09:29:32 +00:00
Hugo Mano
e9022db647 ops allReduce 2025-07-31 09:29:32 +00:00
Hugo Mano
2d4202e5bb oupsi 2025-07-31 09:29:32 +00:00
Hugo Mano
1c913e4662 megatron 2025-07-31 09:29:32 +00:00
Hugo Mano
695f85216e add logs 2025-07-31 09:29:32 +00:00
Hugo Mano
27bf92c866 reduce / scatter 2025-07-31 09:29:32 +00:00
Hugo Mano
69e1d19205 rebase 2025-07-31 09:29:32 +00:00
Hugo Mano
10f86e93f1 explo partitioning 2025-07-31 09:29:32 +00:00
1202 changed files with 59643 additions and 229096 deletions

1
.bazelignore Symbolic link

1
.bazelrc Symbolic link

1
.bazelversion Symbolic link

11
.gitignore vendored

1
.nvim.lua Symbolic link

1
.vscode Symbolic link

7702
MODULE.bazel.lock generated

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show more