0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

Compare commits

...

51 commits

Author SHA1 Message Date
Hugo Mano
6b6152a291 cleanup from tpu 2025-08-08 12:42:00 +00:00
Hugo Mano
fe0b994273 fix norm weight sharding 2025-08-06 15:25:46 +00:00
Hugo Mano
bf9369abcc restore writeShardingRepresentation 2025-08-06 15:20:02 +00:00
Hugo Mano
8fc2a7e939 another attempt 2025-08-06 15:14:33 +00:00
Hugo Mano
df2e6eb884 rollback some stuffs 2025-08-06 15:06:49 +00:00
Hugo Mano
ac6fd43f08 tpu sharding fix 2025-08-06 14:06:21 +00:00
Hugo Mano
ca98ce0bea llama tpu 2025-08-06 13:22:44 +00:00
Hugo Mano
8c7afd6b08 physical mesh 2025-08-06 12:34:02 +00:00
Hugo Mano
717b0c405e 100 toks/s 2025-08-05 18:57:31 +00:00
Hugo Mano
f0f11930e2 8b okay 70b ko 2025-08-05 18:20:48 +00:00
Hugo Mano
5294d0b77c cleanup 2025-08-04 15:38:38 +00:00
Hugo Mano
0acedf2ba3 restore uninitialized 2025-07-31 14:44:35 +00:00
Hugo Mano
adc28fa325 remove check 2025-07-31 13:57:26 +00:00
Hugo Mano
acfea5c6ae fixes for tpu 2025-07-31 13:54:32 +00:00
Hugo Mano
c687600280 device description 2025-07-31 11:32:23 +00:00
Hugo Mano
6e885435e6 fix flags 2025-07-31 09:39:01 +00:00
Hugo Mano
b3f84f3179 back to something working 2025-07-31 09:29:32 +00:00
Hugo Mano
77e68344d5 add flags for nccl 2025-07-31 09:29:32 +00:00
Hugo Mano
7d3a2d0441 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
07c58ff185 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
beaf19824b try 2025-07-31 09:29:32 +00:00
Hugo Mano
585b51db44 gather right dim 2025-07-31 09:29:32 +00:00
Hugo Mano
933445a76b fix 70B 2025-07-31 09:29:32 +00:00
Hugo Mano
3d179606ad wip 2025-07-31 09:29:32 +00:00
Hugo Mano
a6b5e550e3 sharding output 2025-07-31 09:29:32 +00:00
Hugo Mano
4897398a82 new sharding 2025-07-31 09:29:32 +00:00
Hugo Mano
a6aa94c6e3 rollback 2025-07-31 09:29:32 +00:00
Hugo Mano
050b98d0de wip 2025-07-31 09:29:32 +00:00
Hugo Mano
c38ea4553c attempt 2025-07-31 09:29:32 +00:00
Hugo Mano
4751c3d666 use allGather 2025-07-31 09:29:32 +00:00
Hugo Mano
c5a87e0ba4 just test nccl 2025-07-31 09:29:32 +00:00
Hugo Mano
78aecbcb30 setup pjrt profiler 2025-07-31 09:29:32 +00:00
Hugo Mano
4874ad497e working llama 2025-07-31 09:29:32 +00:00
Hugo Mano
8b571abbcd collectives 2025-07-31 09:29:32 +00:00
Hugo Mano
214f78cb43 restore zml call 2025-07-31 09:29:32 +00:00
Hugo Mano
3dfca85029 deactivate zml call for fusion 2025-07-31 09:29:32 +00:00
Hugo Mano
354c8cfe0a simpler 2025-07-31 09:29:32 +00:00
Hugo Mano
f027119199 transpose k 2025-07-31 09:29:32 +00:00
Hugo Mano
966da15cc2 correct mlp sharding 2025-07-31 09:29:32 +00:00
Hugo Mano
044d7c4702 another version 2025-07-31 09:29:32 +00:00
Hugo Mano
229a1c3159 attempt 2025-07-31 09:29:32 +00:00
Hugo Mano
8380847390 wip 2025-07-31 09:29:32 +00:00
Hugo Mano
980587e011 restore zml.call 2025-07-31 09:29:32 +00:00
Hugo Mano
7bfc85baa5 fix MLP 2025-07-31 09:29:32 +00:00
Hugo Mano
e9022db647 ops allReduce 2025-07-31 09:29:32 +00:00
Hugo Mano
2d4202e5bb oupsi 2025-07-31 09:29:32 +00:00
Hugo Mano
1c913e4662 megatron 2025-07-31 09:29:32 +00:00
Hugo Mano
695f85216e add logs 2025-07-31 09:29:32 +00:00
Hugo Mano
27bf92c866 reduce / scatter 2025-07-31 09:29:32 +00:00
Hugo Mano
69e1d19205 rebase 2025-07-31 09:29:32 +00:00
Hugo Mano
10f86e93f1 explo partitioning 2025-07-31 09:29:32 +00:00
30 changed files with 4124 additions and 768 deletions