nisarahamedk / swarm-jax Goto Github PK
View Code? Open in Web Editor NEWThis project forked from kingoflolz/swarm-jax
Swarm training framework using Haiku + JAX + Ray for layer parallel transformer language models on unreliable, heterogeneous nodes