fattorib Goto Github PK

cudasoftmax

Softmax CUDA kernel :)

fattorib.github.io

Website

flashy_linear_attention

Flash linear attention kernels in Triton

flax-resnets

CIFAR10 ResNets implemented in JAX+Flax

fundamental-domain

Code to generate a section of the fundamental domain for the action of the special linear group on the space of (integral) binary cubic forms. As it stands, the code is quite inefficient. In the future I hope to optimize it.

fusedswiglu

Fused SwiGLU Triton kernels

geometricdeeplearning

Introductory Geometric Deep Learning Presentation from September 2021

infogan-jax

InfoGAN in Jax with small Gradio app

jaxvae

Variational Autoencoder in JAX

leaguematchscraper

Code to scrape League of Legends matches using the Riot Games API.

little-gpt

GPT* - Training faster small transformers using ALiBi, Parallel Residual Connections and more!

lm-evaluation-harness

monte-carlo-fractal-dimensionality

Efficient algorithm using random sampling to calculate the dimension of many basic fractals. Implemented algorithm in Python.

picograd

picograd - Fully connected neural networks in Python

python-bpe

I wrote Byte-Pair encoding but its 600x slower than 🤗

python-unigram

Unigram tokenization algorithm in Python

repvgg-cifar10

RepVGG models specifically for CIFAR10 and CIFAR 100. Based on RepVGG: Making VGG-style ConvNets Great Again (Ding et. al)

resnets-cifar10

PyTorch implementation of the CIFAR10 ResNets, based on Deep Residual Learning for Image Recognition (He et al.)

stochasticdepthnets

PyTorch implementation of ResNet110 as described in Deep Networks with Stochastic Depth (Huang et al.)

transformer_shmap

Tensor Parallelism with JAX + Shard Map

tritonformer

Differentiable transformer in Triton, matching the performance of PyTorch + cuDNN/cuBLAS

wtf-wikipedia-python

raw wikipedia XML to LM_Dataformat in under 4 hours

zero-transformer

Two implementations of ZeRO-1 optimizer sharding in JAX