Repository navigation
nanogpt
- Website
- Wikipedia
Run GPT model on the browser with WebGPU. An implementation of GPT inference in less than ~1500 lines of vanilla Javascript.
This repository contains the collection of explorative notebooks pure in python and in the language that we, humans can read. Have tried to compile all lectures from the Andrej Karpathy's 💎 playlist on Neural Networks - which we will end up with building GPT.
Symbolic Music NLP Artificial Intelligence Toolkit
Code repository for the paper "Traveling Words: A Geometric Interpretation of Transformers"
Repository nanoGPT from @karpathy accelerated with JAX/Flax! The simplest, fastest repository for training/finetuning medium-sized GPTs.
Arabic Nano GPT Trained on Arabic Wikipedia Dataset from Wikimedia
JAX LLM playground
The simple repository for training/finetuning medium-sized GPTs.
Fast multi-instrumental music transformer with 4k sequence length, pentagram full-range MIDI notes encoding, notes counters and outro tokens
We trained nanoGPT from scratch to have emotional support response to the user
A compact Transformer model for character-level generation with Mixture-of-Experts, Rotary Attention, and F-gram contextual augmentation.
This version of Muon converges slightly faster than the Muon from modded-nanogpt in some cases. The change is RMS-Norm after orthogonalization over the first dimension of the weight matrix (last dimension of nn.Linear). The code here assumes you store the weights like nn.Linear i.e. used like x = x @ W.T.
Training karpathy/nanoGPT on A. Puhskin's poems text corpus
