multihead-attention

list of efficient attention modules

Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.

A Faster Pytorch Implementation of Multi-Head Self-Attention

Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

여러가지 유명한 신경망 모델들을 제공합니다. (DCGAN, VAE, Resnet 등등)

Implementation of "Attention is All You Need" paper

Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in this repository. Particularly, every paper is built on a transformer.

Chatbot using Tensorflow (Model is transformer) ko

Joint text classification on multiple levels with multiple labels, using a multi-head attention mechanism to wire two prediction tasks together.

Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.

An experimental project for autonomous vehicle driving perception with steering angle prediction and semantic segmentation using a combination of UNet, attention and transformers.

This repository contains the code for the paper "Attention Is All You Need" i.e The Transformer.

The implementation of transformer as presented in the paper "Attention is all you need" from scratch.

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

EvoLingua: A Scalable Mixture-of-Experts Language Model Framework

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

Very simple implementation of GPT architecture using PyTorch and Jupyter.

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.

Testing the Reproducibility of the paper: MixSeq. Under the assumption that macroscopic time series follow a mixture distribution, they hypothesise that lower variance of constituting latent mixture components could improve the estimation of macroscopic time series.