quantization

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

Faster Whisper transcription with CTranslate2

[🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ news qbot-mini: https://github.com/Charmve/iQuant

Accessible large language models via k-bit quantization for PyTorch.

Lossy PNG compressor — pngquant command based on libimagequant library

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Neural Network Distiller by Intel AI Lab: a Python package for neural network compression research. https://intellabs.github.io/distiller

Fast inference engine for Transformer models

Sparsity-aware deep learning inference runtime for CPUs

[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Quantized Attention achieves speedup of 2-5x and 3-11x compared to FlashAttention and xformers, without lossing end-to-end metrics across language, image, and video models.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

PyTorch native quantization and sparsity for training and inference