Repository navigation

smoothquant

Website
Wikipedia

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models gptq int8

Python

2474

280

1 天前

ModelTC / LightCompress

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

部署大语言模型 pruning quantization 工具 benchmark evaluation large-language-models internlm2 smoothquant post-training-quantization mixtral vllm deepseek-r1 deepseek-v3

Python

544

7 天前