llm-training

Find secrets with Gitleaks 🔑

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

Low-code framework for building custom LLMs, neural networks, and other AI models

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, 17+ clouds, or on-prem).

Efficient Triton Kernels for LLM Training

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

Code examples and resources for DBRX, a large language model developed by Databricks

MoBA: Mixture of Block Attention for Long-Context LLMs

DLRover: An Automatic Distributed Deep Learning System

Nvidia GPU exporter for prometheus using nvidia-smi binary

A PyTorch Native LLM Training Framework

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in-Group Policy Optimization for LLM Agent Training"

LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

LLM (Large Language Model) FineTuning

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

irresponsible innovation. Try now at https://chat.dev/

Repo for fine-tuning Casual LLMs

The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.