Repository navigation

local-inference

Website
Wikipedia

SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving for Local Deployment

large-language-models llama 大语言模型 llm-inference local-inference

C++

8178

429

2 个月前

efeslab / fiddler

[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

大语言模型 llm-inference mixtral-8x7b mixture-of-experts local-inference

Python

208

5 个月前

BorjaOteroFerreira / IALab-Suite

Tool for test diferents large language models without code.

flask-api 大语言模型 llm-inference chat-application inference-api large-language-models llama2 API llamacpp mixtral-8x7b local-inference

JavaScript

3 个月前

tinyBigGAMES / JetInfero

Local LLM Inference Library

llama-cpp win64 pascal local-inference Library

Pascal

3 个月前

yas-sim / openvino-llm-chatbot-rag

LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).

聊天机器人大语言模型 openvino rag retrieval-augmented-generation edge-computing llama2 intel langchain 自然语言处理 offline cloud-free local-inference huggingface

Python

1 年前

nazago / rag-llm-local

script which performs RAG and use a local LLM for Q&A

langchain-python 大语言模型 local-inference ollama rag

Python

7 个月前

nazago / meeting-minutes-generator

Script which takes a .wav audio file, performs speech-to-text using OpenAI/Whisper, and then, using Llama3, summarization and action point from the transcript generated

meeting-minutes speech-to-text summarization Whisper langchain-python llm-inference local-inference ollama

Python

7 个月前