gguf

Local AI API Platform

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.

Effortlessly run LLM backends, APIs, frontends, and services with one command.

LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG

动手学Ollama，CPU玩转大模型部署，在线阅读地址：https://datawhalechina.github.io/handy-ollama/

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

Go manage your Ollama models

Interface for OuteTTS models.

An open source DevOps tool for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI artifact.

Practical Llama 3 inference in Java

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.

Go library for embedded vector search and semantic embeddings using llama.cpp

Run AI ✨ assistant locally! with simple API for Node.js 🚀

GGUF implementation in C as a library and a tools CLI program

Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!

LM inference server implementation based on *.cpp.

The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

GPU-accelerated Llama3.java inference in pure Java using TornadoVM.