localllama

Model swapping for llama.cpp (or any local OpenAPI compatible server)

✨ Kubectl plugin to create manifests with LLMs

The easiest way to use Ollama in .NET

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.

Social and customizable AI writing assistant! ✍️

🏗️ Build, fine-tune, and run generative models locally!

A local and uncensored AI entity.

LLM RAG Application with Cross-Encoders Re-ranking for YouTube video 🎥

Secure Flutter desktop app connecting Auth0 authentication with local Ollama AI models via encrypted tunneling. Access your private AI instances remotely while keeping data on your hardware.

Run gguf LLM models in Latest Version TextGen-webui and koboldcpp

Full featured demo application for OllamaSharp

Use your open source local model from the terminal

📚 LocalLLaMA Archive — Community-powered static archive for r/LocalLLaMA

Copilot hack for running local copilot without auth and proxying

A set of guides for fully contained, daemonless, secure methods of storing and using LLMs locally on a mounted SSD. Uses Podman, supports AMD with Vulkan, uses llama.cpp, llamafiles, ollama w/ Openhands, Zendriver

A chat interface in Streamlit for LLMs using Ollama.

Local AI Search assistant web or CLI for ollama and llama.cpp. Lightweight and easy to run, providing a Perplexity-like experience.