Repository navigation

distributed-inference

Website
Wikipedia

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving

gpu large-large-models CUDA PyTorch llm-inference jit attention Nvidia distributed-inference moe

Cuda

3586

445

16 小时前

gpustack / gpustack

Simple, scalable AI model deployment on GPU clusters

ascend CUDA deepseek distributed-inference genai inference llama llamacpp 大语言模型 maas metal openai qwen rocm vllm mindie llm-inference llm-serving local-ai heterogeneous-cluster

Python

3366

344

11 小时前

Lizonghang / prima.cpp

prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters

distributed-ai llm-inference on-device-llms llama-cpp distributed-inference

C++

997

1 个月前

mzbac / mlx_sharding

Distributed Inference for mlx LLm

MLX distributed-inference

Python

1 年前

ADT109119 / llamacpp-distributed-inference

一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。

distributed-inference gguf llamacpp 大语言模型 llm-inference 远程过程调用 (RPC)distributed-llm

JavaScript

24 天前

ipc-lab / collaborative-inference-oac

Source code of the paper "Private Collaborative Edge Inference via Over-the-Air Computation".

differential-privacy distributed-inference ensemble-learning 机器学习

Python

7 个月前

JiangkaiWu / Attribute_Reid

Official impl. of ACM MM paper "Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds". A distributed inference model for pedestrian attribute recognition with re-ID in an MEC-enabled camera monitoring system. Jointly training of pedestrian attribute recognition and Re-ID.

distributed-inference edge-computing re-identification

Python

5 年前