distributed-inference

prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters

Distributed Inference for mlx LLm

Source code of the paper "Private Collaborative Edge Inference via Over-the-Air Computation".

Official impl. of ACM MM paper "Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds". A distributed inference model for pedestrian attribute recognition with re-ID in an MEC-enabled camera monitoring system. Jointly training of pedestrian attribute recognition and Re-ID.