edge-ai

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

Olares: An Open-Source Sovereign Cloud OS for Local AI

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment

MicroPython for K210 RISC-V, let's play with edge AI easier

Deep learning gateway on Raspberry Pi and other edge devices

Real-time portrait segmentation for mobile devices

speech to text benchmark framework

AI-in-a-Box leverages the expertise of Microsoft across the globe to develop and provide AI and ML solutions to the technical community. Our intent is to present a curated collection of solution accelerators that can help engineers establish their AI/ML environments and solutions rapidly and with minimal friction.

On-Device Training Under 256KB Memory [NeurIPS'22]

A curated list of awesome edge computing, including Frameworks, Simulators, Tools, etc.

The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

💜 The best free Telegram bot for ChatGPT, Microsoft Copilot (aka Bing AI / Sidney / EdgeGPT), Microsoft Copilot Designer (aka BingImageCreator), Gemini and Groq with stream writing, requests with images, multiple languages, admin control, data logging and more!

Dual Edge TPU Adapter to use it on a system with single PCIe port on m.2 A/B/E/M slot

Easily create AI projects with Python on edge device

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

Jetson Nano-based smart camera system that measures crowd face mask usage in real-time.

Neuralet is an open-source platform for edge deep learning models on edge TPU, Jetson Nano, and more.

Server for Physical AI Inference