inference-engine

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step

Rule engine implementation in Golang

OneDiff: An out-of-the-box acceleration library for diffusion models.

Large-scale LLM inference engine

FeatherCNN is a high performance inference engine for convolutional neural networks.

Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.

A highly optimized LLM inference acceleration engine for Llama and its variants.

Adlik: Toolkit for Accelerating Deep Learning Inference

🔥 (yolov3 yolov4 yolov5 unet ...)A mini pytorch inference framework which inspired from darknet.

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Python Computer Vision & Video Analytics Framework With Batteries Included

A library for high performance deep learning inference on NVIDIA GPUs.

A common base representation of python source code for pylint and other projects

High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.

A Machine Learning System for Data Enrichment.

A rule engine written in Ruby.

PyKnow: Expert Systems for Python

Julia package for automated Bayesian inference on a factor graph with reactive message passing

校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。