Repository navigation

model-inference-service

Website
Wikipedia

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

model-serving mlops llmops generative-ai llm-inference model-inference-service inference-platform 深度学习 llm-serving 机器学习 Python multimodal ml-engineering 大语言模型 ai-inference

Python

7995

869

1 天前

bentoml / CLIP-API-service

CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

ai-applications clip cloud-native mlops model-inference model-inference-service model-serving

Jupyter Notebook

22 天前

bentoml / transformers-nlp-service

Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more

大语言模型 mlops model-deployment model-inference-service model-serving 自然语言处理 transformer llmops

Python

1 年前

ksm26 / Efficiently-Serving-LLMs

Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and Low Rank Adapters (LoRA), and gain hands-on experience with Predibase’s LoRAX framework inference server.

batch-processing inference-optimization machine-learning-operations model-inference-service model-serving text-generation

Jupyter Notebook

1 年前

spirabr / SPIRA-serving-predictor-v1

SPIRA Serving Predictor v1 by @daitamae and @vitorguidi

mlops model-inference-service model-serving Python

Python

2 年前