Repository navigation

visual-language-learning

Website
Wikipedia

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

gpt-4 聊天机器人 ChatGPT llama multimodal llava foundation-models instruction-tuning multi-modality visual-language-learning llama-2 llama2 vision-language-model

Python

22263

2445

8 个月前

NExT-GPT / NExT-GPT

Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model

ChatGPT foundation-models gpt-4 instruction-tuning large-language-models 大语言模型 multi-modal-chatgpt multimodal visual-language-learning mllm

Python

3485

352

6 个月前

EvolvingLMMs-Lab / Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

gpt-4 visual-language-learning artificial-inteligence 深度学习 foundation-models multi-modality 机器学习 ChatGPT instruction-tuning large-scale-models embodied-ai

Python

3248

213

1 年前

InternLM / InternLM-XComposer

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

ChatGPT visual-language-learning multi-modality foundation gpt-4 instruction-tuning mllm multimodal vision-language-model language-model large-language-model large-vision-language-model 大语言模型 vision-transformer gpt

Python

2813

172

3 个月前

xiaoachen98 / Open-LLaVA-NeXT

An open-source implementation for training LLaVA-NeXT.

聊天机器人 ChatGPT gpt-4 gpt4o large-multimodal-models llama llama3 llava multi-modality multimodal vision-language-model visual-language-learning

Python

392

6 个月前

RLHF-V / RLHF-V

[CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

聊天机器人 gpt-4 llama multi-modality multimodal visual-language-learning

Python

276

7 个月前

mlpc-ucsd / BLIVA

(AAAI 2024) BLIVA: A Simple Multimodal LLM for Better Handling of Text-rich Visual Questions

blip2 聊天机器人 instruction-tuning llama 大语言模型 multimodal visual-language-learning lora

Python

257

1 年前

thomas-yanxin / KarmaVLM

🧘🏻‍♂️KarmaVLM (相生)：A family of high efficiency and powerful visual language model.

llama2 llava qwen2 vlm vision-language-model visual-language-learning

Python

1 年前

AdrianBZG / llama-multimodal-vqa

Multimodal Instruction Tuning for Llama 3

聊天机器人 ChatGPT gpt-4 huggingface instruction-tuning language-models llama llama2 llama3 multimodal visual-language-learning visual-question-answering vqa

Python

1 年前

xinyanghuang7 / Basic-Visual-Language-Model

Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖

large-language-models visual-language-learning visual-language-models

Python

10 个月前

Skyline-9 / Shotluck-Holmes

[ACM MMGR '24] 🔍 Shotluck Holmes: A family of small-scale LLVMs for shot-level video understanding

大语言模型自然语言处理 Python video-captioning multi-modality vision-language-model visual-language-learning

Python

6 个月前

MuhammadAliS / CLIP

PyTorch implementation of OpenAI's CLIP model for image classification, visual search, and visual question answering (VQA).

深度神经网络 huggingface pytorch-implementation transformers visual-language-learning visual-question-answering

Jupyter Notebook

7 个月前

ashleykleynhans / llava-docker

Docker image for LLaVA: Large Language and Vision Assistant

人工智能聊天机器人 ChatGPT Docker Docker Image foundation-models gpt-4 instruction-tuning llama llama-2 llama2 llava 大语言模型 multimodal runpod vision-language-model visual-language-learning

Shell

10 个月前

ecoxial2007 / EffVideoQA

Efficient Video Question Answering

机器视觉 video-question-answering visual-language-learning

Python

2 年前