Repository navigation

vision-language-transformer

Website
Wikipedia

LAVIS - A One-stop Library for Language-Vision Intelligence

深度学习 deep-learning-library image-captioning salesforce vision-and-language vision-framework vision-language-pretraining vision-language-transformer visual-question-anwsering multimodal-datasets multimodal-deep-learning

Jupyter Notebook

10470

1020

5 个月前

IDEA-Research / GroundingDINO

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

object-detection open-world open-world-detection vision-language vision-language-transformer

Python

7888

791

8 个月前

salesforce / BLIP

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

vision-language vision-and-language-pre-training image-text-retrieval image-captioning visual-question-answering vision-language-transformer

Jupyter Notebook

5196

681

8 个月前

AlibabaResearch / AdvancedLiterateMachinery

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

C++

1688

191

11 天前

henghuiding / ReLA

[CVPR2023 Highlight] GRES: Generalized Referring Expression Segmentation

multimodal-learning referring-expression-comprehension referring-expression-segmentation vision-language-transformer cvpr2023

Python

693

2 年前

shenyunhang / APE

[CVPR 2024] Aligning and Prompting Everything All at Once for Universal Visual Perception

image-segmentation object-detection open-world referring-expression-comprehension vision-language-transformer

Python

563

1 年前

henghuiding / Vision-Language-Transformer

[ICCV2021 & TPAMI2023] Vision-Language Transformer and Query Generation for Referring Segmentation

vision-language transformer Tensorflow Keras iccv2021 vision-language-transformer

Python

353

3 年前

haoliuhl / instructrl

Instruction Following Agents with Multimodal Transforemrs

flax instruction-following instructions jax 机器学习 reinforcement-learning transformer vision-language-transformer

Python

2 年前

sMamooler / CLIP_Explainability

code for studying OpenAI's CLIP explainability

机器视觉机器学习 model-explainability vision-language-transformer

Jupyter Notebook

3 年前

yiren-jian / BLIText

[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

multimodal-deep-learning vision-language-pretraining vision-language-transformer

Python

1 年前

unitaryai / VTC

VTC: Improving Video-Text Retrieval with User Comments

multimodal-deep-learning video-understanding vision-language-pretraining vision-language-transformer comments

Python

20 天前

ThomasVonWu / Awesome-VLMs-Strawberry

A collection of VLMs papers, blogs, and projects, with a focus on VLMs in Autonomous Driving and related reasoning techniques.

大语言模型 multimodal-learning vision-language-transformer

5 个月前

akusayudodograu / Agentic-RAG-Story-Generation-with-Multimodal-GenAI

Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling

agentic-ai agentic-rag agentic-workflow generative-ai multimodal multimodal-deep-learning multimodal-large-language-models multimodal-learning vision-language vision-language-model vision-language-transformer

37 分钟前

deepmancer / vlm-toolbox

Vision-Language Models Toolbox: Your all-in-one solution for multimodal research and experimentation

clip 深度学习 deep-learning-library multimodal-datasets multimodal-deep-learning multimodal-learning prompt-tuning vision-and-language vision-framework vision-language-transformer zero-shot-classification PyTorch transformers

Jupyter Notebook

2 个月前

marialymperaiou / knowledge-enhanced-multimodal-learning

A list of research papers on knowledge-enhanced multimodal learning

image-text-matching image-text-retrieval knowledge-graph multimodal-deep-learning vision-and-language vision-and-language-pre-training vision-language-transformer visual-commonsense-reasoning visual-question-answering multi-task-learning

2 年前

fork123aniket / Multi-Round-VLM-powered-Multimodal-Conversational-AI-Navigation-Bot

Streamlit App Combining Vision, Language, and Audio AI Models

conversational-agent conversational-ai internvl multimodal multimodal-deep-learning multimodal-large-language-models multimodal-learning vision-language vision-language-model vision-language-transformer generative-ai

Python

3 个月前

fork123aniket / Agentic-RAG-Story-Generation-with-Multimodal-GenAI

Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling

Python

3 个月前

PrateekJannu / Vision-GPT

Coding a Multi-Modal vision model like GPT-4o from scratch, inspired by @hkproj and PaliGemma

gemini Google large-language-model large-language-models Open Source transformer-architecture transformer-models vision-language-model vision-language-transformer vision-transformer 人工智能 gpt-4o 大语言模型机器学习

Python

5 个月前

aurooj / VLM_SS

Mini-batch selective sampling for knowledge adaption of VLMs for mammography.

Medical imaging multimodal-learning vision-and-language vision-language-transformer

Jupyter Notebook

6 个月前

atharva-naik / MMML-TermProject-VizWiz-VQA-Challenge

VizWiz Challenge Term Project for Multi Modal Machine Learning @ CMU (11777)

carnegie-mellon-university 机器视觉图像处理自然语言处理 Open Source open-source-project OpenCV PyTorch question-answering vision-language vision-language-transformer visual-question-answering

Python

2 年前