Repository navigation

vision-language-transformer

Website
Wikipedia

LAVIS - A One-stop Library for Language-Vision Intelligence

深度学习 deep-learning-library image-captioning salesforce vision-and-language vision-framework vision-language-pretraining vision-language-transformer visual-question-anwsering multimodal-datasets multimodal-deep-learning

Jupyter Notebook

10841

1059

9 个月前

IDEA-Research / GroundingDINO

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

object-detection open-world open-world-detection vision-language vision-language-transformer

Python

8720

889

1 年前

salesforce / BLIP

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

vision-language vision-and-language-pre-training image-text-retrieval image-captioning visual-question-answering vision-language-transformer

Jupyter Notebook

5440

712

1 年前

AlibabaResearch / AdvancedLiterateMachinery

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

C++

1759

197

4 个月前

henghuiding / ReLA

[CVPR2023 Highlight] GRES: Generalized Referring Expression Segmentation

multimodal-learning referring-expression-comprehension referring-expression-segmentation vision-language-transformer cvpr2023

Python

690

2 年前

shenyunhang / APE

[CVPR 2024] Aligning and Prompting Everything All at Once for Universal Visual Perception

image-segmentation object-detection open-world referring-expression-comprehension vision-language-transformer

Python

583

1 年前

henghuiding / Vision-Language-Transformer

[ICCV2021 & TPAMI2023] Vision-Language Transformer and Query Generation for Referring Segmentation

vision-language transformer Tensorflow Keras iccv2021 vision-language-transformer

Python

358

4 年前

haoliuhl / instructrl

Instruction Following Agents with Multimodal Transforemrs

flax instruction-following instructions jax 机器学习 reinforcement-learning transformer vision-language-transformer

Python

3 年前

sMamooler / CLIP_Explainability

code for studying OpenAI's CLIP explainability

机器视觉机器学习 model-explainability vision-language-transformer

Jupyter Notebook

4 年前

yiren-jian / BLIText

[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

multimodal-deep-learning vision-language-pretraining vision-language-transformer

Python

2 年前

akusayudodograu / Agentic-RAG-Story-Generation-with-Multimodal-GenAI

Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling

agentic-ai agentic-rag agentic-workflow generative-ai multimodal multimodal-deep-learning multimodal-large-language-models multimodal-learning vision-language vision-language-model vision-language-transformer

1 小时前

unitaryai / VTC

VTC: Improving Video-Text Retrieval with User Comments

multimodal-deep-learning video-understanding vision-language-pretraining vision-language-transformer comments

Python

8 天前

deepmancer / vlm-toolbox

Vision-Language Models Toolbox: Your all-in-one solution for multimodal research and experimentation

clip 深度学习 deep-learning-library multimodal-datasets multimodal-deep-learning multimodal-learning prompt-tuning vision-and-language vision-framework vision-language-transformer zero-shot-classification PyTorch transformers

Jupyter Notebook

6 个月前

ThomasVonWu / Awesome-VLMs-Strawberry

A collection of VLMs papers, blogs, and projects, with a focus on VLMs in Autonomous Driving and related reasoning techniques.

大语言模型 multimodal-learning vision-language-transformer vlms

9 个月前

marialymperaiou / knowledge-enhanced-multimodal-learning

A list of research papers on knowledge-enhanced multimodal learning

image-text-matching image-text-retrieval knowledge-graph multimodal-deep-learning multimodal-retrieval vision-and-language vision-and-language-pre-training vision-language-transformer visual-commonsense-reasoning visual-question-answering multi-task-learning

3 年前

fork123aniket / Multi-Round-VLM-powered-Multimodal-Conversational-AI-Navigation-Bot

Streamlit App Combining Vision, Language, and Audio AI Models

conversational-agent conversational-ai internvl multimodal multimodal-deep-learning multimodal-large-language-models multimodal-learning vision-language vision-language-model vision-language-transformer generative-ai

Python

7 个月前

fork123aniket / Agentic-RAG-Story-Generation-with-Multimodal-GenAI

Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling

Python

7 个月前

jhakrraman / Deep-Learning-NYU-Spring-2025

This reporsitory contains all the Homeworks, and Projects from the Deep Learning Course by Prof. Chinmay Hegde, in Spring 2025, at NYU.

机器视觉 computer-vision-algorithms 深度学习 deep-learning-models 大语言模型自然语言处理 natural-language-understanding reinforcement-learning reinforcement-learning-algorithms vision-language-model vision-language-transformer

3 个月前

PrateekJannu / Vision-GPT

Coding a Multi-Modal vision model like GPT-4o from scratch, inspired by @hkproj and PaliGemma

gemini Google 大语言模型 large-language-models Open Source transformer-architecture transformer-models vision-language-model vision-language-transformer vision-transformer 人工智能 gpt-4o 机器学习

Python

9 个月前

aurooj / VLM_SS

Mini-batch selective sampling for knowledge adaption of VLMs for mammography.

Medical imaging multimodal-learning multimodal-retrieval vision-and-language vision-language-transformer

Jupyter Notebook

10 个月前