Repository navigation

multimodal-learning

Website
Wikipedia

Reading list for research topics in multimodal machine learning

multimodal-learning 机器学习 representation-learning 自然语言处理机器视觉 speech-processing Robotics healthcare reading-list 深度学习 reinforcement-learning

6584

885

1 年前

mlfoundations / open_flamingo

An open-source framework for training large multimodal models.

机器视觉深度学习 in-context-learning language-model multimodal-learning PyTorch flamingo

Python

3996

310

1 年前

KaiyangZhou / CoOp

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

foundation-models multimodal-learning prompt-learning

Python

2035

228

1 年前

Eurus-Holmes / Awesome-Multimodal-Research

A curated list of Multimodal Related Research.

Awesome Lists multimodal-research multimodal-learning multimodal

Python

1368

150

2 年前

AILab-CVC / UniRepLKNet

[CVPR 2024 & TPAMI 2025] UniRepLKNet

architecture 人工智能 convolutional-neural-networks 深度学习 multimodal-learning

Python

1010

10 天前

PreferredAI / cornac

A Comparative Framework for Multimodal Recommender Systems

recommender-system recommendation-algorithms recommendation-engine matrix-factorization collaborative-filtering multimodal-learning recommendation-system multimodality

Python

976

156

4 个月前

ArrowLuo / CLIP4Clip

An official implementation for "CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval"

multimodal-learning multimodality multimodal search ranking retrieval-model retrieval activitynet clip

Python

976

133

1 年前

DmitryRyumin / ICCV-2023-Papers

ICCV 2023 Papers: Discover cutting-edge research from ICCV 2023, the leading computer vision conference. Stay updated on the latest in computer vision and deep learning, with code included. ⭐ support visual intelligence development!

Python

954

1 年前

declare-lab / multimodal-deep-learning

This repository contains various models targetting multimodal representation learning, multimodal fusion for downstream tasks such as multimodal sentiment analysis.

multimodal-deep-learning multimodal-learning multimodal-interactions

OpenEdge ABL

857

156

2 年前

HuaizhengZhang / Awsome-Deep-Learning-for-Video-Analysis

Papers, code and datasets about deep learning and multi-modal learning for video analysis

深度学习 video-analysis Bukkit multimodal-learning 机器学习 video-classification

810

173

4 年前

richard-peng-xia / awesome-multimodal-in-medical-imaging

A collection of resources on applications of multi-modal learning in medical imaging.

Medical imaging multimodal-deep-learning multimodal-learning visual-question-answering large-language-models large-multimodal-models multimodal-large-language-models

806

16 天前

henghuiding / ReLA

[CVPR2023 Highlight] GRES: Generalized Referring Expression Segmentation

multimodal-learning referring-expression-comprehension referring-expression-segmentation vision-language-transformer cvpr2023

Python

690

2 年前

georgian-io / Multimodal-Toolkit

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

huggingface-transformers transformer 自然语言处理 tabular-data multimodal-learning

Python

605

10 个月前

pliang279 / MultiBench

[NeurIPS 2021] Multiscale Benchmarks for Multimodal Representation Learning

机器学习 multimodal-learning Robotics 自然语言处理机器视觉深度学习 healthcare representation-learning speech-processing

HTML

568

2 年前

sangminwoo / awesome-vision-and-language

A curated list of awesome vision and language resources (still under construction... stay tuned!)

Awesome Lists vision-and-language multimodal-learning

545

10 个月前

henghuiding / MeViS

[ICCV 2023] MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions

multimodal-learning referring-expression-comprehension referring-expression-segmentation referring-video-object-segmentation video-understanding

Python

518

9 天前

subho406 / OmniNet

Official Pytorch implementation of "OmniNet: A unified architecture for multi-modal multi-task learning" | Authors: Subhojeet Pramanik, Priyanka Agrawal, Aman Hussain

机器学习深度学习神经网络人工智能 transformer 自然语言处理 image-captioning video-recognition multitask-learning multimodal-learning

Python

513

5 年前

microsoft / XPretrain

Multi-modality pre-training

multimodal-learning pre-training multimedia 机器视觉自然语言处理

Python

502

1 年前

DmitryRyumin / ICASSP-2023-24-Papers

ICASSP 2023-2024 Papers: A complete collection of influential and exciting research papers from the ICASSP 2023-24 conferences. Explore the latest advancements in acoustics, speech and signal processing. Code included. Star the repository to support the advancement of audio and signal processing!

asr denoising domain-adaptation face-recognition language-modeling self-supervised-learning semantic-segmentation signal-processing speech-recognition vad generative-models image-generation music-generation multimodal-learning

Python

499

3 个月前

MMMU-Benchmark / MMMU

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

机器视觉深度学习深度神经网络 evaluation foundation-models large-language-models large-multimodal-models 大语言模型机器学习 multimodal multimodal-deep-learning multimodal-learning multimodality 自然语言处理 question-answering STEM visual-question-answering

Python

482

3 个月前