How to Bridge the Gap Between Modalities: Survey on Multimodal Large Language Model
Sign Up to like & getrecommendations! Published in 2025 at "IEEE Transactions on Knowledge and Data Engineering"
DOI: 10.1109/tkde.2025.3527978
Abstract: We explore Multimodal Large Language Models (MLLMs), which integrate LLMs like GPT-4 to handle multimodal data, including text, images, audio, and more. MLLMs demonstrate capabilities such as generating image captions and answering image-based questions, bridging… read more here.
Keywords: large language; multimodal large; language; modality alignment ... See more keywords
Complete 3D Relationships Extraction Modality Alignment Network for 3D Dense Captioning.
Sign Up to like & getrecommendations! Published in 2023 at "IEEE transactions on visualization and computer graphics"
DOI: 10.1109/tvcg.2023.3279204
Abstract: 3D dense captioning aims to semantically describe each object detected in a 3D scene, which plays a significant role in 3D scene understanding. Previous works lack a complete definition of 3D spatial relationships and the… read more here.
Keywords: modality alignment; relationship; complete relationships; extraction modality ... See more keywords
Cross Task Modality Alignment Network for Sketch Face Recognition
Sign Up to like & getrecommendations! Published in 2022 at "Frontiers in Neurorobotics"
DOI: 10.3389/fnbot.2022.823484
Abstract: The task of sketch face recognition refers to matching cross-modality facial images from sketch to photo, which is widely applied in the criminal investigation area. Existing works aim to bridge the cross-modality gap by inter-modality… read more here.
Keywords: modality alignment; cross task; modality; task ... See more keywords