GitHub

031-supported-models-multimodal-language-models.md

summary This document explains how to deploy and configure multimodal language models in SGLang, covering server setup, video input integration, and performance optimization flags. These models accept ...
IEEE

DM-FNet: Unified Multimodal Medical Image Fusion via Diffusion Process-Trained Encoder-Decoder

Abstract: Multimodal medical image fusion (MMIF) extracts the most meaningful information from multiple source images, enabling a more comprehensive and accurate diagnosis. Achieving high-quality ...
IEEE

VATS: Visual–Audio Multitask Transformer With Specialty Audio Encoder for Multimodal Deepfake Detection in CPSS

Abstract: Detecting multimodal deepfakes has become a pressing concern due to the rising sophistication of generative techniques capable of creating highly convincing visual-speech synchronized ...
GitHub

Unifying Computational Imaging via a Multi-Modal Foundation Model

Flux by Black Forest Labs — we use their pretrained diffusion model and autoencoder. JointDiT by Microsoft Research Asia — we adopt and extend their RGBD autoencoder infrastructure. This code was ...
marktechpost

How to Build a Vision-Guided Web AI Agent with MolmoWeb-4B Using Multimodal Reasoning and Action Prediction

In this tutorial, we explore MolmoWeb, Ai2’s open multimodal web agent that understands and interacts with websites directly from screenshots, without relying on HTML or DOM parsing. We set up the ...