Skip to content

fmenat/multiviewRS-models

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 

Repository files navigation

Remote Sensing-based Multi-view Learning (MVL) Models that perform Data Fusion

List created based on our work Common Practices and Taxonomy in Deep Multiview Fusion for Remote Sensing Applications


Foundational models can be found at TorchGeo or HuggingFace

List of MVL models proposed to learn from remote sensing (RS) multi-view data. 📡 🌎 📡

Name Reference Description Code
MVL Mena et al. 2025 Multiple fusion strategies with temporal data https://github.com/fmenat/optimal-multiview-crop-classifier
SSL4EO-S12 Wang et al. 2024 Self-supervised model https://github.com/zhu-xlab/SSL4EO-S12
EarthGPT Zhang et al. 2024 Feature-level fusion with transformer layers https://github.com/wivizhang/EarthGPT
ContextFormer Benson et al. 2024 Feature-level fusion with transformer https://github.com/vitusbenson/greenearthnet
SEnSeIv2 Francis 2024 Sensor-invariant model https://github.com/aliFrancis/SEnSeIv2
OmniSat Astruc et al. 2024 Feature-level fusion with transformer layers and pre-training https://github.com/gastruc/OmniSat
MMEarth Nedungadi et al. 2024 Single-view input to predict another views (pretext task) https://github.com/vishalned/MMEarth-train
MambaDifussion Du et al. 2024 Feature level fusion with Mamba over different layers (skip connection) including diffusion models https://github.com/WenliangDu/MambaDiffusion
FusionMamba Peng et al. 2024 Dense fusion with Mamba layers https://github.com/PSRben/FusionMamba
SatViT Fuller et al. Input-level fusion with ViT (with self-supervised training) https://github.com/antofuller/SatViT
ELECTS Russwurm et al. 2023 Input-level fusion with LSTM. https://github.com/marccoru/elects
MV CNN Ferrari et al. 2023 Multiple fusion strategies with 2D CNN (encoder-decoder) https://github.com/felferrari/deforestation-from-data-fusion
AFCF3D-Net Ye et al. 2023 Input-level fusion with 3D CNN. https://github.com/wm-Githuber/AFCF3D-Net
UnCRtainTS Ebel et al 2023 Input fusion with 2D CNN and attention. https://github.com/PatrickTUM/UnCRtainTS
MFT Roy et al. 2023 Feature-level fusion with transformer modules (one source - LIDAR- is used as a query over the main source - optical) https://github.com/AnkurDeria/MFT
PRESTO Tseng et al. 2023 Input-level fusion with transformer modules (self-supervised pretraining) https://github.com/nasaharvest/presto
Cross-HL Roy et al. 2023 Feature-level fusion with directed attention in transformer layers https://github.com/AtriSukul1508/Cross-HL
SCT Fusion Hoffman et al. 2023 Dense fusion with tranformer layers and class tokens in all https://git.tu-berlin.de/rsim/sct-fusion
MMST-ViT Lin et al. 2023 Feature-level fusion with transformer layers https://github.com/fudong03/MMST-ViT
DiffusionSat Khanna et al. 2023 Multi-modal diffusion generative model https://github.com/samar-khanna/DiffusionSat
AMM-FuseNet Ma et al. 2022 Feature-level fusion with 2D CNN and attention. https://github.com/oktaykarakus/ReSIF/tree/main/AMM-FuseNet
MCANet Li et al. 2022 Dense fusion with 2D CNN and cross attention. https://github.com/yisun98/SOLC
ChangeFormer Bandara et al. 2022 Dense fusion with transformer and attention. https://github.com/wgcban/ChangeFormer
CMAFF Qingyun et al. 2022 Dense fusion with 2D CNN and cross attention. https://github.com/DocF/CMAFF
OmbriaNet Drakonakis et al. 2022 Feature fusion with 2D CNN and skip-connections https://github.com/geodrak/OMBRIA
DCSA-Net Wang et al. 2022 Hybrid fusion with 2D CNN and attention. https://github.com/Julia90/DCSA-Net
Siamese U-Net Cummings et al. 2022 Dense fusion with 2D CNN and skip-connections https://github.com/solcummings/earthvision2021-weakly-supervised
AM$^3$Net Wang et al. 2022 Feature-level fusion with 2D CNN and cross attention. https://github.com/Cimy-wang/AM3Net_Multimodal_Data_Fusion
MAHiDFNet Wang et al. 2022 Dense feature fusion with 2D CNN. https://github.com/SYFYN0317/-MAHiDFNet
EndNet Hong et al. 2022 Feature-level fusion with 2D CNN and view-reconstruction. https://github.com/danfenghong/IEEE_GRSL_EndNet
SE$^2$Net Fang et al. 2022 Feature-level fusion with 2D CNN. https://github.com/likyoo/Multimodal-Remote-Sensing-Toolkit
MV CNN Lu et al. 2022 Feature-level fusion with 2D CNN and adaptive attention. https://github.com/GeoX-Lab/UnifiedDL-UFZ-extraction
IP-CNN Zhang et al. 2022 Feature-level fusion with 2D CNN and view-reconstruction. https://github.com/HelloPiPi/IP-CNN-code
ASF2N Gao et al. 2022 Feature-level fusion with 2D CNN and attention. https://github.com/zhonghaocheng/ELSEVIER_IJAEOG_AS2F2N << Empty code
SEnSeI Francis et al. 2022 Sensor invariant model based on 2D CNN https://github.com/aliFrancis/SEnSeI
MV NN Danilevicz et al. 2021 Feature-level fusion with tabular NN and 2D CNN. https://github.com/mdanilevicz/maize_early_yield_prediction
CFCNN He et al. 2021 Feature-level fusion with 2D and 1D CNN. https://github.com/SysuHe/MultiSourceData_CFCNN
S2FL Hong et al. 2021 Feature-level fusion with feature constraints. https://github.com/danfenghong/ISPRS_S2FL
CMGFNet Hosseinpour et al. 2022 Dense fusion with 2D CNN and gated attention. https://github.com/hamidreza2015/CMGFNet-Building_Extraction
MDL-RS Hong et al. 2021 Multiple fusion strategies with NN. https://github.com/danfenghong/IEEE_TGRS_MDL-RS
MV PSE-TAE Ofori-Ampofo et al. 2021 Multiple fusion strategies with PSE-TAE. https://github.com/ellaampy/CropTypeMapping
CCR-Net Wu et al. 2021 Feature-level fusion with 2D CNN and cross-view reconstruction. https://github.com/danfenghong/IEEE_TGRS_CCR-Net
LFMC from SAR Rao et al Input-level fusion with LSTM https://github.com/kkraoj/lfmc_from_sar
FusAtNet Mohla et al. 2020 Feature-level fusion with 2D CNN and cross attention. https://github.com/ShivamP1993/FusAtNet
HRWN Zhao et al. 2020 Input-level fusion with 2D CNN and pixel graph constraints. https://github.com/xudongzhao461/HRWN
UNet-CLSTM Rustowicz et al. 2019 Decision-level fusion with 2D CNN and convolutional-LSTM https://github.com/roserustowicz/crop-type-mapping
Multi3Net Rudner et al. 2019 Feature-level fusion with 2D CNN. https://github.com/FrontierDevelopmentLab/multi3net
V-FuseNet Audebert et al. 2018 Dense fusion with 2D CNN and central model. https://github.com/nshaud/DeepNetsForEO
MV CNN Xu et al. 2018 Feature-level fusion with 2D CNN. https://github.com/Hsuxu/Two-branch-CNN-Multisource-RS-classification << Not available

Feel free to ask me to update some content!

List of MVL models robust to missing data and views

Name Reference Description Code
MDiCo Mena et al. 2025 Decision-level co-learning to handle single-view prediction https://github.com/fmenat/MDiCo
DSensD+ Mena et al. 2025 Decision-level fusion with sensor dropout and mutual distillation https://github.com/fmenat/dsensdp
FCoM-* Mena et al. 2025 Feature-level fusion with all combinations of missing views and dynamic fusion https://github.com/fmenat/CoM-views
OOD Fusion Gawlikowski et al. 2023 Input-level, Feature-level, and Decision-level fusion with CNN and weighted average aggregation https://github.com/JakobCode/OOD_DataFusion

Feel free to ask me to update some content!


Some abbreviations

Abbrevation name
CNN convolutional neural network
LSTM long-short term memory
NN neural network
PSE-TAE pixel set encoder - temporal attention encoder

Releases

No releases published

Packages

 
 
 

Contributors