Federated learning framework for medical image analysis with perspective-aware contrastive and mixture of experts

Abstract

Medical image analysis faces persistent challenges due to the distributed data, limited annotations, and variations in imaging modalities, acquisition protocols, and patient demographics. Centralized deep learning approaches compromise data privacy, while Federated Learning (FL) enables decentralized model training without sharing raw data. However, conventional FL frameworks struggle with non-IID distributions and heterogeneous clinical environments, limiting their generalization and stability. We propose FedPAC-ME, a novel Federated Learning Framework for Medical Image Analysis that integrates Perspective-Aware Contrastive Learning with a Mixture of Experts (MoE) architecture to address heterogeneity and data imbalance. The framework introduces Multi-Perspective Augmentation (MPA) to emulate diverse clinical views, and a Perspective-Aware contrastive Module (PACM) that aligns representations across modalities and clients. Additionally, a Mixture of Experts routing layer dynamically allocates specialized experts to client-specific data distributions, enhancing adaptability and collaboration across sites. A Perspective-Aware Contrastive Loss (PACL) further enforces cross-view consistency during local training while maintaining global coherence across institutions. Extensive experiments on the BraTS2020 multi-institutional brain tumor segmentation dataset demonstrate that FedPAC-ME achieves 98.80% accuracy, surpassing state-of-the-art FL baselines by over 2.5%. These results confirm the framework's effectiveness in improving feature alignment, generalization, and privacy preservation under diverse clinical conditions.