DeepInsight-Net: a CBAM-enhanced ResNet50 framework with focal loss for robust cervical cancer classification on multi-center datasets

Abstract

Background Cervical cancer remains one of the leading causes of gynecological mortality worldwide, largely due to the limitations of manual cytological screening, which is time-consuming and susceptible to inter-observer variability. Although deep learning has demonstrated strong potential for automating cervical cytopathology, existing Convolutional Neural Network (CNN) models are hindered by two critical challenges: spatial irrelevance, where diagnostically meaningful nuclear regions are overshadowed by background artifacts such as blood and mucus, and severe class imbalance, where the dominance of normal cells impedes the accurate learning of rare dysplastic patterns. Methods To address these limitations, we propose DeepInsight-Net, a robust three-stage deep learning framework for cervical cell classification. The core architecture integrates Convolutional Block Attention Modules (CBAM) into a ResNet50 backbone to enhance spatial and channel-wise feature discrimination, enabling the network to selectively emphasize nuclear and nuclear-cytoplasmic boundary regions while suppressing irrelevant background noise. To further mitigate class imbalance, the conventional cross-entropy loss is replaced with Focal Loss, which dynamically down-weights easily classified samples and prioritizes hard, misclassified instances during training. Results Extensive experiments conducted on the benchmark SiPaKMeddataset demonstrate that DeepInsight-Net achieves a state-of-the-art classification accuracy of 99.63%, outperforming 15 competitive deep learning models, including EfficientNet-B6 and DenseNet169. Moreover, cross-dataset generalization experiments on an independent Liquid-Based Cytology (LBC) dataset yield an accuracy of 98.62%, confirming the robustness and domain adaptability of the proposed framework. Visual interpretability analyses using Grad-CAM and t-SNE reveal that the model consistently focuses on biologically relevant cellular regions, supporting the reliability of its predictions.