This is an unedited manuscript accepted for publication and provided as an Article in Press for early access at the author’s request. The article will undergo copyediting, typesetting, and galley proof review before final publication. Please be aware that errors may be identified during production that could affect the content. All legal disclaimers of the journal apply.
Aditya Patel,
- , Department of Information Technology, Bansal Institute of Engineering and Technology, Uttar Pradesh, India
Abstract
Getting reliable labeled data for medical image analysis is genuinely difficult. Radiologists, pathologists, and other specialist clinicians are expensive, slow, and not universally available — particularly in low-resource clinical environments across developing nations. Unlike natural image benchmarks such as ImageNet, which provide millions of meticulously labeled training examples, most publicly accessible medical imaging repositories offer only a few hundred to a few thousand annotated scans. Supervised deep learning models trained on such limited annotations are prone to severe overfitting and generalize poorly to new patient populations, imaging devices, or clinical protocols. Self-supervised learning (SSL) offers a principled solution to this problem by enabling neural networks to learn rich, transferable visual representations from large volumes of unlabeled data, without requiring any human annotation during the representation learning phase. By constructing pretext tasks that generate training signals directly from the raw image data — through augmentation invariance, masked patch reconstruction, or self-distillation — SSL methods acquire structured feature representations that can subsequently be fine-tuned effectively using only a small labeled set. This paper presents a comprehensive evaluation of four prominent SSL frameworks — SimCLR, MoCo v2, Masked Autoencoders (MAE), and DINO — applied to early disease detection across three clinically significant medical imaging modalities: chest radiographs, retinal fundus photographs, and histopathological tissue slides. We conduct systematic benchmarking under low-label conditions with label fractions of 1%, 5%, and 10% of the available training data. Beyond benchmarking, we introduce MedSSL, a novel hybrid SSL framework specifically designed for medical imaging that integrates contrastive instance discrimination with masked image reconstruction through a modality-adaptive weighted joint loss. Experimental results across all three datasets demonstrate that MedSSL consistently outperforms every single-objective SSL baseline in all nine evaluation conditions (three datasets × three label fractions), achieving a mean AUROC improvement of 4.7% over the best competing SSL method. At the extreme low-label setting of 1%, MedSSL outperforms supervised training from random initialization by over 16 AUROC points. The incorporation of domain-specific augmentation pipelines — tailored separately for chest radiography, retinal imaging, and histopathology — provides an additional 2.1% AUROC gain over generic augmentation strategies. These findings collectively establish SSL as a viable, practical, and scalable pre-training paradigm for clinical AI systems that are constrained by limited annotation budgets.
Keywords: self-supervised learning, medical image analysis, contrastive learning, masked autoencoders, low-label learning, disease detection, ViT, MedSSL
References
Journal of Image Processing & Pattern Recognition Progress
| Volume | 13 | |
| 03 | ||
| Received | 23/07/2026 | |
| Accepted | 10/09/2026 | |
| Published | 08/10/2026 | |
| Publication Time | 77 Days |