NIC AU Research: Multimodal Deep Learning Framework Predicts Breast Cancer Receptor Status

Architecture of the proposed multimodal deep learning framework

NIC AU is proud to share a new research manuscript co-authored by Binh Minh Tran (NIC AU) alongside collaborators from the University of Sydney, Hanoi Medical University, Hanoi University of Industry, Macquarie University and UNSW: “A Multimodal Deep Learning Framework for Breast Cancer Receptor Status Prediction.”

Why It Matters

Accurately determining estrogen receptor (ER), progesterone receptor (PR) and HER2 status is essential for choosing the right breast cancer treatment — but conventional immunohistochemistry and in-situ hybridisation assays are slow, costly and prone to interpretive variability.

What the Model Does

The team’s framework fuses three complementary sources of evidence — clinical variables, whole-slide histopathology images, and tumour gene expression — through modality-specific encoders and a shared self-attention fusion module, jointly predicting all three receptor statuses.

Trained on 937 matched TCGA-BRCA cases and externally validated on the independent CPTAC-BRCA cohort, the model achieved AUCs of 0.954 (ER), 0.868 (PR) and 0.848 (HER2) on the development cohort, generalising strongly to 0.961, 0.926 and 0.907 respectively on external validation — while attention maps and gene-level interpretability analysis kept the model’s reasoning transparent to clinicians.

Read more on the Resources page or contact admin@nicau.org for the full manuscript.