Revisiting masked auto-encoders for ECG-language representation learning

• Main Authors: PHAM, Hung Manh, SAEED, Aaqib, MA, Dong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Programming Languages and Compilers
• Online Access: https://ink.library.smu.edu.sg/sis_research/9938; https://ink.library.smu.edu.sg/context/sis_research/article/10938/viewcontent/42_Revisiting_Masked_Auto_Enco.pdf
• Institution: Singapore Management University

We propose C-MELT, a novel framework for multimodal self-supervised learning of Electrocardiogram (ECG) and text encoders. C-MELT pre-trains a contrastive-enhanced masked auto-encoder architecture using ECG-text paired data. It exploits the generative strengths with improved discriminative capabilities to enable robust cross-modal alignment. This is accomplished through a carefully designed model, loss functions, and a novel negative sampling strategy. Our preliminary experiments demonstrate significant performance improvements with up to 12% in downstream cardiac arrhythmia classification and patient identification tasks. Our findings demonstrate C-MELT's capacity to extract rich, clinically relevant features from ECG-text pairs, paving the way for more accurate and efficient cardiac diagnoses in real-world healthcare settings.