Brief Review — MedAug: Contrastive Learning Leveraging Patient Metadata Improves Representations for Chest X-Ray Interpretation

MedAug, Leverages Patient Metadata to Select Positive Pairs

  • MedAug is proposed to select positive pairs, by leveraging patient metadata to improve representations, for medical image self-supervised learning.
  • This is a paper by Prof. Andrew Ng’s research group.


  1. MedAug
  2. Results

1. MedAug

1.1. Dataset

  • CheXpert, is a large collection of de-identified chest X-ray images.
  • The dataset consists of 224,316 images from 65,240 patients labeled for the presence or absence of 14 radiological observations. These images are used for pretraining, and random samples of 1% of these images for fine-tuning.
  • The test set consists of 500 additional labeled images from 500 studies not included in the training set.

1.2. Self-Supervised Learning

  • ResNet-18 is used as backbone.
  • MoCo v2 is used for self-supervised learning.
  • Given an input image x, encoder g, and a set of augmentations T, most contrastive learning algorithms involve minimizing the InfoNCE loss (CPC / CPCv1):
  • The negative pairs (~x1, zi), are pairs of augmentations of different images.

1.3. MedAug

Selecting positive pairs for contrastive learning with patient metadata
  • Formally, patient metadata is used to obtain an enhanced augmentation set dependent on x as follows:
  • where Sc(x) is the set of all images satisfying some predefined criteria c in relation to the properties of x. The criteria for using the metadata could be informed by clinical insights about the downstream task of interest.

2. Results

  • The best result is obtained when using S_same study_all_lateralities(x), the set of images from the same patient and same study as that of x, regardless of laterality, in respective gains of 0.029 (3.4%) and 0.021 (2.4%) in AUC for the linear model and end-to-end model.
  • Authors tried to pick up the hard negative pairs using metadata, but no improvement is observed.
  • Authors also provides other ablation experimental results, please feel free to read the paper.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store