Brief Review — PhysioNet/CinC Challenge 2016: An Open Access Database for the Evaluation of Heart Sound Algorithms

PhysioNet/CinC Challenge 2016 Dataset for Normal/Abnormal Classification

Sik-Ho Tsang
6 min readNov 5, 2023

An open access database for the evaluation of heart sound algorithms
PhysioNet/CinC Challenge 2016, by Numerous Organizations
2016 IOP Physiol. Meas., Over 570 Citations (Sik-Ho Tsang @ Medium)

Heart Sound Classification
2013 [PASCAL] 2018 [RNN Variants] [SVM, DNN, kNN] 2023 [2LSTM+3FC, 3CONV+2FC] [NRC-Net]
==== My Other Paper Readings Are Also Over Here ====

  • This paper describes a public heart sound database, assembled for an international competition, the PhysioNet/Computing in Cardiology (CinC) Challenge 2016. The archive comprises 9 different heart sound databases sourced from multiple research groups around the world.
  • It includes 2,435 heart sound recordings in total collected from 1,297 healthy subjects and patients with a variety of conditions, including heart valve disease and coronary artery disease.

Outline

  1. Preliminaries
  2. PhysioNet/CinC Challenge 2016 Dataset

1. Preliminaries

Phonocardiogram (PCG)

An audio recording (or graphical) time series representation of the resultant sounds, transduced at the chest surface is known as a heart sound recording or phonocardiogram (PCG).

  • 4 locations are most often used to listen to and transduce the heart sounds:
  1. Aortic area: centred at the second right intercostal space.
  2. Pulmonic area: in the second intercostal space along the left sternal border.
  3. Tricuspid area: in the fourth intercostal space along the left sternal edge.
  4. Mitral area: at the cardiac apex, in the fifth intercostal space on the midclavicular line.
  • Fundamental heart sounds (FHSs) usually include the first (S1) and second (S2) heart sounds.
  • There are also other audible sounds, such as the third heart sound (S3), the fourth heart sound (S4), systolic ejection click (EC), mid systolic click (MC), the diastolic sound or opening snap (OS), as well as heart murmurs caused by turbulent, high velocity flow of blood.
Frequency Range

S1 for 10 — 140 Hz; S2 for 10 — 200 Hz and S3 & S4 for 20 — 70 Hz.

  • Murmurs tend to manifest diverse frequency ranges and depending on their nature they can be as high as 600 Hz.
  • Respiration usually has a frequency range of 200 — 700 Hz.
Common 3 Steps
  • Pre-processing, segmentation and classification are the 3 common steps for automatic heart sound classification.

2. PhysioNet/CinC Challenge 2016 Dataset

2.1. Problems of Prior Datasets

  • Prior to the PhysioNet/CinC Challenge 2016, there were only 3 public heart sound databases available:
  1. The Michigan Heart Sound and Murmur database (UMHS): it includes only 23 heart sound recordings with a total of time length of 1496.8 s.
  2. The PASCAL database: it comprises 176 recordings for heart sound segmentation and 656 recordings for heart sound classification, limited time length from 1 s to 30 s, and a limited frequency range below 195 Hz which removes many of the useful heart sound components for clinical diagnosis.
  3. The Cardiac Auscultation of Heart Murmurs database (eGeneralMedical): It includes 64 recordings. It is not open and requires payment for access.

In the PhysioNet/CinC Challenge 2016, a large collection of heart sound recordings was obtained from different real world clinical and non-clinical environments (such as in home visits). The data include not only clean heart sounds but also very noisy recordings. The data were recorded from both normal subjects and pathological patients, and from both children and adults. The data were also recorded from different locations.

9 Datasets in PhysioNet/CinC Challenge 2016
  • There are a total of 9 heart sound databases collected independently by 7 different research teams from 7 countries and 3 continents, over a period of more than a decade. As a result, the hardware, recording locations, data quality and patient types differ substantially.
  • The PhysioNet/CinC Challenge 2016 was focused on adult heart sounds, this SUFHSDB dataset was excluded only from the challenge; but has been included in the online database.
  • (Please read the paper directly for the details of each dataset.)

2.2. Problem of Prior Approaches

  • The 2016 PhysioNet/CinC Challenge aims to encourage the development of algorithms to classify heart sound recordings collected from a variety of clinical or non-clinical environments (such as in home visits). The practical aim is to identify, from a single short recording (10 60s).
  • However, many of the investigations are flawed because:
  1. The studies were marred by poor methodology.
  2. The studies did not clearly describe the database used.
  3. The studies tended to use hand picked clean data.
  4. Failure to use enough or a variety of heart sound recordings.
  5. Failure to post the data (and any code to process the data) publicly.
Good and Poor Signal Quality
  • In this Challenge, we focused only on the accurate classification of normal and abnormal heart sound recordings, particularly in the context of real world (extremely noisy) recordings with low signal quality.
  • Some recordings are as ‘unsure’. Classifications for the heart sound recordings were therefore three level: normal (do not refer), abnormal (refer for further diagnostics) and unsure (too noisy to make a decision; retake the recording).

2.3. Data Split

Data Split
  • The dataset is sourced from 7 contributing research groups (with the exception of the SUFHSDB since it was from fetal and maternal heart sounds), were used in the Challenge, resulting in 8 independent heart sound databases.
  • 4 of the databases were divided into both training and test sets with a 70:30 training test split. The other 4 databases were exclusively assigned to either training or test set.
  • Thus, the Challenge training set includes data from 6 databases, containing a total of 3,153 heart sound recordings from 764 subjects/patients, lasting from 5s to just over 120s.
  • The Challenge test set also included data from 6 databases, containing a total of 1,277 heart sound recordings from 308 subjects/patients, lasting from 6s to 104s.
  • The total number of recordings was 4,430 and is different from the reported number of 2,435 in Table 1. It is because DLUTHSDB recordings are generally longer than 100s and each recording was segmented into several relatively short recordings.
  • All recordings were resampled to 2,000 Hz using an anti alias filter and provided as .wav format.
Example Which (a) Segmented from Springer’s Segmentation Algorithm and (b) After Hand-Correction
  • 20.7% of the recordings in the training set and 15.3% of the recordings in the test set required hand correction.
Dataset Before and After Balancing
  • Since both training and test sets are unbalanced, a balanced heart sound database from training set was selected.
  • (There are segmentation and classification reviews by the paper, please feel free to read if interested.)

--

--

Sik-Ho Tsang

PhD, Researcher. I share what I learn. :) Linktree: https://linktr.ee/shtsang for Twitter, LinkedIn, etc.