PulseHeart-AutoML

Article

https://www.mdpi.com/2075-4418/14/13/1439

Instructions for interested researchers

General Flow Chart

(a) data curation; (b) upload the study managed dataset to Google Cloud Storage; (c) balanced data sets S, M and L, the size of which depends on the minority class (rEF, mEF and npEF respectively); (d): video classifications to test; (e): create a per-model annotation dataset and upload the corresponding CSV files (one for the train and one for the test videos); (f) specify the model to be trained; (g) run the model for training; (h) analyze the test set; (i) human panel reassessment of false positives and false negatives.

Get access to the EchoNet dataset

• Make a fork of the GitHub directory
• Download locally the forked directory
• Open the GitHub link echonet-dynamic and there open the link to access the dataset at Standford AIMI (log in or sign up if you don't have an acccount).
• Download the 7.04 GB EchoNet-Dynamic.zip file.
• Unzip the file in your local directory
• Create a folder "curated"
• Extract the 3064 AVI files of our study dataset in "curated" using the notebook transfer-from-source.ipynb

Watch individual videos

The all-data cvs file describes the 3064 videos used in this study.

Three examples of correctly classified video of high quality:

Frames from videos pertaining to the study dataset. End-diastolic frames are shown on the upper part of the slide and end-systolic frames below: (a) 0X2753C50A8B05D7D5.avi, label pEF, EF 58.3% by Simpson method; (b) 0X2F3141F00A232601.avi, label mEF, EF 42.1% by Simpson method; (c) 0X41563E2CC2230C0E.avi, label rEF, EF 21.9% by Simpson method.

Three examples of misclassified video of low quality

Frames from misclassified videos. End-diastolic frames are shown on the upper part of the slide and end-systolic frames below: (a) 0X41ECEC7AAEEFD0E6.avi, label pEF, EF 57.3% by Simpson method, pEF by unanimous panel, misclassified as rEF; (b) 0X7923B6B4614AF456.avi, label mEF, EF 49.1% by Simpson method, mEF for four raters and rEF for one rater, misclassified as rEF; (c) 0X3503A92D7637451.avi, label rEF, EF 31.8% by Simpson method, rEF by unanimous panel, misclassified as nrEF.

Use the original train/split of our nine experimentations

Use the corresponding annotation dataset CSV files.

These files include our bucket name in the video file paths to be changed to your's, (for instance by a Replace_All operation in Excel or a modification using Python code).

Launch experimentations, either replicating those of this research or training your own

Create a storage bucket and upload there the "curated" folder of study video files

Refer to the following section of this tutorial.

Setting up your project

Chose an appropriate name, like "echocg-video-classification"

Creating a video classification dataset and importing videos

At the step 2a: choose the name your Dataset in relation with the train and test annotation datasets that you will use: for example L-3-classes

At step 3:

• Select "Upload import files from your computer"
• Choose Data split "Manual"
• Upload the training csv file (example: training_L_3_classes) specifying "training"
• Clicking ADD ANOTHER FILE
• Upload the testing csv file (example: test_L_3_classes) specifying "testing"
• Indicate your video file path, for instance: gs://bucket name/curated
• Click Continue

Training a video classification model

• Review the experiment videos
• Click TRAIN NEW MODEL
• Choose a training method (AutoML in this example)
• Click "Define your model"
• Make sure that data split is manual 75/25
• Edit Model Name if necessary (for example: ternary_classification_large_set)
• Click "START TRAINING"