Setup

Step 1: System requirements

Label3D is supported for Matlab 2019b and 2020a across Windows, Mac, and Linux.

Some convenience functions require ffmpeg for reading and writing videos, although this is not required for using Label3D.

Step 2: Camera calibration

Label3D is a tool for multi-view 3D keypoint labeling, and thus requires both intrinsic and extrinsic camera calibration. There are many publicly available tools for multi-view camera calibration and you should feel free to use whichever you like. However, we recommend using the calibration method outlined in dannce, which takes advantage of Matlab's camera calibration suite.

A brief overview of camera calibration basics can be found here.

Step 3: Frame synchronization

Depending on the details of your image acquisition, it may be necessary to synchronize the frames captured from each camera to a global temporal reference frame. Although this step is not necessary to run Label3D, unsynchronized frames could hinder labeling accuracy and should be avoided. We recommend using the synchronization protocol described in dannce.

Step 4: Defining a keypoint skeleton

Label3D/skeletons includes predefined skeletons for a number of common laboratory animals including mice, rats, birds and marmosets. To build your own custom skeleton, you need to define a set of keypoints to label as well as any connections between them. This skeleton is parameterized by an undirected graph, which can be defined in three steps.

1. Define a set of keypoints in any reasonable order and name each one in a cell array of strings.
2. Define a set of connections between keypoints, denoting undirected edges or segments.
3. Define the color of each segment from step 2 in a nSegment x 3(4) of RGB(A) values.

The example below shows how to define a skeleton between three points to form a triangle with red, green, and blue edges and a transparency of .5.

# Build a triangle.
skeleton.joint_names = {'Front','Middle','End'};
skeleton.joints_idx = [1 2; 2 3; 3 1];
skeleton.color = [1 0 0 .5; 0 1 0 .5; 0 0 1 .5];

Step 5: Putting it all together

Once you have calibrated camera parameters, synchronized frames, and a skeleton, simply pass the parameters, videos, and skeleton to Label3D to begin labeling.

labelGui = Label3D(params, videos, skeleton);

A demonstration of this formatting can be found in Label3D/example.m.