Repository for 3D tracking of a tracking die

Preparation

1. The cad files for 3D printing is located in the cad-folder.
2. Film a video where the tracking device is used in surgery.

Installation

1. Install python3 on your system.
2. Install the dependencies (either run install.sh) or manually:

python3 -m pip install numpy opencv-python opencv-contrib-python argparse transforms3d scikit-spatial pyquaternion matplotlib

Camera calibration

1. Print the CalibrationGrid.png
2. Take at least 10 sharp images of the grid from different angles.
3. Save the images in a separate folder.
4. Run the calibration script calibrate.py.

python3 -f <path_to_image_folder> -o <yaml_file> [-sz size_of_one_square_in_calibration_grid]

5. This will save the camera calibration in the yaml-file.

Analyze the video

1. Run the log_in_video.py script to track the 3D printed die in the video. Arguments to the scripts are:

• the video from the surgery: -v, --video
• the yaml-file with the camera calibration: -cd, --camera_data
• the name of an output video where the identified markers will be showed: -o, --out_video
• a log-file for storing the position data: -l, --log
• optionally, the minimum number of markers that must be visible: -mn, --min

python3 log_in_video.py -v <path_to_video> -cd <camera_calibration_yaml_file> -o <output_video> -l <log_file> -m <minimum required sides>

2. Analyze the data, e.g., plot the position. The plot_pos.py script will draw the positions recorded in an interval of sz frames.

python3 plot_pos.py -l <log> -sz <interval_size>

Calculate the F1 score

Calculates the F1 score for visibility.

1. Generate data for the visibility score by running the gen_data_f1.py. The scripts takes up to four arguments:

• The video from the surgery: -v, --video
• The yaml file with camera calibration parameters: -cd, --camera_data
• An output video for saving the tracking result: -o, --out_video
• A log file to save the result: -l , --log
• An optional argument to show or hide the video while it is processing, default is to show: -s, --show

python3 gen_data_f1.py -v <path_to_video> -cd <camera_data_yaml> -o <output_video> -l <log> -s False

2. The script label.py is used to label images as either correctly or incorrectly SEEN or UNSEEN. A random 1000 images (with a fixed seed) are selected from the video. The program iterates through the images one-by-one displaying each for 1 s before moving to the next. First the frames where the die is visible is displayed with the detected Aruco markers drawn, the last frame is shown for 10 s before moving on to image frames where the die is not detected.

The program can be controlled with the follwoing keys:

• space: pauses and starts the images slide show
• B: when the program is paused, pressing B goes back one frame at the time
• N: when the program is paused, pressing N goes forward to the next frame one frame at the time
• W: labels the image as wrong.

The workflow is as following: a. Start the program b. When an incorrect classified image is shown, press SPACE. c. Navigate back and forth with B and N to find select the incorrectly classified image. d. Press W to label the image as incorrect. e. Press SPACE to continue.

The script has three arguemtns:

• The marked video from step 1: -v, --video
• The log file from step 1: -l, --log
• The output file for the labels: -r, --labels

python3 label.py -v <marked_video> -l <log> -r <result_labels>

The first two images are labeled correctly (seen and unseen), the last is incorrecly labeled as not seen and should be relabeled by the user (with W).

2. The script calc_f1_score.py calculates the precision, recall and F1 score from the result label file (the only argument).

python3 calc_f1_score.py result_label_file