Trouble importing BORIS annotations

[GKNM995]

Hi,

I am trying to import BORIS annotations for a single video with two animals, but I keep getting the error below:

Any idea how I can fix this?

I can send you the csv with the behavior annotations if seeing it would be helpful.

Thank you!

[sronilsson]

Thanks for reporting @GKNM995! To troubleshoot, would you mind sharing the BORIS file, and I can see if I can replicate ?

Also, which version of SimBA are you using (pip show simba-uw-tf-dev) ?

[sronilsson]

PS. If you could also share the SimBA project_config.ini at the same time that would help too

[GKNM995]

Hi,

Thank you for getting back to me so quickly. Here's the info you requested:

Version: 1.95.7
Python 3.10

I have attached the BORIS csv.
2DPI_2573L_2572R.csv

Couldn't attach the config file for some reason, so I have pasted the contents here:

[General settings]
project_path = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\project_folder
project_name = OF_classifier_training
workflow_file_type = csv
animal_no = 2
os_system = Windows

[SML settings]
model_dir = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models
model_path_1 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Body_groom.sav
model_path_2 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Face_groom.sav
model_path_3 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Locomotion.sav
model_path_4 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Rear.sav
model_path_5 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Sniff.sav
model_path_6 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Stationary.sav
model_path_7 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Tail_rattle.sav
model_path_8 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Turn.sav
no_targets = 8
target_name_1 = Stationary
target_name_2 = Locomotion
target_name_3 = Turn
target_name_4 = Sniff
target_name_5 = Tail_rattle
target_name_6 = Rear
target_name_7 = Face_groom
target_name_8 = Body_groom

[threshold_settings]
threshold_1 = None
threshold_2 = None
threshold_3 = None
threshold_4 = None
threshold_5 = None
threshold_6 = None
threshold_7 = None
threshold_8 = None
bp_threshold_sklearn = 0.0

[Minimum_bout_lengths]
min_bout_1 = None
min_bout_2 = None
min_bout_3 = None
min_bout_4 = None
min_bout_5 = None
min_bout_6 = None
min_bout_7 = None
min_bout_8 = None

[Frame settings]
distance_mm = 450

[Line plot settings]

[Path plot settings]

[ROI settings]

[Directionality settings]

[process movements]

[create ensemble settings]
pose_estimation_body_parts = 16
classifier = None
train_test_size = 0.2
under_sample_setting = None
under_sample_ratio = None
over_sample_setting = None
over_sample_ratio = None
rf_n_estimators = 2000
rf_min_sample_leaf = 1
rf_max_features = sqrt
rf_n_jobs = -1
rf_criterion = entropy
generate_rf_model_meta_data_file = None
generate_example_decision_tree = None
generate_example_decision_tree_fancy = None
generate_features_importance_log = None
generate_features_importance_bar_graph = None
compute_feature_permutation_importance = None
generate_sklearn_learning_curves = None
generate_precision_recall_curves = None
n_feature_importance_bars = None
learning_curve_k_splits = None
learningcurve_shuffle_data_splits = None

[Multi animal IDs]
id_list = Mouse_1,Mouse_2

[Outlier settings]
movement_criterion = 1
location_criterion = 1.5
mean_or_median = mean
movement_bodypart1_mouse_1 = Nose_1
movement_bodypart2_mouse_1 = Tail_base_1
location_bodypart1_mouse_1 = Nose_1
location_bodypart2_mouse_1 = Tail_base_1
movement_bodypart1_mouse_2 = Nose_2
movement_bodypart2_mouse_2 = Tail_base_2
location_bodypart1_mouse_2 = Nose_2
location_bodypart2_mouse_2 = Tail_base_2

[sronilsson]

Hi @GKNM995 - yes, I can see what is happening.

Your BORIS file is a little different from what SimBA expects. See THIS file as an example for what SimBA expects.

The difference seems to be that in your file, every behavior START and STOP is represented by a single row (which I haven't seen before, but it is great, easy to read, and ensures that every behavior has a start and stop event). However, the BORIS file I link to above (which SimBA expects) has a row for every behavior START event, and a row for every behavior STOP event.

When SimBA finds no START and no STOP in the Behavior type column, it thinks that no behaviors exist in your file (there are only STATE which SimBA doesn't know what to do with).

There are two possible ways of solving it: either (i) is it possible to go back to BORIS and easily save the data in the format that SimBA expects? Or, (ii) if that is not possible, we can write a bit of code that transforms the BORIS filetype you have into the BORIS filestype SimBA expects before feeding it into SimBA?

[GKNM995]

I see. That's helpful. I will get the data in the desired format from BORIS and let you know how it goes (Will report back on Tuesday). Thank you!

[GKNM995]

Hi,

I tried the format you suggested but I get the same error message as before. BORIS csvs attached.
2DPI_LR12PUMP5MG_2572.csv
2DPI_LR12PUMP5MG_2573.csv

[GKNM995]

Hi! I saw your comments on the related issue. Gonna try out those fixes. Thank you!

[GKNM995]

Hi,

I tried the fixes you outlined and got much further than before. However, it is taking a suspiciously long time to append the BORIS data so I think it might be stuck here:

Here's the error message in my command prompt:

Thank you for your help!

[sronilsson]

Thanks for reporting @GKNM995! Let me take a look at this - which version of SimBA are you running when that error happens?

[sronilsson]

I can see one potential issue with the BORIS file:

When SimBA reads in the BORIS annotations in 2DPI_LR12PUMP5MG_2572.csv, it looks to see what FPS is associated with the videos annotated in that file.

Inside 2DPI_LR12PUMP5MG_2572.csv, SimBA finds a single unique video inthe Media file column. Next, it checks what FPS you annotated that video at in the FPS column. However, in this column, SimBA finds something like 10 values separated by ;, like in screengrab below. So the code gets confused, which FPS should it use for the video?

[sronilsson]

And a second issue: I find several NA in the Image index column. THis should be a value representing the frame index when the behavior started/ended, but a bunch of them are missing, do you know why?

I have seen them missing before in the other thread, but that was events that the annotator had forgot to annotate as STOP before the video ended, and BORIS inserted them automatically. In these files however, they seem to be randomly occurring.

[GKNM995]

Those were missing because I forgot to stop a behavior when I started another which I designated as mutually exclusive. I revised the observations in BORIS, and have selected a single FPS for the video. I am using Simba 2.1.7. I updated like you said to do in the other thread
2DPI_LR12PUMP5MG_2572.csv

[sronilsson]

Ah and if you try to import that BORIS data into SimBA, does it still fail?

[GKNM995]

Nope. The new file works!