Skip to content

Latest commit

 

History

History
71 lines (58 loc) · 4.55 KB

README.md

File metadata and controls

71 lines (58 loc) · 4.55 KB

MM-SADA_Domain_Adaptation_Splits

This repository contains the annotations for the domain adaptation dataset used in the paper Multi-Modal Domain Adaptation for Fine-Grained Action Recognition.

BibTeX

If this repository was utilised, please cite:

@InProceedings{munro20multi,
author = "Munro, Jonathan and Damen, Dima",
title = "{M}ulti-modal {D}omain {A}daptation for {F}ine-grained {A}ction {R}ecognition",
booktitle = "Computer Vision and Pattern Recognition (CVPR)",
year = "2020"
}

Annotations

Three domains are defined as D1, D2 and D3 from individual kitchens in the EPIC Kitchens dataset (P08, P01 and P22 respectively).

D*_train.pkl - Contains action segments for either a labelled source or unlabelled target domain. For an unlabelled target domain only video id's and timestamps should be used.

D*_test.pkl - Contains action segments for evaluation only.

verb_class is a numeric id used as the ground truth action prediction in this work.

Each pickle file contains a pandas.DataFrame with 10 columns:

Column Name Type Example Description
uid int 12917 Unique ID of the segment.
video_id string P08_01 Video the segment is in.
narration string close fridge English description of the action provided by the participant.
start_timestamp string 00:00:07.29 Start time in HH:mm:ss.SSS of the action.
stop_timestamp string 00:00:08.95 End time in HH:mm:ss.SSS of the action.
start_frame int 437 Start frame of the action (WARNING only for RGB frames extracted as detailed in Video Information).
stop_frame int 537 End frame of the action (WARNING only for RGB frames extracted as detailed in Video Information).
participant_id string P08 ID of the participant.
verb string close Parsed verb from the narration.
verb_class int 3 Numeric ID of the parsed verb's class.

Flow modality start and stop times

Optical Flow was calcuated with a stride=2 in EPIC Kitchens, therefore the start and stop frames for the Flow modality are (start_frame/2, stop_frame/2).

Downloading Frames

download_script.sh will download the frames from the relevent participants P08, P02 and P22 into the below directory structure. Unless an argument is specfied, the directory structure will be created in "$HOME/Downloads/EPIC_KITCHENS_UDA".

~/Downloads/EPIC_KITCHENS_UDA/
└── frames_rgb_flow
    ├── rgb
    │   ├── test
    │   │   ├── D1
    │   │   │   ├── P08_10.tar
    │   │   │   ├── ...
    │   │   ├── D2
    │   │   │   ├── P01_11.tar
    │   │   │   └── ...
    │   │   └── D3
    │   │       ├── P22_01.tar
    │   │       └── ...
    │   └── train
    │       ├── D1
    │       │   └── ...
    │       ├── D2
    │       │   └── ...
    │       └── D3
    │           └── ...
    └── flow
        ├── ... same file structure as rgb