Incremental Audio-Visual Fusion for Person Recognition in Earthquake Scene

Dataset & Features

1. Datasets：The proposed AVEPR and AVELPR datasets can be download from Google Drive link
2. Features
   extract frame-level image from videos:

python extract_frames.py

extract 2D frame-level features:

python extract_3D_feat.py

extract 3D snippet-level features:

python extract_rgb_feat.py

extract audio samples from videos:

python extract_audio.py

extract audio features:

python wave audio_feature_extractor.py

(You can also extract audio features with PANNs)

Requirements

conda env create -f IAVF.yml

Incremental Audio-Visual Fusion

Training

python main.py --gpu 0 --K 1 --dataset AVE --mode train --batch-size 32 --epochs 10 

Testing

python main.py --gpu 0 --K 1 --dataset AVE --mode test --batch-size 32 --epochs 10