Loss Functions for Pose-guided Person Image Generation

Code for the conference paper Loss Function for Person Image Generation in BMVC2020 and the submitted journal paper A Comprehensive Study of Loss Functions for Pose Guided Person Image Generation.

Get Start

Requirements

• pytorch
• torchvision
• numpy
• scipy
• scikit-image
• pillow
• pandas
• tqdm
• dominate

Installation

Clone this repo:

git clone https://github.com/shyern/Pose-Transfer-pSSIM.git
cd Pose-Transfer-pSSIM

Dataset

We build the market-1501 dataset and DeepFashion dataset following PATN. The details for building these two datasets are shown here.

Market-1501

• Download the Market-1501 dataset from here. Rename bounding_box_train and bounding_box_test as train and test, and put them under the ./datasets/market_data directory
• Download train/test key points annotations from Google Drive including market-pairs-train.csv, market-pairs-test.csv, market-annotation-train.csv, market-annotation-train.csv. Put these files under the ./datasets/market_data directory.
• Generate the pose heatmaps. Launch

python tool/generate_pose_map_market.py

DeepFashion

• Download the DeepFashion Dataset from In-shop Clothes Retrieval Benchmark.
• Unzip img.zip. You will need to ask for password from the dataset maintainers. Then put the obtained folder img under the ./datasets/fashion_data directory.
• Download train/test key points annotations and the dataset list from Google Drive including fashion-pairs-train.csv, fashion-pairs-test.csv, fashion-annotation-train.csv, fashion-annotation-train.csv, train.lst, test.lst. Put these files under the ./datasets/fashion_data directory.
• Run the following code to split the train/test dataset.

python tool/generate_fashion_datasets.py

• Generate the pose heatmaps. Launch

python tool/generate_pose_map_fashion.py

Training

Training with Perceptual loss, Adversarial loss, and part-based SSIM loss.

• Market-1501

bash ./script/train_market.sh

• DeepFashion

bash ./script/train_fashion.sh

Testing

Download the trained weights from Fashion, Market. Put the obtained checkpoints under ./checkpoints_fashion and ./checkpoints_market respectively.

• Market-1501

bash ./script/test_market.sh

• DeepFashion

bash ./script/test_fashion.sh

Evaluation

We adopt SSIM, IS, mask-SSIM, mask-IS, DS, and pSSIM for evaluation of Market-1501. SSIM, IS, DS, pSSIM for DeepFashion.

SSIM, IS, mask-SSIM, mask-IS, DS

Please follow PATN to acquire SSIM, IS, mask-SSIM, amask-IS, and DS.

Citation

Acknowledgement

We build our project base on Pose-Transfer.