Baseline models for VQuAnDa

VQuAnDa Repository

• https://github.com/AskNowQA/VQUANDA

Introduction

The repository contains sequence to sequence models for evaluating VQuAnDa.

The models we use are based on:

• RNN Seq2seq:
  • Neural Machine Translation by Jointly Learning to Align and Translate
  • Effective Approaches to Attention-based Neural Machine Translation
• CNN Seq2seq:
  • Convolutional Sequence to Sequence Learning
• Transformer Seq2seq:
  • Attention Is All You Need

Experiments

Requirements and Setup

Python version >= 3.7

PyTorch version >= 1.3.0

# first download the dataset
git clone https://github.com/AskNowQA/VQUANDA.git
cd VQUANDA
# inside the dataset download the baseline models
git clone https://github.com/endrikacupaj/VQUANDA-Baseline-Models.git
cd VQUANDA-Baseline-Models
pip install -r requirements.txt

Run models

RNN-1 with Bahdanau attention:

python run.py --model rnn --attention bahdanau
# with covered entities
python run.py --model rnn --attention bahdanau --cover_entities
# query as input
python run.py --model rnn --attention bahdanau --input query
# query as input with covered entities
python run.py --model rnn --attention bahdanau --input query --cover_entities

RNN-2 with Luong attention:

python run.py --model rnn --attention luong
# with covered entities
python run.py --model rnn --attention luong --cover_entities
# query as input
python run.py --model rnn --attention luong --input query
# query as input with covered entities
python run.py --model rnn --attention luong --input query --cover_entities

CNN:

python run.py --model cnn
# with covered entities
python run.py --model cnn --cover_entities
# query as input
python run.py --model cnn --input query
# query as input with covered entities
python run.py --model cnn --input query --cover_entities

Transformer:

python run.py --model transformer
# with covered entities
python run.py --model transformer --cover_entities
# query as input
python run.py --model transformer --input query
# query as input with covered entities
python run.py --model transformer --input query --cover_entities

Results

License

The repository is under MIT License.

References

Links we considered for implementing the baseline models:

• https://github.com/pytorch/fairseq
• https://github.com/IBM/pytorch-seq2seq
• https://github.com/bentrevett/pytorch-seq2seq
• http://nlp.seas.harvard.edu/2018/04/03/attention.html
• https://pytorch.org/tutorials/