| Model | EM | F1 | code | paper |
|---|---|---|---|---|
| Match-LSTM with Ans-Ptr | 60.5 | 70.7 | ICLR17 | |
| Globally Normalized Reader | 66.6 | 75.0 | Tensorflow | EMNLP17 |
| MPCM | 65.5 | 75.1 | arxiv-16 | |
| RASOR | 67.4 | 75.5 | arxiv-16 | |
| DCN | 66.2 | 75.9 | ICLR17 | |
| BiDAF | 68.0 | 77.3 | Tensorflow | ICLR17 |
| SEDT | 68.5 | 78.0 | EMNLP17 | |
| Document Reader | 69.5 | 78.8 | Pytorch | ACL17 |
| FastQAExt | 70.8 | 78.9 | CoNLL17 | |
| ReasoNet | 69.1 | 78.9 | CNTK | SIGKDD17 |
| RNET1 | 71.1 | 79.5 | ACL17 | |
| Smartnet | 71.4 | 80.2 | arxiv-17 | |
| RNET2 | 72.3 | 80.6 | MS share | |
| BiDAF + selfatt | 72.1 | 81.1 | Tensorflow | arxiv-17 |
| MneReader | 71.8 | 81.2 | arxiv-17 | |
| PhaseCond | 72.1 | 81.4 | arxiv-17 | |
| MEMEN | 75.4 | 82.7 | arxiv-17 | |
| QANet | 73.6 | 82.7 | ICLR18 | |
| FusionNet | 76.0 | 83.9 | Pytorch | ICLR18 |
| RaSoR + TR + LM | 77.6 | 84.2 | arxiv-17 | |
| MneReaderV2 | 79.5 | 86.6 | IJCAI18 |
| Repository | RNET | MneReader | QANet | Document Reader | FusionNet |
|---|---|---|---|---|---|
| ML Results in paper | 71.1/79.5 | 71.8/81.2 | 73.6/82.7 | 69.5/78.8 | 76.0/83.9 |
| Pytorch | X | X | -3.6/-3.3 | X | X |
| Pytorch | -0.9/-0.3 | +0.5/+0.2 | X | -0.1/-0.2 | X |
| TensorFlow | -/- | X | X | X | X |
- SQuAD / most popular, EMNLP best resource paper
- MARCO
- NewsQA
- Quasar
- RACE
- CNN/Daily Mail
- DuReader / multi document for CQA
- NarrativeQA / QA within a book or story, and supporting a summary version (SQuAD like)
- SAN
1.join hidden states of previous reasoning steps - FUSIONNET
1.a fusion version of other networks - ELMo
1.pretrained word embedding by LM, which is somewhat similar with Tricks [Learned in Translation ...] - R-Net
1.self att;
2.soft att - Reinforced Mnemonic Reader
1.RL for F1;
2.feature rich;
3.memory answer pointer - MEMEN
- ReasoNet
- SEDT
- BiDAF
1.(widely used) coopreation for question and contexts - BiDAF + self attention
1.self attention with simplified BiDAF;
2.new data;
3.vairous comparision about diff functions - jNet
- Multi-Perspective Matching
- Dynamic Coattention Networks
- Document Reader
- FastQAExt
- RaSoR
- Fine-Grained Gating
- Dynamic Chunk Reader
- Conductor-net
- RaSoR + TR + LM
(have report results on adversarial SQuAD) - Smartnet
- Globally Normalized Reader / code
1.Three stage model include sentence selection, start prediction, end prediction
2.Data augmentation (as the model runs faster, the authors replace the NEs to generate more data)
- Towards Human-level Machine Reading Comprehension: Reasoning and Inference with Multiple Strategies
datasets: RACE
multi step reasoning by update a Memory
- A Comparative Study of Word Embeddings for Reading Comprehension
- Learned in Translation: Contextualized Word Vectors
- R3 : Reinforced Reader-Ranker for Open-Domain Question Answering