Neural Relation Extraction(NRE)

This is a forked version, which updates the code for compatibility of Python 3 and Tensorflow 1.x

Neural relation extraction aims to extract relations from plain text with neural models, which has been the state-of-the-art methods for relation extraction. In this project, we provide our implementations of a word-level and sentence-level combined Bidirectional GRU network (BGRU+2ATT).

We come up with the idea from the paper "Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification" [Zhou et al.,2016] and the paper "Neural Relation Extraction with Selective Attention over Instances" [Lin et al.,2016]. And we compare our results with PCNN+ATT [Lin et al.,2016] on the same dataset. #Evaluation Results P@N comparison between PCNN+ATT and our method (BGRU+2ATT):

Precision/Recall curve of our method (BGRU+2ATT) compared to others':

Data

We use the same dataset(NYT10) as in [Lin et al.,2016]. And we provide it in origin_data/ directory. NYT10 is originally released by the paper "Sebastian Riedel, Limin Yao, and Andrew McCallum. Modeling relations and their mentions without labeled text."

Pre-Trained Word Vectors are learned from New York Times Annotated Corpus (LDC Data LDC2008T19), which should be obtained from LDC (https://catalog.ldc.upenn.edu/LDC2008T19). And we provide it also in the origin_data/ directory.

To run our code, the dataset should be put in the folder origin_data/ using the following format, containing four files

• train.txt: training file, format (fb_mid_e1, fb_mid_e2, e1_name, e2_name, relation, sentence).
• test.txt: test file, same format as train.txt.
• relation2id.txt: all relations and corresponding ids, one per line.
• vec.txt: the pre-train word embedding file

Before you train your model, you need to type the following command:
python3 initial.py
to transform the original data into .npy files for the input of the network. The .npy files will be saved in data/ directory.

Codes

The source codes are in the current main directory. network.py contains the whole neural network's defination.

Requirements

• Python (>=3.5)
• TensorFlow (>=r1.0)
• scikit-learn (>=0.18)
• Matplotlib (>=2.0.0)
• itchat (optional)

Train

Unable to reproduce the training with small_train. (thunlp#7)

Probably using full sized training data will work?

For training, you need to type the following command:
python3 train_GRU.py
The training model file will be saved in folder model/

You can lauch the tensorboard to see the softmax_loss, l2_loss and final_loss curve by typing the following command: tensorboard --logdir=./train_loss

Test

currently due to TensorFlow version upgrade, the pre-trained model ATT_GRU_model-10900 which relies on tf 0.x becomes unusable.

Fortunately, we implemented an temporary compatibility switch inside test_gru.py (USE_LEGACY = True)

For testing, you need to run the test_GRU.py to get all results on test dataset. BUT before you run it, you should change the pathname and modeliters you want to perform testing on in the test_GRU.py. We have add 'ATTENTION' to the code in test_GRU.py where you have to change before you test your own models.

As an example, we provide our best model in the model/ directory. You just need to type the following command:
python3 test_GRU.py
The testing results will be printed(mainly the P@N results and the area of PR curve) and the all results on test dataset will be saved in out/ directory with the prefix "sample"

To draw the PR curve for the sample model, you just need to type the following command:
python3 plot_pr.py
The PR curve will be saved as .png in current directory. If you want to plot the PR curve for your own model, you just need to change the modeliters in the plot_pr.py where we annotated 'ATTENTION'.

Link to WeChat (optional)

We use the python package of WeChat, ItChat (https://github.com/littlecodersh/ItChat) to send training details and testing results to your wechat number. If you want to use the function, just change the codes in train_GRU.py or test_GRU.py: itchat_run=False to itchat_run=True. And also you have to change the string FLAGS.wechat_name to the wechat number you want to use. (default is filehelper)

Then at the start of your training or testing, you have to scan the QR code printed in the command line to login your own wechat number. And then you can receive messages about the training/testing results.

One example of testing results:

Update Logs

2017.03.20

• Fix the bugs of multi-label. (In training/testing dataset, some entity pairs have multi-relation labels).
• Seperate the network definition codes from train/test operation codes.
• Add the function of tensorboard to show loss curve online.
• (Optional) Use ItChat package to send training details and testing results to wechat.

Reference

[Zeng et al., 2014] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. Relation classification via convolutional deep neural network. In Proceedings of COLING.

[Zeng et al.,2015] Daojian Zeng,Kang Liu,Yubo Chen,and Jun Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of EMNLP.

[Zhou et al.,2016] Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification[C] Meeting of the Association for Computational Linguistics. 2016:207-212.

[Lin et al., 2016] Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Neural Relation Extraction with Selective Attention over Instances. In Proceedings of ACL.