Skip to content

Latest commit

 

History

History
11 lines (6 loc) · 1.17 KB

2007.58.md

File metadata and controls

11 lines (6 loc) · 1.17 KB

Learning to tag OOV tokens by integrating contextual representation and background knowledge, He et al., 2020

Paper, Tags: #nlp

We propose a novel knowledge-enhanced slot tagging model to integrate contextual representation of input text and the large-scale lexical background knowledge. We use multilevel graph attention to explicitly model lexical relations.

Slot tagging is identifying lunch as meal_description type. Previous state-of-the-art context-aware models only learn contextual information based on a multi-layer BiLSTM encoder and self-attention layer.

We aim to leverage both linguistic regularities covered by deep LMs and high-quality knowledge derived from curated KBs. We propose a BERT-based model for slot tagging, and we append a knowledge integration mechanism

Our method makes a notable difference in a scenario where samples are linguistically diverse, and large vocab exists. The results also demonstrate that incorporating external knowledge does not bring in much noise since we use a knowledge sentinel for the better tradeoff between the impact of background knowledge and information from the context.