Learning to tag OOV tokens by integrating contextual representation and background knowledge, He et al., 2020
Paper, Tags: #nlp
We propose a novel knowledge-enhanced slot tagging model to integrate contextual representation of input text and the large-scale lexical background knowledge. We use multilevel graph attention to explicitly model lexical relations.
Slot tagging is identifying lunch as meal_description type. Previous state-of-the-art context-aware models only learn contextual information based on a multi-layer BiLSTM encoder and self-attention layer.
We aim to leverage both linguistic regularities covered by deep LMs and high-quality knowledge derived from curated KBs. We propose a BERT-based model for slot tagging, and we append a knowledge integration mechanism
Our method makes a notable difference in a scenario where samples are linguistically diverse, and large vocab exists. The results also demonstrate that incorporating external knowledge does not bring in much noise since we use a knowledge sentinel for the better tradeoff between the impact of background knowledge and information from the context.