Philip's blog

[p208p2002]

https://blog.philip-huang.tech/?page=blog_nlp_pytorch_self-attention

- tags: nlp pytorch self-attention - date: 2021/03/24

輸入準備

我們準備了兩個句子來進行這次實驗

sentences = ['helo attention','have a nice day']

一開始先建立詞表與對應的單詞one-hot encoding

vocabs = ' '.join(sentences).split()
vocabs = list(set(vocabs))
one_hots = []
vocab_dict = {}
for i,vocab in enumerate(vocabs):
    one_hots.append([0]*len(vocabs))
    one_hots[i][i]=1
for i,(vocab,one_hot) in enumerate(zip(vocabs,one_hots)):
    vocab_dict[vocab] = one_hot
pri