You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for sharing this very interesting work. I had a question about how the bidirectional attention mask is implemented here
Based on this implementation, it seems like even the padding tokens in a batch will get unmasked, whereas they should remain masked in both unidirectional and bidirectional attention. Is my understanding correct?
The text was updated successfully, but these errors were encountered:
Hi,
Thanks for sharing this very interesting work. I had a question about how the bidirectional attention mask is implemented here
Based on this implementation, it seems like even the padding tokens in a batch will get unmasked, whereas they should remain masked in both unidirectional and bidirectional attention. Is my understanding correct?
The text was updated successfully, but these errors were encountered: