Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

想请教一下attention2d里的temperature是干嘛的？ #4

Open
rainylt opened this issue Sep 23, 2020 · 2 comments
Open

想请教一下attention2d里的temperature是干嘛的？ #4

rainylt opened this issue Sep 23, 2020 · 2 comments

Comments

@rainylt
Copy link

rainylt commented Sep 23, 2020

一直限制attention的输出大小，这是warmup的手法吗？为什么是加在attention后面而不是卷积后面呢？
@kaijieshi7
Copy link
Owner

这和知识蒸馏里面的内容相关，原文有提到为什么用tempeature，知识蒸馏可以看《Distilling the Knowledge in a Neural Network》这篇文章。

@fxYOLO
Copy link

fxYOLO commented Nov 2, 2023

您好，请问应该怎么让这个程序跑起来呀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kaijieshi7 @rainylt @fxYOLO