Why zero_init_last_bn?
#1076
Answered
by
rwightman
OverLordGoldDragon
asked this question in
Q&A
Why `zero_init_last_bn`?
#1076
-
|
Throws out main block's outputs in early training - is there a citation? |
Beta Was this translation helpful? Give feedback.
Answered by
rwightman
Jan 6, 2022
Replies: 1 comment 1 reply
-
|
@OverLordGoldDragon Section 5.3 of https://arxiv.org/abs/1706.02677 ... but lots of other mentions of it in papers or code |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thanks - is it explained anywhere? |
Beta Was this translation helpful? Give feedback.
Answer selected by
OverLordGoldDragon
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@OverLordGoldDragon Section 5.3 of https://arxiv.org/abs/1706.02677 ... but lots of other mentions of it in papers or code