You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think I misunderstood the purpose of the "Fallback train data type". This isnt surprising, since a google search for OneTrainer "Fallback train data type" returns NO official documentation on the value.
My tentative understanding now, is that it is usually best to leave it at "float32", even if your main data type is bf16?
If so, it would be really nice to explicitly say in the mouseover, "it is usually best to leave this as float32" Although I'm unclear why it would ever be good to set it otherwise?
What would you like to see as a solution?
Clearer mouse-over docs, or better still explicit documentation in the wiki about it.
Have you considered alternatives? List them here.
No response
The text was updated successfully, but these errors were encountered:
Describe your use-case.
I think I misunderstood the purpose of the "Fallback train data type". This isnt surprising, since a google search for
OneTrainer "Fallback train data type"
returns NO official documentation on the value.My tentative understanding now, is that it is usually best to leave it at "float32", even if your main data type is bf16?
If so, it would be really nice to explicitly say in the mouseover, "it is usually best to leave this as float32" Although I'm unclear why it would ever be good to set it otherwise?
What would you like to see as a solution?
Clearer mouse-over docs, or better still explicit documentation in the wiki about it.
Have you considered alternatives? List them here.
No response
The text was updated successfully, but these errors were encountered: