Can performance on a small dataset become worse after fine-tuning? #111
-
|
After fine-tuning on a small data set, the performance on this data set is worse. Is this normal? What are the possible reasons? |
Beta Was this translation helpful? Give feedback.
Answered by
abdulfatir
May 28, 2024
Replies: 1 comment
-
|
Hi @TPF2017! Fine-tuning performance depends on a variety of factors, including hyperparameters such as the learning rate and training steps. If your dataset is small, it is possible that a specific selection of these hyperparameters (e.g., high learning rate or large number of step) worsens the performance. There are other potential causes such as distribution shifts in the time series. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
lostella
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi @TPF2017!
Fine-tuning performance depends on a variety of factors, including hyperparameters such as the learning rate and training steps. If your dataset is small, it is possible that a specific selection of these hyperparameters (e.g., high learning rate or large number of step) worsens the performance. There are other potential causes such as distribution shifts in the time series.