Skip to content

Commit

Permalink
Merge commit '988c987e50d426415d68ec9476e96fdcedf3e433' into issue/00…
Browse files Browse the repository at this point in the history
…8-coarser-data
  • Loading branch information
peterdudfield committed Jun 20, 2024
2 parents 6cf19ca + 988c987 commit 5e942d3
Showing 1 changed file with 5 additions and 6 deletions.
11 changes: 5 additions & 6 deletions experiments/india/008_coarse4/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ https://wandb.ai/openclimatefix/india/runs/sk5ek3pk


### early stopping on MAE/val
Changing from quantile_loss to MAE/val to stop early on. This should mean the model does more training epochs, and the results we are interested int.
Changing from quantile_loss to MAE/val to stop early on. This should mean the model does more training epochs, and the results we are interested int.
https://wandb.ai/openclimatefix/india/runs/a5nkkzj6


Expand All @@ -48,11 +48,11 @@ Need to check that 32 batches were being passed through.
## Results

The coarsening data does seem to improve the experiments results in the first 10 hours of the forecast.
DA forecast looks very similar. Note the 0 hour forecast has a large amount of variation.
DA forecast looks very similar. Note the 0 hour forecast has a large amount of variation.



Still spike results in the individual runs.
Still spike results in the individual runs.

| Timestep | b8_s1 MAE % | b8_s2 MAE % | b32_s3 MAE % | epochs MAE % | small MAE % | mae/val MAE % | old MAE % |
| --- | --- | --- | --- | --- | --- | --- | --- |
Expand All @@ -73,6 +73,5 @@ Still spike results in the individual runs.

![](mae_step_smooth.png "mae_steps")

I think its worth noting the model traing MAE is around `3`% and the validation MAE is about `7`%, so there is good reason to believe that the model is over fit to the trianing set.
It would be good to plot some of the trainin examples, to see if they are less spiky.

I think its worth noting the model traing MAE is around `3`% and the validation MAE is about `7`%, so there is good reason to believe that the model is over fit to the trianing set.
It would be good to plot some of the trainin examples, to see if they are less spiky.

0 comments on commit 5e942d3

Please sign in to comment.