Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic Evaluation #23

Merged
merged 52 commits into from
Dec 19, 2023
Merged

Basic Evaluation #23

merged 52 commits into from
Dec 19, 2023

Conversation

peterdudfield
Copy link
Contributor

@peterdudfield peterdudfield commented Dec 12, 2023

Pull Request

Description

Add evaluations model which pull data from

  • ICON and PV data, all from hugging face
  • Make data pull using multi processing
  • add to documentation, on how to run evaluation

Fixes #2

Screenshot 2023-12-19 at 12 27 52

How Has This Been Tested?

Added tests + ran on a small tests set

  • Yes

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

@peterdudfield peterdudfield mentioned this pull request Dec 18, 2023
@peterdudfield peterdudfield marked this pull request as ready for review December 19, 2023 12:27
README.md Outdated Show resolved Hide resolved
| 0 | 0.202 +- 0.03 | 6.2 |
| 1 | 0.211 +- 0.03 | 6.4 |
| 2 | 0.216 +- 0.03 | 6.5 |
| 3 - 4 | 0.211 +- 0.02 |6.3 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very odd the 0th hour horizon is the same as the 24-48 hour horizon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea it is a bit. I suspect this might be due to the model being trained expecting live pv data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ive added this to the Readme.md under the evaluation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that could explain it. Is the current model trained without any pv_dropout at all?


site = PVSite(
latitude=pv_row["latitude"],
longitude=pv_row["longitude"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good next task, that i could do could be add in the tilt and orientation code

Copy link
Contributor Author

@peterdudfield peterdudfield Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I agree, we have to do that for the trained model, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we would need to.

@peterdudfield peterdudfield merged commit 53e21d2 into main Dec 19, 2023
2 checks passed
@peterdudfield peterdudfield deleted the issue/make-testset branch December 19, 2023 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Evaluate model on 2022 data
2 participants